Hi Karen,
there's no s because congé is not an adjective but a noun.
Jours de congé -- days of vacation.
If you said instaid "jours congés" it would be an adjective and hence also pick up the s. The difference lying in the preposition "de".
-- Chris (not a native speaker).