Le Réveillon ("du Noël" and "de la Saint-Sylvestre" or "du 31") is a long, drawn-out evening meal on either the 24th or the 31st of December. It can also mean the corresponding evening celebration (including the meal). If you only want to refer to the day, i.e., New Years Eve, you use "la Saint-Sylvestre"

Both "célébrer" and "fêter" mean "to celebrate". "Célébrer" has a bit more of a festive, ceremonious character. Note, that "to party" would be "faire la fête" and NOT "fêter".