As I understand it, if you use être you are describing a "state of being". So to say "Je suis chaud" could convey a sexual connotation. Similarly with "Je suis froid". That is why in the company of native french speakers you may sense that they seem a bit embarrassed.