Hi tom,

 I have listened to the audio and can't hear 'nous'. 

You couldn't use 'nous' here with the 'on' but you could say-

Nous avons passé le samedi à nous baigner.

Bonne Continuation!