Seems kinda weird, and potentially a lot of people will find themselves in a similar position to you, and get weirdly turned on when they smell something baking.

My sympathies for your gym session, not sure what the answer would be in that situation!