If we look at it, "wearing everything but a shirt" is pretty vague and subjective: when do we judge a shirt could have been wear in the occasion?
And most importantly, there is already something else better to "cover" the situation : for female there are several layers of undressing, but for a male, there is no reason we're not directly on bare_pectorals. I see it's often ambiguous between the two, especially with things like capes.
The last situation it could work with is if it's almost topless_male, but in fact the pectorals themselves are still strategically hidden somehow (like bra does for girls)
Updated