I think it was unsaid but close to the surface that Isaac was disappointed with himself for a few reasons. First, because he reacted in a negative, possibly homophobic way. Second, because he’d not made it clear to Colin that he wasn’t homophobic, or that he’d made it clear that he was at least slightly homophobic. And lastly, all the lines that went with what he said - he wasn’t open and honest enough with Colin to make Colin comfortable enough about it.

His reaction was a bad one. He knew it.