I have the same experience but it makes me love these books even more. When I grew and re-read it, and notice how actually realistically non-ideal characters, how often Potter pisses you off with his teenager behavior, I start to enjoy it even more, and like those nuances and double-morale in a lot of cases.

Also, Snape arc about Lilly Potter hurts you way more when you read it as adult, and experienced already a feeling of deep love.