Do you think I am "less" of a Harry Potter fan for not having been there from the beginning?

I don't think that not being there from the beginning means you're any less of a Harry Potter fan! I myself have watched the films since they came out, but didn't actually start reading the books until a few months ago. The fact that you like it so much and that it has changed your life is enough indication that you are a very big harry potter fan, a bigger fan than many people who grew up with the series.