Random thoughts...
Some people see the hobby as quirky and interesting. Other people get downright weirded out by it. They have a mistrust of anyone who as an adult still likes "childish things". Still other people are just jealous that we can enjoy something that might embarrass them.
People might think if you like kids' toys, maybe you also like kids too much.
There's an episode of Law & Order where they are trying to accuse someone of being a pedophile. They tell him something like "We know you're a pedophile... we already found your Transformers collection". I kid you not. I don't know why more TF fans don't talk about this scene more. It's absurd and offensive, but it shows a glimpse of how others might see us.
Other people are simply afraid to show their childish side. Always trying to prove they are grown up. When they see us embracing toys and "children's fiction", they feel threatened the way someone might feel around gay people if they have homosexual tendencies. It's really the same thing. If you're not comfortable with your own adulthood, you might respond by condescending to anyone who is comfortable with theirs.
CS Lewis said it best though. He pointed out that there's a stage in life where trying NOT to be childish is natural. Adolescents are supposed to act like adults as part of the process of growing up. When they still act juvenile compared to their peers, they feel awkward and might be ridiculed. It helps give them that nudge toward adulthood and it is natural. But then later in life if you're still hanging on to that tendency to always prove you're a grown-up, that in itself is a sign of arrested development. So according to Lewis, anyone ridiculing you for playing with toys as an adult, is not fully matured.
