I'm starting to get annoyed?

Get over it. Sorry, that sounds rude but you'll HAVE to get over it. Most people of age knows what autism is. Most people just don't know how to detect it. If you tell someone that he is autistic, then they'll look at him accepting his actions better because people know that people with autism acts a certain way. He might just look normal. If he looks normal then starts screaming, people are going to think that he is just a normal person screaming and acting crazy.