According to Fear Of, pogonophobia can be caused by a negative or traumatic experience involving a man with a beard. That even includes the 9/11 attacks in the U.S., in which the attackers were “a bearded group of religious fanatics.”

Fear Of also says beards can be associated with being unclean — being homeless, sick, or dealing with hard times, negative life situations in which shaving could go by the wayside. Stereotypes about bearded men include them perceived as not being trustworthy, per Fear Of.

Psychology Today says that a lot of evolutionary theorists think that men evolved to grow beards as a function of trying to prove their “dominance, masculinity, and aggression.” To illustrate, they reported on an experiment done in Australia in which researchers showed hundreds of volunteers photos of bearded men looking happy or angry, and later looking sad. Then they compared those reactions to the same men making the same faces while clean-shaven. Turns out, bearded dudes making angry faces looked angrier than non-bearded dudes making angry faces, according to the results of the polls, and it’s likely most of them didn’t even have pogonophobia.

Still, the results of that poll though showed that perceptions of beards aren’t all bad. While most people thought men with beards looked more threatening when they were angry, they also said they looked friendlier and “more helpful” than the clean-shaven guys when they smiled or looked happy, according to Psychology Today.