You'd think it would be the feeling that you are, yourself, going a bit crazy as you're surrounded daily by behaviors that are confusing your internal sense of what constitutes "normal." It's not quite that. It's also not the constant waft of bleach, nor the endless banging of filing-cabinet drawers in rhythmic patterns.
No, for my money, the really maddening thing about working with crazy people is the things you have to do to accommodate and/or work around the crazy person. Such as: inadvertently encountering undisposed personal hygiene products when one innocently heads to the loo; eating frozen carrots; putting personal belongings, trash cans, and other office paraphernalia into someone's else's precise yet arbitrary idea of just so; scurrying about behind partially closed doors because the person in question saw a stinging insect in a hallway (never mind that cracked doors allow flying insects, as do the louvered transoms above the doors).
That last one is just today, and we are all going about our day gingerly closing doors. Me, I can't so much as get anyone to make my deadlines, so I'm thinking I'm obviously going about things the wrong way.