I feel like wasps seem more interested in both sweaty humans and our food compared to bees, so they get in your face in another way than bees do.
Our southern red brick wall houses a lot of red mason bees every year and even though they live just next to the porch they leave us be. I’ve had a tired bee take a break on my sun-warm shoulder once or twice, but that’s pretty much it.