I'd be going out there just after dark to let 'em "Have a drink on me". A little 97 octane drink, perhaps 3-4 ounces right in the entry hole. Be careful, though, because if they're in a stump hole that's covered with debris, they may have several entrances. Need to put an eyeball on the normal traffic first.
I'm not allergic to them at all, but the pheromones in the sting are pretty much my Jekyl/Hyde drug. And I didn't say they don't hurt, they do indeed. I go into a rage, and my goal is to send every one I can find back to Hell where they come from. That's why they build in the ground, IMO, to get that little bit closer back to Lucifer what sent 'em to this side.
I don't mind the big black/white hornets, because their nest is visible, and they're pretty easilly avoided. I'll even excuse red wasps in a barn that isn't near human interface with barn. But I ain't gonna tolerate them building close to my house, and ANY bug that builds in the ground to ambush me cannot stay here (except ground dwelling bumblebees). I try not to kill pollinators, but I have yet to find ANY reason that would dissuade me from wiping yellow jackets off the planet. They're one of the most aggressive insects I've ever seen, and have the punch to back it up. One worker hits me, payback is a b**ch.