I can't relate to the rubbing issue with goggles, but I find that goggles fog easier on myself too, and for that reason I'd say go with a pair on glasses. It's definitely worth investing a bit more since you only get one pair of eyes and you sometimes don't really know what kind of protection you're getting with some of the unbranded shyte you find on eBay.
ESS is a good brand. I run ESS Crossbows with clear lenses, and have done for awhile (can't remember exactly how long, but it's certainly been over a year). I don't think they've ever fogged on me, and they're pretty comfy.
Also, glasses will probably feel better in conjunction with your lower face protection. With goggles, especially the kinda big style ones, it'll all begin to feel bulky and will be hella uncomfortable and probably make you sweat.
If fogging is really an issue, invest in some good mesh goggles. Just be aware that it's possible for small fragments of shattered BBs to pass through and potentially come into contact with your eye(s) at high velocity. I don't really know all that much about mesh eye pro though. From what I've heard 'Bitter End' is a good brand for them but I don't know of their availability here.