Lots of people have an issue with knocking the selector with the side of their hand while using it. If you are right handed hold the gun and and move around a bit and see if the right side touches the side of your hand when in semi.
If it does then you will keep knocking it to safe. It is what made me stop using one.
Oh yea I had only been shooting it and clicking back to saftey. I see what you mean easy to flip it back to safe by accident and thats without gloves on.
Have you found the selector issue yet.