Watch the BB and if you're getting distorted flight i.e. BB flying down or up significantly then it's the hop up (it won't necessarily be that exaggerated). If things don't seem to be lining up quite right then it'll be your sight. You could always just start with your sight and if you find you're pointing the gun at a stupid angle then it's the hop up.
Can't really see you getting the two confused though - once you've got your BB flying 'straight' etc then it's time to zero the sights.