Huh. Maybe you're supposed to imagine you're manipulating the object, instead of manipulating the camera?

I always get thrown for a loop (so to speak) when I use something that uses the flight-simulator-style "move the mouse/joystick up in order to look down, but move it right in order to look right" arrangement.