Looks like Hagrid was correct, I am rising up in the seat in order to turn around to see when I'm backing up. So the seat switch works while I'm moving, but not when I'm stopped. When stopped, I can step off the tractor and it will keep running, is that normal?