What if you pull the fuse to the headlights? What else is on the same circuit?
And the silly question what harm are they doing being turned on?
Turn'em off with the switch. If that doesn't work then the switch may have a short and you can either take it apart and try to do a refurb on it or replace it.