With a dynamo the regulator rectifies and controls the voltage output (except where they have a separate rectifier).
With an alternator, the regulator controls the excitation winding of the alternator to set the voltage generally at the output terminal of the alternator, or elsewhere if externally sensed. I don't have your circuit info, but if it is an alternator which i suspect it is, then the regulator is not limiting the excitation, so you may have a broken wire or bad connection on the connections at the back of the alternator, between the alt and reg, or grounding issue at the reg etc. Check to ensure you have a functioning lamp, and fuse in circuit as well. If it is externally sensed and it looses its sense then it will output high, which yours sounds like it is doing.