Sounds to me like you're probably not loading the trailer properly.
Bumper (hitch type) pull trailers will handle horribly if the weight bias is not on the tongue. In other words, more weight needs to be in front of the axle on the trailer. If not it will sway and pull terribly. To judge this when you load the tractor, pull it on the trailer till it levels out, then continue to move forward 'till the rear of the tow vehicle starts to sag some and you're sure there is weight on the hitch.