This is coming from left field, but are you maybe wiggling a power wire from the battery clamp when you connect the charger clips? Have you tried starting it with the charger connected but turned off/unplugged?This is very weird indeed. The battery is pretty new and shows 12.8V. My charger is 30amp and runs at ~14 volts. The battery has no trouble running the starter alone. However, it will only start if I have the charger on and it starts right up, on the first crank. I am thinking that the slightly higher voltage 14 vs. 12.8 is the key
You wouldn't happen to have a schematic? Since this is a stationary installation, it might be significantly different from a tractor.