Do you have a voltmeter? This will tell you if you have sufficient voltage available. It sounds like a problem with voltage drop or insufficient voltage.
Test at the battery, battery terminal, and solenoid terminal when activating the starter. Then test for voltage drop by connecting the leads...