There's no easy answer.
It'd have been easier if you'd noticed before he delivered, you didn't, that can't be changed.
On the question of buying from other dealers, they'll all service tractors bought elsewhere. Plenty of people buy a second hand tractor or tractors under other arrangements, and I believe all Kubota dealers are obligated to honour Kubota's warranty. Sure, he's probably gonna remember you now, and if you return his tractor then turn up with a new one, he'll know what happened. And you might be back of the queue for any service work you ask for. But let's be honest, there's good odds you're back of the queue already. Mostly you can service it yourself, and they don't break that often, so you're really just talking about buying fluids and filters, maybe new implements. No reason you'd get worse service than anyone else in doing that. So I wouldn't rule out going elsewhere, but I also agree I'd give him a chance to sort it first.
On other tractors (such as a green one). There's nothing wrong with green tractors. I wouldn't buy one, because I know Kubota. But that's just like being a Ford guy or a Chev guy. I could get over that if I had reason to, I currently don't, maybe you do. My feel is that the problem you have isn't a Kubota problem, the problem you have is this specific tractor, so I wouldn't say this means rush off to the JD dealer, again it's still worth time to try to fix it, and then your next step is a different Kubota dealer, not a different tractor.
Although in no way would I want to excuse the dealer - I think the dealership has done a bad thing here - I'd also think it through from his perspective. You ordered a tractor, he couldn't get a new one in because Kubota had a hold on deliveries. He (or his guys) did extra work to get you in one from another dealer, to make you happy. They worked through a bunch of stuff with you on implements, they spent time and effort to prep this tractor for you. Now you maybe don't want it, all that time, effort and what he may perceive as trying to meet your needs goes to waste.
For the dealership owner (who I think is your key guy here), he probably hasn't seen the tractor and for all he knows you're being picky. His guys haven't told him it was a shambles - they didn't notice or didn't care. He didn't supervise them well, that's his fault, but he doesn't know that yet because he hasn't been to look at it. So it's still an unknown quantity for him, I wouldn't write him off yet. It's possible he doesn't yet know/understand, he thinks you've got a couple rust spots and need a cut and polish on the hood, and you'll be fine. So he's thinking "OK, I've got a couple thousand in this sale already moving the tractor around and prepping it, but a bit of time with a polisher and we're all good, I'm keeping the sale."
The real question is what happens when he comes out and sees it. Is he going to make excuses, say that the tractor's fine and really just needs a tidy and you should be happy? He'll do that initially, because he's got money in it, the question is how hard he pushes.
If it were me I'd let him do that, and calmly work him through it. The tractor's in a state you might not buy if it was second hand. You definitely wouldn't buy it new that way. Cleaning it up won't address that, because as you say the plastics may be brittle, the rusted pins won't really just buff out, the rusted powder coat will just rust again if you buff it off. And there's real question how the thing was stored - was it in a salt environment? That's a lot of rust. Do you want a tractor in that state? Work him through it, and your statement that you may need to trade on a B2601 or some other model at a later date. Taking this tractor will directly impact your trade in value. Ask him if you kept it for a week then brought it to him second hand, what price would he give you for a low hour machine that's actually nearly 2 years old, and with that condition? Pretty sure it'd be nothing like new price.
It's possible he'll see that and say yep, I'll take that back, we'll put the order in, we'll get you a new one. Then he sorts it with Kubota or the dealer who gave it to him, and sure, he'll be out of pocket the prep expenses (he'll have to prep the new one). It's possible he'll not be willing to do that, and that's when I think you need to tell him you don't want it, and you'll be shopping elsewhere. In that situation, a green one is an option, so is an orange one from another dealer. It's not personal, it's that you'll be unhappy with that tractor as long as you own it.
He may offer you a discount to keep it. Realistically it's going to cost him $1-2K to take that tractor back and get you a new one. He might get some of that back from Kubota and he might not. Doesn't matter that it's his fault or not, that's the reality for him. So he might offer you $1K off, or similar. You need to have made your mind up what you do in that situation before it happens - don't be put in a spot where you have to decide without thinking first. Would you be happy with that tractor if it was $1K cheaper? I suggest not, so you need to be ready to say "no, I want a different one". You also need to know whether you're prepared to put any money in - he's been offering you deals on implements etc, are you prepared to give some of that up (whether buying from him, or with another dealer)? How much is it worth to you to have this sorted, in time and/or money? If he says "I can get you a new stock one, it'll be a month and some of the discount I gave you on this tractor I have to take back" - would that be OK? If it were me I'd probably give $500-$1,000 extra to be out of the situation and have the tractor I wanted, even though it's not really my fault.