First sign of trouble on anything with regards to this type of sale is if everything is handled over e-mail with no real personal discussion. What kind of buyer does not personally get involved on the discussions and process. I mean, any used collector type of cars I have ever bought across North America over the years, I have:
1. Flew out to look at the car as a first step. If I test drove it? Great! If the owner only test drove it with me in the car? Fine as well because you get the same feedback anyway. If I felt concerned on some points and the owner allowed for a mechanic to look at it, at my cost? Great! If not, I rely on my own powers of deduction so no biggie.
2. Put the owner in direct contact with my credit union so that both parties are protected during the transaction. Ends up with both party's financial institutions talking directly and the exchange of money is between the each other's banks.
3. Once check, cashier's check, etc. has cleared, and clear title has been FED-EX to me. I have transport come and pick the car up or I fly out and drive it home. It is my car at that point and transport is all my own handling. With my own insurance taken care of prior.
I have never understood the rush, rush, rush aspect of used car sale transactions. If you follow the steps above, you get a pretty good understanding of the indivudal you are dealing with and if there are any issues on the surface. If you have the financial institutions of both parties involved, there is no risks of not being paid. Once you have bought the car, you insure it immediately so if the old owner destroys it, at least insurance will cover what you just paid for. You arrange your own shipping for your own piece of mind and the transport will have insurance during the move, if you choose the right one.
I just explain and we agree on the process right up front with I am dealing with. There is no reason they would not do this as it is risk free for all involved. If they want some crazy scheme, then I walk away right up front. I have never had to lay out a deposit for any of my cars upfront. Most times the owners just want it to go to a good home so the timing is not the issue.
As far as the test drive mandate, that is just BS. That is a kiddy thing that unknowledgable people feel they need to do to understand a vehicle that they know nothing about. I hate driving other people's cars, even dealer test drives. I feel uncomfortable. Sometimes people need to man up for their decisions. Used cars without a warranty are always a risk anyway.
You did the right thing. If someone did not come out to look at my car in person and let me qualify them as a buyer first, I would not give them the time of day. That is just common sense, to me anyway.