It seems that ideally, to use a PAS with a VTA driver board, one should do both. First remove the 100K stepped attenuator (if present), and then remove the 510K fixed internal resistor from the PAS. This would make remaining internal load almost exactly 50K and keep the PAS happy. Does this sound right?
This puzzles me a bit too. Power amps don't have volume controls. A VTA (or any) power amp with a volume control wouldn't seem appropriate or needed when using a preamp since preamps have volume controls. I believe the intent of the VTA amp with attenuator is to run it straight off a source such as a CD player. Maybe a better solution for the VTA amp would be to have separate "preamp" input which bypasses the attenuator.
Rule of thumb is power amp input Z is a minimum of 10x the preamp's output Z. Isn't the PAS3x output Z 1k Ohm as per specs I've seen? I don't understand the 510k Ohm you mentioned?
Anyway, it sounds like you've solved your situation...