I had the same thing and was told here, which was confirmed by mechanic, that it was "the center support bearing on the driveshaft is blown out (fairly common)".
The part is inexpensive but it takes a bit of labour to get to it. It cost me about $350 CAD before tax to get it done. I had other stuff done at the same time so I'm not exactly sure of the breakdown of the price but it was around that. Now it drives beautifully without hesitation.
Just a thought.