I have a rehomed Grey, and went through the same thing. In fact, I'm impressed he warmed up to you so fast. It was at least 3 months before we could handle Rosie. About that time she started picking up words, although she knew some from her old owners. Now she wont shut up

But like I said, it took months. Be patient and keep trying.
-Chris

Rosie (African Grey)

D'Artagnan--Dart for short (Jenday conure)

Lyra (Baby Budgie-hatched April 1st, 2012)