I bet you that terrible breeder gavage-feeds his babies -no other reason for a baby to be afraid of the person he has seen every single day of his life and who brings only warm food and comfort to him! But I am glad you bought him because he needs somebody like you that is understanding and keeps on trying as you are doing.
Now, quakers scream for no good reason all the time. Lord knows mine does! She loves to lift the edge of the papers on the bottom trays of the cages and get under them (she also loves fighting me to change them

) and, even when she is in there all by herself or with her husband, she screams. Their loud screams is the number one reason why they are given up so be prepared for them because there will be screams all his life.
Why don't you try putting some branches on the outside of the cage so he can climb up and chew on them a bit?
And, yes, I think he is coming along very nicely taking into consideration his rough infancy, poor thing! And I think he is beginning to find comfort in your presence and this is the very first and most difficult step in his way to been bonded to you so keep on doing what you are doing.