hi imran,
i have grey too,and i face the same problem like you before.
my apartement is only 1 bedroom so i put momo

in the living room.but he need to sleep around 9 pm and we still active at that time.i dont have another choice except to move him to my big wardrobe(we have small room as a wardrobe).at first he really mad,growl,scream,and very very scared like wanna die.i ignored it,i talk to him softly and i put him sleep.for 1 week he still doing that but slowly(now i have him 2 months) he will whistle to ask me to put him in the wardrobe because he wants to sleep.
but in that first week,i realize his poop change.the vet said is because of stress.i think i push him too far.the other night i put him 1 hour earlier to the wardrobe so he can slowly see the room and he sure there is nothing to be scare of.slowly his poop change to normal again.
and the same like yours,momo scared of new object.now not anymore.i hold the object in front of him alot,play with him and just talk to him about this object.if u frequently move new object near him,sudenly he will understand that changing and new things are okay.now i can move momo to any place i want(even to my mother in law house),car,bathroom,bedroom without any problem.he become more flexible day by day.
but remember to not push him too far,surely u know him for how much he can tolerated the new object.do it slowly,confident,and..little bit force.u can change him,be sure
hope this will help you!!