I would absolutely move to set up on him. Set the stand in the afternoon quietly and make certain the wind is favorable and go for it. I would just want to make sure I didn't screw up a bedding area. I have a similar situation where I hunt. One of my stands is in a position to observe..and deer do come by it. I could have shot 3 small bucks already. But there is not tree that I could set up in where the big boys usually come and go to their beds. And the current stand is really only about 100 yards from where they come out. I would set up in a heart beat is I could. This happens every year and like you, I have to wait it out and hope they follow a doe by the existing stand. Count your blessings that you can move to set up on him and go for it!