If shooting long range (500 yds) is what you really like, stay at the range. If you want to get into deer hunting, I recommend your first priority is becoming a deer hunter. In just about every kind of terrain there is, a good hunter can put himself much closer than 500 yards from his deer. I consider half that range a very long shot. In fact I've never shot a deer much over 100 yds.
IMO, hunting should be about the hunt, not the length of the shot, regardless of your ability to hit at long range. What have you done when you shoot a deer that far away? You didn't hunt it, all you did was make a long shot.