I think your idea of 4.5 miles in to hunt is not even an option for the reason that if you kill an elk in at that distance, sounds like you're not in the shape needed to get the elk out. Packing out elk is not for the faint of heart, or the out-of-shape.
Maybe plan on going in 1.5 or 2 miles for starters,..... and if you want to get in better shape, you'll find time to do it. Weather it be 45 minutes earlier out of bed or find 45 minutes after sun down, ya gotta find time to get some workouts in, otherwise, the mtns. and the elk are going to eat your lunch. Because more often then not, ya gotta find the elk before ya can even hunt the elk. And sometimes, finding them takes a lot of hiking.
Set some goals for yourself but don't set them too high. For instance, decide if you'd be happy with meat/cow, or even a 4x4 bull. Don't push it too hard on the first day. If you do, the days following won't be much fun. At the very least, don't push it so hard that you don't enjoy the scenery. The changing colors should be good for the soul while you're hunting.