I thought I'd report back what my favorite gunsmith's action and barrel man told me today on the phone.
He said they do not advise using the 30/06 chambering in the 96 actions because of pressure issues. He admitted that there are some 96 actions out there up to the task, but more often than not one encroaches on the pressure limits of the 96 action so it is not recommended to go with the 30/06 OR the 308.
I asked if it was also then out of the question for the 7mm/08 round in the 96 action, figuring he'd say "Don't do it" since it is the same case as the 308 that is not recommended. Interestingly enough he claimed that the 7mm/08 in the 96 action was fine.
I think that may be the direction I go, if I can just find a nice 7mm/08 small ring barrel now!
Thanks to all for the replies and advice. I always find it best to hear a variety of views and experiences since it enhances my overall knowledge much more than relying on one answer.