So long as you did everything possible to retrieve the deer.....it happens. It is and should be a sick feeling to lose a deer but sometimes you can't help it. 30-45 min should be fine, to be honest but know when to back off when you can't find him and give it some time. Most of the time, they'll run a bit and go down and waiting hours is a little bit of overkill.
Grayish material? Could be liver....if it were a gutshot, there should be a noticable odor.
If you can't find him that day, try again the next morning.
I'm sorry for your loss, though. If you hunt enough, its bound to happen. You can zero your firearm in and practice your heart out and still bump it climbing a tree of something, as an example. You can be a great shot and still pull one, get buck fever, any number of things. The main thing is to exhaust all methods and effort to recover the deer.