The WHIP plantings are for CRP, not deer specifically. The mix you used sounds like something our DNR biologist recommended for use on field edges to make a transition zone between the field and the timber. This is more of a upland/small game management tool than deer.

Depending on the amount of Orchard grass, I wouldn't be too concerned to start with. Let it go until next year and see what happens. Worst case, you can use a grass specific herbicide to get rid of it and still keep the clover.