Yes, you should log, but do so carefully and selectively! Too much clear cutting and the few deer you have will be gone for many years except at night to browse - otherwise they will hide out in your neighboring big forest. Your situation sounds similar to what I have (small woods surrounded by agriculture and bigger woods), but my property was logged so hard (by a logger that said he was managing it for hunting) that the few deer I had left only to return at night. Get someone that professionally manages forests for hunting to mark only the trees to be cut, and then if you get a logger in there, cut only those trees. And don't let the logger leave a big tangled mess that serves as an obstacle course. I wish I would have gone this route - my land is just now starting to recover as a place of daytime cover a whole 4 years later.