Check your local regulations first. If you're in a development like I'm going to be, it may not be permitted to fire a weapon...of any kind...where you live.
I'm going to have this same "problem"...if you can call it that, in a couple of weeks when we move into our new house. It's in a development, but most folks have over 2.5 acres and there is a lot of undisturbed timber. Deer everywhere and I'm the only hunter. I have already been warned by the neighbors that they are "the neighborhood deer" and I better not shoot them. We have already seen them crossing our property right through our front yard. Neighbors or not, I have no problems and plan to shoot one...or more if I can get permission from the DNR folks based on the fact that they are destroying our plants...which they do. If I get that permission, I'll shoot one out of my garage if I can.