I'm thinking maybe it's only a coincidence and what really is happening is that the deer located an alternate, more favorable, food source. Don't know where you are at, but there might be a mast drop in your area that is feeding them for now. If that is the case, you need not worry because they will be back.
Out of curiousity, can you describe or post pictures or a drawing/sketch of what your feeders you built look like?
iSnipe