It sounds like you have described a bear that has gotten used to associating people with an easy meal. That is becoming more common these days. Bears generally have no desire to attack people. They are all about finding the best source of food the easiest way they can. I would not be to concerned of that kind of incident happening on a regular basis though.

I would make it a point to do a little target practicing each time I went to that property, bears do not like gunfire.
And it would give you the opportunity to gain confidence in your shooting ability.

Would a 30-06 be enough to stop a bear? You bet, bears are not hard to kill, but you need to place your shot just right if you want to drop him in his tracks. Either way once you put a round in him, he's going to have a whole new respect for you.

And just remember when you have a gun available, you are the BOSS!!!

Hope this helps.....I understand your concern, but it is not something you should let deprive you from enjoying your property.