That one person spent $350 on a bow and another spent $650, does not mean that the man buying the more expensive bow is being frivolous. For heaven's sake its only a few hundred bucks difference. In the whole scheme of things that is not that much money to spend on a sport that you enjoy so much.
In a way it reminds me of guys complaining about an extra $5 when fishing license fees go up. "I can't catch enough fish to make it worth it for me to buy a license." As if you don't spend a ton of money on tackle, bait, lures, a boat and so on.
Aside from an arrow that is quality from tip to nock, and a good release, what other piece of gear is more important than the bow itself? Your grunt call? Your hat? (Well maybe if its a Mathews hat

)
Buy whatever you want, spend as much as you please, and let those guys see if they can "shoot circles" around you with their older bow (as if an newer bow with less vibration and/or torque wouldn't improve
their archery somewhat.) Even if they can, you won't enjoy a quality high-end bow any less.
Enjoy yourself. Buy the best equipment you can afford...which is always a good idea anyway...and remember, you only go around once.