Depends on the species of panfish in many cases, but big bull bluegills (bream) are very finicky. If you can see them, they can see you. The game is over.
If you are only seeing small fish on the beds, I would suggest fishing deeper water in close proximity of the beds. Larger fish are staging somewhere close if the body of water holds bigger fish at all.