If the fruit are dropping more than just a few wind blown fruits then you know they are ripe. They are bitter almost up till they start to drop. Try one out for yourself, you will definitely know.