My guess either the hole was to close to the bank or thick cover for their liking
Or they saw something they didn't like
Like you , a dog , a silver candy wrapper ect

I have done the same thing a lot of times with great luck
But like you , sometimes the birds just will not play

Was it sunny ?
Had problems a few times on sunny days , never could figure it out

Tryed it again a day or 2 later and the ducks and geese poured in

John