If you’re getting refusals, (assuming your drift and tippet sizes are correct), make adjustments to your fly. Legendary Pennsylvania fly fisherman, Charlie Fox called it “a game of nods”. If a fish would refuse his pattern, he would start trimming down his fly. He would continue to trim his fly until the fish took it. If the fish didn’t take it, he would give it a nod, then be on his way.