I have to agree with the above comments. I, like everyone, try to keep everything even when I cut and mount bands but I've never been crazed about it and it's never really been an issue.
One thing that was not mentioned above is the possibility of a bad pouch. I've been doing this a long time now and on rare occasions I've had pouches that shed the ball inconsistently due to grain direction on leather or a stiff synthetic pouch. If the pouch is synthetic consider putting a ball in the pouch, holding it tight with your fingers and then pass a flame around the stretched pouch. Not much heat is needed. A few seconds will form a permanent shallow cupping that will help secure the ball. If the pouch is leather then wet the pouch, put a ball in it and, after stretching it tight, secure it and let it dry.
Twisting the pouch as part of your draw before shooting is fine but any bending of the pouch will throw you off as will a quick and inconsistent release. The problem is most generally the release.