Using the rough side of the leather increases friction between the two pouch sides holding the steel ammo when you're pressing the pouch together for a shot in full draw: this helps to ensure a linear release, assuming that the ammo is centered, as opposed to the potential risk of the inner pouch sides slipping from alignment during the release - with the consequence of possible stray ammo trajectories or fork hits.
In contrast, the smoother outer surface of the leather of a pouch avoids bad releases linked to unwanted friction on the index finger or thumb and misalignment during the release - not to mention the absence of abrasive action on the skin during longer shooting sessions.
Thus, the rough side is the inner side of a pouch for me.
My 2 cents worth...