Anyone whose gender presentation does not align with society's conventions would face fewer difficulties.
Anyone whose appearance deviates from societies narrowly defined conventions of gender can face scrutiny in a public restroom. Cismen with feminine features and ciswomen with masculine features could face difficulties in public restrooms if transgenders are challenged on their freedom to use their preferred restroom.
If the law dictates that transgender citizens cannot use the restroom of the gender they identify with, it opens to door for cismen and ciswomen who do not have a physical appearance which matches their birth sex to face scrutiny. For example, it could lead to ciswomen who are often mistaken for males due to their masculine features being challenged over their right to use the female restroom. By allowing transgendered citizens to use the restroom of their choice, the law would reduce the margin for confrontation among people whose appearances do not meet the societies narrowly-defined preconceptions. This occurred most recently in a Walmart bathroom when a cisgender woman was insulted after she was mistaken for a trans woman. The victim of the abuse said she believed the perpetrator singled her out because she had short hair. 
Cis bathroom users have their birth certificates which show their legal sex. Any confrontation could be easily diffused through the production of documentation showing their official, state-recognised sex. 
Banning trans people using the bathroom of their choice would lead to increased confrontation in restrooms. Even cismen and women could face challenges in public restrooms if they have features atypical of their gender.
Any confrontation with cis citizens would be easily diffused by showing ID with the individuals sex assigned at birth