I know exactly what it is since I’m doing the same thing for a sweater I’m knitting.
What you’ll be doing is folding or “turning” the hem to the inside along the that row…the row with the YO, k2tog. It creates a picot edge and is very pretty. Just do as the pattern says.
The turning round is created when you do the YO k2tog; later that’s where you turn up the lower edge to make the hem and because of the YO k2tog it makes a picot edge.
for now: just knit taht round. You will end up with the same amount of stitches as before. only you have created a line of hole and stitch and hole and stitch.
Then you keep working your mitten. Do not worry about the hem until the pattern tells you so.
Then you fold the work over in the place of this line. Fold the edge inside and make it sit right on the holes. The stitches between the holes stick out, the holes retreat.
in the end there will be a picot edge as said above. A wavy edge that looks nice and holds the fabric from curling over.