Hi there prickypearbloom, welcome to the Knitting Help forum.
Yes, I think you are knitting inside out. It has something to do with the way you got started at the very beginning that causes this to happen. You can push the work through and continue on, but I think you also need to turn your work around so that you are knitting in the opposite direction from what you have been doing. You will also want to figure out how to get started the right way on the second mitten. Pushing the yarn through might make a bit of a hole where you change direction.
But.... there are people who knit as you are doing on a regular basis and find it works just fine, at least for some things (and maybe everything). One really good knitter on the forum who knits that way is ArtLady1981. I think you might send her a Private Message (she is a very gracious and helpful lady and will not mind if you contact her that way) via the forum here and ask her if she thinks you can do the mittens that way without a problem. It is possible that you can, and like you said just turn them right side out later. Then if you get this mitten figured out you can either choose to keep knitting on the far side of the circle, looking at the inside as the outside, or switch to the more common method. I really think either way will work, but I may be wrong. Contact ArtLady1981 and see what she would do.