my first idea would be:
count your stitches. If the number is the same as the cast on you are not adding stitches. Good thing, you shouldn't.
Then the question is: are you knitting the same tightness? you might be getting lose in your work. That could make all the difference. Are the stitches about the same "open and lose"? if so, that is not your problem so much, either (but it might be a minimal change from row to row.)
another frequent thing is: you might have done a very tight cast on. then the knitting is often more lose than the cast on and therefore the knitting flares out from the outer edge.
If that is your problem you would not get an increase in size anymore after you go a couple of inches further.
to fix that, though, I would pull it all out and cast on again (maybe with a different method? maybe with extra lose method (second needle)?)
Since I do not want to confuse you: I will give you tips on how to propperly fix the problem once you have identified the matter. Does any of the 3 things above happen?
- more stitches than in cast on?
- more and more lose knitting?
- flare after the cast on edge only?
do not worry, by the way, we all started small at some point!