It is unlikely that you dropped, and then picked up, so many sts that the sweater got larger as you worked it.
The more likely scenario is that your gauge is off and/or your tension is very loose. Check the gauge on your, now finished, sweater and see if it is "true" to the pattern. You probably need to go down a needle size (or two) and/or make a smaller size (S not M, M not L) sweater.
Overall, a very nice job on your first sweater. The garter stitch ridges and deco at the top stand out, which means you got the hang of knitting/purling and changing between the two.