That can happen. Sometimes people knit tighter when they're stressed, like trying to match gauge, and then relax when they knit the larger item. Best thing to do is knit a very large gauge square and also check as you knit the sweater that it's still the same.
Also, if the sweater is knit in the round, the swatch should be as well. Sometimes people start with a sleeve as the gauge sample as it's a little larger piece in the same pattern so it can be more accurate.