How to attach sleeves

I’m working on my second baby sweater. For the first one, I was disappointed to find that the sleeves were a big challenge to attach, the sleeve openings were much larger than the sleeves. But I forced it to work and it came out a bit bunchy. Now I’m on the second sweater (different pattern) and find I’m in a similar situation. HOWEVER, I have noticed that the sleeves DO match if I only go as far as the “corner” were I decreased to shape the armhole. So do I only sew to that point? And then when I sew the side seams, that will fall into place? (By the way, I measured the pieces and they are pretty close to what they should be so I don’t think it’s a gauge problem).

Are you knitting the arms flat or in the round? I’ve done a couple of baby sweaters that the sleeves were knit flat… I seamed the flat sleeve to the flat sweater (side not seamed yet), lining up the center of the sleeve to the shoulder seam. Then I brought the side seams together and finished off with the sleeve seam. Does that make sense?