Mix and match sizes for a better fit?

I am probably being way to ambitious to even consider trying to make this, but I can’t stop dreaming about wearing it. However, I always worry about size when I am making sweaters, and since short rows don’t seem to be an option with this particular pattern I was thinking about following the instructions for a smaller size for the back, and then going up a size for the two front pieces.

Will this work?

You could do that, just make sure you follow the measurments for length (along the side seams and armholes) and that you end up with the right number of sts to sew the shoulder seams.