First, just to double check that fudging is necessary: Have you counted your rows carefully? You're sure you worked all 34 rows? And the decrease lines look good to your eye? Then don't unravel it....try this....
Unravel 4 rows, and do a decrease on both ends of the needle, on both sides of the work, for these four rows. This will give you 20 stitches in the end (close enough), and you'll have the same 34 rows, which you want, so it lines up properly with the front and with the sleeves.
I've done a little experimenting on a small sample, and here's what I think looks best. These extra decreases look almost invisible, so they won't interfere with the nice pattern of your decrease. I bet you'll hardly notice the difference. You can certainly try it anyway!
Row 1: k2, ssk, knit until there are 4 sts., and k2tog tbl (as usual), k2
Row 2: p2, p2tog, purl until there are 4 sts., and p2tog (as usual), p2.
Row 3: repeat row 1
Row 4: repeat row 2.
The ssk decrease is on the decrease page.
Good luck! Let us know how it goes!