I applaud your creativity in adding an unexpected design feature to correct the problem. :yay:
It may just be that the patterns are written for a certain build and you don't fit into that mold.
That was my first thought also. I'm made up of mismatched parts so I encounter this type of thing in all items of clothing.
You could cast on the sts for a smaller size, but do more increases until you reach the stitch number for your regular size. The great thing about top down sweaters is that you can try them on as you go so you can see how it's fitting you and do more incs or go
back a few.
I've just started my first top down sweater, and this is one reason I wanted to learn to do them. This one isn't for me so it can be a more standard size but not getting the neck right is too frustrating even when it can be corrected later.
I feel your pain.