If it's a button-up sweater, then you're working one side at a time, right?
In that case you do it as you said, at the beginning of the right-side rows if that is the neck edge of the side on which you're working.
For the left side as you wear it, you'd do the neck edges on the beginning of the ws rows, for the right side as you wear it, you'd do them at the beginning of the rs rows.
The neck edges come before any shoulder shaping.
If the front is in one piece, then it's a whole other set of directions.