I’ve never thought of that; but aren’t your color changes only on one side?
Check out the Mattress Stitch. It doesn’t show–you can use one color from the sweater, and mattress stitch hides it quite well. I do think you’d be better off with one continuous seam rather than many short ones.