You didn’t mention any trouble with knowing how to cast on the new stitches after you have already started your project. So I assume you know how to do that. I think I would use a knitted cast on. Is the first row that you purled color A already? I think from what you say that it is.
Purl your row with color A, then work into the first stitch at the beginning of the knit row to work your first knitted cast on stitch. After you have 8 stitches of color A cast on, just leave that color there and use color B when you pull the next stitch through for your knitted cast on. You don’t need to attach it in any way. Cast on your 11 stitches of B and then knit back across them. When you come to the color change be sure to pick up the color A from under color B (you are doing intarsia here)
Knit across all of the stitches in color A. Now you are ready to begin the purl row. With a new ball or the other end of the skein cast on 11 stitches of color B using the knitted cast on, but do it in purling (I never did this before, but tried it and it works), then drop color B and start purling color A on. Purl back across the stitches changing colors as needed and being sure to pick up from underneath.