Yikes! help!

Hi there, I am just about finished with this little jumper and I am beginning to get a little frusterated with the ribbing around the collar and armholes. I can’t figure out were exactly to pick-up the stitches from. Would I pick them up on the edge like you do when you are picking up stitches for the gusset on socks? I did the arm first (on the selvdge (sp?) edge – picking up the front stitch loop) then I worked the collar picking up the back stitch loop. They both look funny.
Any thoughts would be great!

Here is my pattern: