The working wool being between is part of the security of the join! Pull it tight and then knit... if it is really annoying you, you might prefer the way it looks if you turn it around and knit the other way... on the caston row, if you switch this way, you could theoretically start knitting in either direction.
(If this is not an option because you prefer one side of the caston row to the other side, try changing how you switch: instead of passing the first over the last, pass the last over the first, or the other way around).
It's possible that you are turning your caston around so that you cast on stitches 1-30 on that order, but you have gone back so your first row starts with stitch 30, then 29... turning around and going back. Try it the other way from whichever you were using before.