I've never done this for these circumstances, but it seems like you might be able to "fix" it (not perfectly, but perhaps it would be acceptable), by using a sewing machine.
If you've done four or more rows, it might be worth it to you, to at least try. Worst case scenario is you'd be wasting the yarn this way, but it won't waste much time to try....
Using very fine stitched on a sewing machine, run it through a sewing machine twice: once on either side of a "column" of stitches (so, you'd be sewing from the cast-on edge, up towards your needles, or vice versa; and then again, on the other side of the same stitch). Do this in an area that will be the side-seam, so it's least noticable. Then (don't scream--LOL), cut the stitches between these two sewing lines. Untwist your knitting, and seam the two halves together again!
If you want more instruction on this, you can google "steeks" and "sewing machine," and you'll probaby find some more instruction. What you're doing is not a steek, but the technique is similar.
If you've already ripped it out, don't feel bad. You'll get used to ripping things out as a knitter! It's always hard in the beginning, but you do get used to it; it becomes part of the Zen of knitting.