Is the cotton yarn you're working with a chenille? I had a worming problem a couple of years ago on a chenille scarf and knitting on a smaller needle didn't help. My friend also had the same problem (yarn was called something like Sinsation - ?rayon chenille). Since she liked the feel of this yarn so much, she made yet another scarf, felted it and she said that solved the worming problem. However, I had a couple of chenille sweaters (purchased!) a few years ago - one wormed even after putting it in the washer and dryer and the other (smaller gauge) is okay - so I don't know what the answer is.
A lady at the LYS told me to knit the chenille with another yarn to solve the worming problem. I didn't try this.