I'm not sure what you can do. All I can think of is to put the stitches on a smaller needle than you are using when you take them off the holder so that they don't get stretched out. But if they are already stretched out I don't know what to say. You could try knitting with a smaller needle but that may create different issues. Sometimes you knit something in the back loop to get rid of looseness, but that would twist all the stitches and I don't know that that would be acceptable in this case.
I just had another idea. If the stitches on the holder are stretched out, maybe you could take a row or two out and reknit them handling them more carefully this time. Just an idea. I don't know how practical it is in your situation.