If it's just the hatband that's stretched, buy some of the elastic you can knit with and thread a needle with it. Run three to four rows through the inside of the band so it doesn't show from the outside, and knot it to itself at just a bit tighter than the finished circumference you want (ie: your head).
You can knit holding a strand of the elastic thread along with your yarn and it will help support the alpaca from stretching. (If you prefer or feel it works better for your application, a nylon thread can be used instead, which doesn't allow stretching.)
If it's a beret or slouch, be sure that the crown isn't too long for your head -- you might want to use a slightly smaller needle or leave out a couple of rows. This means your head will help to hold the hat up. You can see if this is necessary by pinning a couple of darts (running from the center point out to the crown line) into the hat at just the crown line. If this helps hold the hat up, the crown may be too long for your head.