What an intriguing idea!!! I wear headbands ALOT and never thought of making a felted one. I think you could use some leftover yarn and figure it out trial by error!!!
Pick up the inexpensive plastic headbands and start knitting. Jus knit a tube that is bigger than the band, slip it on, sew it up and throw it in the washer. You might want to knit a swtch and felt it first to see what the "approximate" shrinkage is.
The other option would be to knit and felt a rectangle then cut and glue it to the headband.
I just might have to give that idea some furthre consideration!!!