As frustrating as that was, you sure got a cute hat out of it!
Yeah, stick this idea on the back burner and maybe try it again later. I have ideas spinning around in my head that will some day see the light of day (and knitting needles)--just as soon as I figure out how to make them work. That's part of the fun of crafting--trying to figure out how to do something you've dreamed up (at least, I think it's fun).
At any rate, you've got an adorable hat to show for your hard work. That doesn't always happen, so pat yourself on the back!