Heh, sounds like the way I sew... Only I do costuming, so it's usually more like... "So, can I just make this Victorian jacket back without pre-designed darts then sort of try it on and pin it myself (after self-strapping into a period Victorian corset - not fun) and fake the darts later?" Or... "I saw this garmet that looked like this and I think I can recreate that by doing this entirely different thing..."
I say, you've got the measurements, and you've got the picture in your head, all you've got to waste is time. Which will still give you learning experience (oh geeze, not another learning experience?!? I usually say that about life experiences...) if it doesn't work like you want... Just rip the stitches and try again.
If the bag is flat with handles, the "pattern" you've described sounds great. And you can alter the demensions as you want to make it larger or smaller. Or knit it too big in wool and felt it to give it a different texture and more sturdy construction.
My vote, knit on and give it a shot!