Making a patchwork bag

I have several gallon zip lock bags of wool ends of all different brands and colors. My plan was to use them to make a felted patchwork bag. Of course now it occurs to me that all the wools will felt differently. My initial plan was to have it all put together before I felted it. I could felt all the squares then seam them together, but I don’t want the seam to be visible…

Is there a way to make this work that I am not thinking of?