I have been a little disappointed with my first couple projects where my casting on is tighter than my binding off. Also, although similar, the bind off is a thicker, neater edge and for a scarf edges not matching I think looks kinda funny…
There are several different cast on methods. Have you tried a different one that might be more to your liking?
Another thing that some patterns call for is to cast on and knit half the scarf. Then cast on and knit the other half of the scarf. Then you graft the live stitches together. I haven’t done this and probably never will because I’m not a big fan of grafting live stitches, but it’s an idea.