Once youre finished the hole won't be noticable. You're probably the only one who will see it because you made it, just don't point it out to anyone and you should be fine!
I used the fun fur stuff for a few scarves and it's easy to make mistakes with that stuff. But because of the fur it's forgiving. Definitely post a picture when you're done!