Sewing button on loose weave mohair button band of jacket

OK, I don’t like to sew- not even buttons------but this is making me crazy. I’m finding my weave is too loose for regular thread, however the button will only allow for a regular sewing needle and thread. It’s a bakelite toggle - w/ one hole - so difficult to anchor.
Any ideas???

Can you put a small patch of felt (maybe folded a couple times for extra thickness) in a similar color to the yarn on the back (wrong) side of the knitting to help anchor the button?

Can you get the yarn through using a “needle threader” ? Or, sew the button on to a strand of yarn … then put the yarn on the sweater and weave ends?