In knitting what can be done forwards, works backwards also, and you can cast off those stitches to look like a cast on in a bind off using a crochet hook.
As always, I'd recommend, don't try this until you know how to do the technique which works best. Sometimes you will not need additional yarn, but you might in order to give ease to under the arm.
If you make umpteen trials with the yarn in your project to discover what works, in the meanwhile your yarn won't be looking all that lovely which is why you recreate the problem (on a smaller scale which is certainly not reknitting the whole piece). The purpose of a provisional cast on is to allow you to knit in the other direction for which (when removed) you have live stitches even if the only thing you do is to cast off with those live stitches, -- again, there are bind offs and then there are bind offs, even sewn ones and they vary in the amount of stretch the garment requires more in some parts than others. Though knitters aren't keen on sewing by hand which is why I love to knit, it is sometimes the best alternative and since it sounds like the problem may hide itself in a seam, if you haven't done the other side of the garment, do that side right and then find a way to match it, as that way you will not be unhappy.
Even if you use a crochet hook, you can figure the mm of the hook if you have a metal knitting gauge which haply give you that info. My old one didn't do it, I had to buy a new one. This allows you to pick the right crochet hook which is the exact same gauge as your knitting needles. This works most of the time except for size 7 needle, often it most difficult to find a 4.5 crochet hook though they can be ordered separately if you have a better crochet hook set as I have which is also a Tunisian hook set. In an experiment you can find which hook works best to do a cast off. If you know how to cast off for a Tunisian project, this looks very nice also and has enough give or stretch for many purposes. There are examples of Tunisian bind off on YouTube and it is not hard to learn how to do this as whether I knit or crochet, I am just as skilled (usually).
Do what looks best which means you have to experiment but I do find that to convince myself of what to do also makes me more cautious in reading knitting instructions though I consider myself enough of an old pro that I should be able to fix everything. Let me tell you... I have a lot of practice of doing it wrong before an epiphany hits me.
Best wishes in your learning experience. If you choose to work a bind off after removing the provisional cast-on and now you have live stitches on your knitting needle, you may also find that you may need to take out one or two more rows to do the cast-off which will match your other part of your garment. No worries, but taking out stitches is not easy at the end of the rows as slipped stitches don't rip up from the bottom very nicely at all. Fortunately you have a short end from the cast on and you can pull that long piece of yarn through. Then rip to where you are comfortable but this is where when you are keeping in pattern that it is fiddly. It will indeed hide in a seem and determine you aren't going to be staring at the armpit once you wear it.
All armpits need a degree of stretch so using thread which cannot stretch is not a good idea as you will find it is uncomfortable.