On looking at the pattern it seems you are folding the cast off edge against itself and seaming it there. A mattress stitch is used to seam edges of knitting, like the side seams of a sweater. You don’t have that situation here.
I did a little experimenting. There are several ways the pattern stitch could be ended before you bind off, and I don’t know what differences those combinations might make. What I did was a row of KPKPKP and then an identical row like it, lined up (like ribbing) and then I bound off in the opposite of what appeared on the next row, which was where I was in the pattern. Then I folded it so that the cast on edges butted against each other. I used a stitch like what you might use to sew two shoulders that were bound off together. I went beyond the cast off and went under a purl bump or a knit stitch on either side (whatever appeared) going back and forth from one side to the other. I was satisfied with the look of the seam I came up with doing it this way.
This page from an online knitting book will be helpful I think. This shows knit stitches on each side, but it is the idea, except you will have alternating knits and purls. This shows one way to sew a shoulder seam.
Try your best to begin with the outer most stitch on each side as you begin and if you keep the stitches one for one you should turn out even in the middle.