Miniature Ampersand Sweater confusing pattern or confused knitter

I have twice started to knit this cute miniature sweater (ravelry.com/patterns/miniature-ampers-sweater) and twice got frustrated and given up. I have knit mini sweaters and socks, lace and amigurimi so I’m not a beginner. But I find this pattern confusing. Just wondering if there’s anyone else who has had the same experience.

I haven’t done this one but where are you running into problems?