Gauge problem

Have some yarn direct from Peru Alpaca soft. It comes with no gauge size only needle size which is a 6, not sure if that is 6 mm or not. Beautiful yarn, but little information. Looked it up on line (US) they say (if its the same yarn) gauge is 20-22 stitches per 4" size 4mm needles. Not sure if its the same yarn.
So my problem is the yarn that the pattern wants me to use (Blue Sky Alpacas Alpaca silk) is 23.5 stitches per 4" to 31 rows. I have tried what I think is every needle size to get that gauge with no success.
Any idea, would appreciate any help I can get. Wonderful pattern.
Many thanks

Wraps per inch will give you some idea of the gauge.

How close are you coming to the stitch gauge? What is the name of your pattern?