Yes, you'd have to adjust the number of stitches. Figure out how many inches around the hat should be by dividing the pattern gauge (stitches per inch) into the number of stitches. Then see how many stitches you get with the other needles and multiply by the measurement you got before. You may have to adjust the number a little to work any ribbing or other stitch repeat - for example, k2 p2 ribbing won't work if the stitch number is a multiple of 5, you have to have a multiple of 4 sts. Then you may have to adjust the decrease ratio a little.
It helps if you can link a pattern, or tell us what it is, so we can look at it to help you in more detail.