You should make the swatch 4 inches long over the 19 stitches. Then it is a good idea to take it off the needles and lay the swatch on a hard, flat surface, but yes, you are measuring the stitches as they sit on the needles, in other words from side to side on the swatch. You want the middle 15 stitches (excluding the 2 on each side) to measure 4 inches to be on gauge.
If it doesn't measure that much, say it is 3 1/2" you need to try again with larger needles. If it measures more than 4", say 4 1/4" you should try one size smaller needles and actually you should make another gauge swatch. (But in truth I don't always, if it is not close to being right and I think I should be right on with one size different.)