I highly recommend it. You just have to inspect the side seams and make sure they're not surged (which cuts the yarn at the seam). It's OK if the shoulder seams are surged, because you can just cut that part off. Then you just eyeball the "donor" sweater to guesstimate whether it'll be enough to make the item you have in mind. Certainly a sweater is plenty to make a smaller project, such as a hat, wristwarmers, mug cozy, etc. I suppose you could also weigh the donor sweater and compare the weight with what your pattern calls for. I unravel and knit right off the sweater pieces rather than unraveling and winding the yarn into balls.
I regularly go to Good Will and buy sweaters to harvest yarn and buttons. I look for wool and wool blends, cotton, silk, angora, and cashmere. Occasionally, I'll take an acrylic if I especially like the colors.