I figured it out. QGis assumes that the first three channels are blue-green-red, whereas they are red-green-blue in these GeoTiff images. I'm now getting reasonably looking color images.
Is there a specific colormap I should be using? I can extract the band values with gdal, but still need a colormap.