Xref: IRO.UMontreal.CA comp.graphics:26445 alt.graphics.pixutils:844 alt.binaries.pictures.utilities:890 alt.binaries.pictures.d:2228 alt.binaries.pictures.erotica.d:3270 news.answers:2757 Path: IRO.UMontreal.CA!bonnie.concordia.ca!uunet!ogicse!das-news.harvard.edu!cantaloupe.srv.cs.cmu.edu!tgl From: tgl+@cs.cmu.edu (Tom Lane) Newsgroups: comp.graphics,alt.graphics.pixutils,alt.binaries.pictures.utilities,alt.binaries.pictures.d,alt.binaries.pictures.erotica.d,news.answers Subject: JPEG image compression: Frequently Asked Questions Summary: Useful info about JPEG (JPG) image files and programs Keywords: JPEG, image compression, FAQ Message-ID: Date: 8 Sep 92 14:38:29 GMT Expires: Tue, 6 Oct 1992 14:38:22 GMT Sender: news@cs.cmu.edu (Usenet News System) Reply-To: jpeg-info@uunet.uu.net Followup-To: alt.binaries.pictures.d Organization: School of Computer Science, Carnegie Mellon Lines: 809 Approved: news-answers-request@MIT.Edu Supersedes: Nntp-Posting-Host: g.gp.cs.cmu.edu Archive-name: jpeg-faq Last-modified: 8 September 1992 This FAQ article discusses JPEG image compression. Suggestions for additions and clarifications are welcome. New since version of 23 August 1992: * More on color quantization: 256 colors in does not mean 256 colors out. * More info on Mac and Amiga JPEG programs. * Some info on Acorn Archimedes programs too. This article includes the following sections: 1) What is JPEG? 2) Why use JPEG? 3) How well does it work? 4) What are good "quality" settings for JPEG? 5) When should I use JPEG, and when should I stick with GIF? 6) Where can I get JPEG software? 6A) "canned" software, viewers, etc. 6B) source code 7) What's all this hoopla about color quantization? 8) How does JPEG work? 9) What about lossless JPEG? 10) Why all the argument about file formats? 11) And what about arithmetic coding? 12) Does loss accumulate with repeated compression/decompression? 13) What are some rules of thumb for converting GIF images to JPEG? Sections 1-6 are basic info that every JPEG user needs to know; sections 7-13 are advanced info for the curious. This article is posted every 2 weeks. You can always find the latest version in the news.answers archive at rtfm.mit.edu (18.172.1.27). By FTP, fetch /pub/usenet/news.answers/jpeg-faq; or if you don't have FTP, send e-mail to mail-server@rtfm.mit.edu with body "send usenet/news.answers/jpeg-faq".) ---------- 1) What is JPEG? JPEG (pronounced "jay-peg") is a standardized image compression mechanism. JPEG stands for Joint Photographic Experts Group, the original name of the committee that wrote the standard. JPEG is designed for compressing either full-color or gray-scale digital images of "natural", real-world scenes. It does not work so well on non-realistic images, such as cartoons or line drawings. JPEG does not handle black-and-white (1-bit-per-pixel) images, nor does it handle motion picture compression. Standards for compressing those types of images are being worked on by other committees, named JBIG and MPEG respectively. JPEG is "lossy", meaning that the image you get out of decompression isn't quite identical to what you originally put in. The algorithm achieves much of its compression by exploiting known limitations of the human eye, notably the fact that small color details aren't perceived as well as small details of light-and-dark. Thus, JPEG is intended for compressing images that will be looked at by humans. If you plan to machine-analyze your images, the small errors introduced by JPEG may be a problem for you, even if they are invisible to the eye. A useful property of JPEG is that the degree of lossiness can be varied by adjusting compression parameters. This means that the image maker can trade off file size against output image quality. You can make *extremely* small files if you don't mind poor quality; this is useful for indexing image archives, making thumbnail views or icons, etc. etc. Conversely, if you aren't happy with the output quality at the default compression setting, you can jack up the quality until you are satisfied, and accept lesser compression. 2) Why use JPEG? There are two good reasons: to make your image files smaller, and to store 24-bit-per-pixel color data instead of 8-bit-per-pixel data. Making image files smaller is a big win for transmitting files across networks and for archiving libraries of images. Being able to compress a 2 Mbyte full-color file down to 100 Kbytes or so makes a big difference in disk space and transmission time! (If you are comparing GIF and JPEG, the size ratio is more like four to one. More details below.) Unless your viewing software supports JPEG directly, you'll have to convert JPEG to some other format for viewing or manipulating images. Even with a JPEG-capable viewer, it takes longer to decode and view a JPEG image than to view an image of a simpler format (GIF, for instance). Thus, using JPEG is essentially a time/space tradeoff: you give up some time in order to store or transmit an image more cheaply. It's worth noting that when network or phone transmission is involved, the time savings from transferring a shorter file can be much greater than the extra time to decompress the file. I'll let you do the arithmetic yourself. The other reason why JPEG will gradually replace GIF as the standard Usenet posting format is that JPEG can store full color information: 24 bits/pixel (16 million colors) instead of 8 or less (256 or fewer colors). If you have only 8-bit display hardware then this may not seem like much of an advantage to you. Within a couple of years, though, 8-bit GIF will look as obsolete as black-and-white MacPaint format does today. Furthermore, for reasons detailed in section 7, JPEG is far more useful than GIF for exchanging images among people with widely varying color display hardware. Hence JPEG is considerably more appropriate than GIF for use as a Usenet posting standard. 3) How well does it work? Pretty darn well. Here are some sample file sizes for an image I have handy, a 727x525 full-color image of a ship in a harbor. The first three files are for comparison purposes; the rest were created with the free JPEG software described in section 6B. File Size in bytes Comments ship.ppm 1145040 Original file in PPM format (no compression; 24 bits or 3 bytes per pixel, plus a few bytes overhead) ship.ppm.Z 963829 PPM file passed through Unix compress compress doesn't accomplish a lot, you'll note. Other text-oriented compressors give similar results. ship.gif 240438 Converted to GIF with ppmquant -fs 256 | ppmtogif Most of the savings is the result of losing color info: GIF saves 8 bits/pixel, not 24. (See sec. 7.) ship.jpg95 155622 cjpeg -Q 95 (highest useful quality setting) This is indistinguishable from the 24-bit original, at least to my nonprofessional eyeballs. ship.jpg75 58009 cjpeg -Q 75 (default setting) You have to look mighty darn close to distinguish this from the original, even with both on-screen at once. ship.jpg50 38406 cjpeg -Q 50 This has slight defects; if you know what to look for, you could tell it's been JPEGed without seeing the original. Still as good image quality as many recent postings in Usenet pictures groups. ship.jpg25 25192 cjpeg -Q 25 JPEG's characteristic "blockiness" becomes apparent at this setting (djpeg -b helps some). Still, I've seen plenty of Usenet postings that were of poorer image quality than this. ship.jpg5o 6587 cjpeg -Q 5 -o (-o reduces table overhead) Blocky, but perfectly satisfactory for preview or indexing purposes. Note that this file is TINY: the compression ratio from the original is 173:1 ! In this case JPEG can make a file that's a factor of four or five smaller than a GIF of comparable quality (the -Q 75 file is every bit as good as the GIF, better if you have a full-color display). This seems to be a typical ratio for real-world scenes. 4) What are good "quality" settings for JPEG? (Note: the -Q settings discussed in this article apply to the free JPEG software described in section 6B. Other JPEG implementations, such as Image Alchemy, may use a completely different quality scale.) The name of the game in using JPEG is to pick the lowest quality setting (smallest file size) that decompresses into an image indistinguishable from the original. This setting will vary from one image to another and from one observer to another, but here are some rules of thumb. The default quality setting (-Q 75) is very often the best choice. This setting is about the lowest you can go without expecting to see defects in a typical image. Try -Q 75 first; if you see defects, then go up. Except for experimental purposes, never go above -Q 95; saying -Q 100 will produce a file two or three times as large as -Q 95, but of hardly any better quality. If the image was less than perfect quality to begin with, you might be able to go down to -Q 50 without objectionable degradation. On the other hand, you might need to go to a HIGHER quality setting to avoid further degradation. The second case seems to apply most of the time when converting GIFs to JPEG. The default -Q 75 is about right for compressing 24-bit images, but -Q 85 to 95 is usually better for converting GIFs (see section 13 for more info). If you want a very small file (say for preview or indexing purposes) and are prepared to tolerate large defects, a -Q setting in the range of 5 to 10 is about right. -Q 2 or so may be amusing as "op art". Another recommendation: when you are making a final version of an image for posting on Usenet or archiving, specify "-o" to cjpeg. This will make the file a little smaller without affecting image quality; it will take longer to do the compression, but not any longer to decompress. 5) When should I use JPEG, and when should I stick with GIF? As a rule of thumb, JPEG is superior to GIF for storing full-color or gray-scale images of "realistic" scenes; that means scanned photographs and similar material. JPEG is superior even if you don't have 24-bit display hardware, and it is a LOT superior if you do. (See section 7 for details.) GIF does significantly better on images with only a few distinct colors, such as cartoons and line drawings. In particular, large areas of pixels that are all *exactly* the same color are compressed very efficiently indeed by GIF. JPEG can't squeeze these files as much as GIF does without introducing visible defects. This sort of image is best kept in GIF form. (In particular, single-color borders are quite cheap in GIF files, but they should be avoided in JPEG files.) JPEG also has a hard time with very sharp edges: a row of pure-black pixels adjacent to a row of pure-white pixels, for example. Sharp edges tend to come out blurred unless you use a very high quality setting. Again, this sort of thing is not found in scanned photographs, but it shows up fairly often in GIF files: borders, overlaid text, etc. The blurriness is particularly objectionable with text that's only a few pixels high. If you have a GIF with a lot of small-size overlaid text, don't JPEG it. Computer-drawn images (ray-traced scenes, for instance) usually fall between scanned images and cartoons in terms of complexity. The more complex and subtly rendered the image, the more likely that JPEG will do well on it. Plain black-and-white (two level) images should never be converted to JPEG. You need at least about 16 gray levels before JPEG is useful for gray-scale images. If you have an existing library of GIF images, you may wonder whether you should convert them to JPEG. You will lose some image quality if you do. (Section 7, which argues that JPEG image quality is superior to GIF, only applies if both formats start from a full-color original. If you start from a GIF, you've already irretrievably lost a great deal of information; JPEG can only make things worse.) However, the disk space savings may justify converting anyway. This is a decision you'll have to make for yourself. If you do convert a GIF library to JPEG, see section 13 for hints. 6) Where can I get JPEG software? 6A) If you are looking for "canned" software, viewers, etc: The first part of this list is system-specific programs that only run on one kind of system. If you don't see what you want for your machine, check out the portable JPEG software described at the end of the list. X Windows: John Bradley's free XV (version 2.00 and up) is an excellent viewer for JPEG, GIF, and other image formats. It's available for FTP from export.lcs.mit.edu or ftp.cis.upenn.edu. The file is called 'xv-???.tar.Z' (where ??? is the version number, currently 2.21); it is located in the 'contrib' directory on export or the 'pub/xv' directory at upenn. XV reduces all images to 8 bits internally, which means it's not a real good choice if you have a 24-bit display (you'll still get only 8-bit color). Also, you shouldn't use XV to convert full-color images to JPEG, because they'll get color-quantized first. With the exception of those two limitations, XV is as good as they come. "xli" is less featureful than XV, but it will do the right thing on 24-bit displays. xli is free and available from export.lcs.mit.edu, files contrib/xli.*. The current version is 1.10. (At last report, the files at export were a tar archive of 1.08 and a file of patches to bring 1.08 up to 1.10; be sure to retrieve both files.) Alternately, you can use xloadimage in combination with the free JPEG software described in 6B. Another good choice for X Windows is John Cristy's free ImageMagick package, also available from export, file contrib/ImageMagick.tar.Z. The viewer included in this package handles 24-bit displays correctly; for colormapped displays, it does better (though slower) color quantization than XV or the basic free JPEG software. MS-DOS: There are at least two freeware JPEG viewers for plain MS-DOS (non-Windows). One is Eric Praetzel's DVPEG. The current version, 2.0 beta, is available by FTP from sunee.waterloo.edu (129.97.50.50), file pub/jpeg/viewers/dvpeg2.zip. This is a good basic viewer that works on either 286 or 386/486 machines. The user interface is not flashy, but it's functional (it does more than Hiview, for instance). Does not work with some display cards. A more recent arrival is Mohammad Rezaei's Hiview. The current version, 1.2, is available from Simtel20 and mirror sites (see NOTE below), file msdos/graphics/hv12.zip. Hiview is noticeably faster than DVPEG and works on a fairly wide variety of display types. However, Hiview requires a 386 or better CPU and a VCPI-compatible memory manager (QEMM386 and 386MAX work; Windows and OS/2 do not). Also installation is a bit tricky; read the directions carefully! If neither of these viewers work on your hardware, you'll need to use one of the following conversion programs to convert JPEG to GIF, then view with your favorite GIF viewer. (If you have hi-color hardware, don't use GIF as the intermediate format; try to find a TARGA-capable viewer instead. VPIC5.0 is reputed to do the right thing with hi-color displays.) The Independent JPEG Group's free JPEG converters are FTPable from Simtel20 and mirror sites (see NOTE below), file msdos/graphics/jpeg3.zip (or jpeg3386.zip if you have a 386 and extended memory). The same files were posted to comp.binaries.ibm.pc (volume 18, issues 123-130) and should be available from any c.b.i.p archive site. These files are DOS compilations of the free source code described in section 6B. Handmade Software offers two rather pricy shareware programs: Image Alchemy and GIF2JPG/JPG2GIF (contact hsi@netcom.com for details). The PC versions of these programs are FTPable from Simtel20 and mirror sites (see NOTE below), files msdos/graphics/alch16.zip and gif2jpg5.zip. GIF2JPG/JPG2GIF only performs JPEG<=>GIF format conversion. Image Alchemy converts files between these and many other formats, and can also display images on some types of hardware. The display option is limited and not very high quality, so you'll still want a separate viewer program. (CAUTION: GIF2JPG produces a proprietary file format unless you specify -j. Be sure to use -j if you want to exchange JPEG files with other Usenet users. For that matter, it's not real clear that you should be posting JPEG files made from GIFs; see section 5.) In my biased opinion, the free JPEG software is a better choice than GIF2JPG/JPG2GIF; it's faster, as good or better image quality, and free :-). On the other hand, Image Alchemy may be worth its price, if you need the additional conversion and image manipulation capabilities it provides. NOTE ABOUT SIMTEL20: The Internet's key archive site for PC-related programs is Simtel20, full name wsmr-simtel20.army.mil (192.88.110.20). Simtel20 runs a non-Unix operating system; where this document refers to directory (eg) "msdos/graphics" at Simtel20, that really means "pd1:". If you are not physically on MILnet, you should expect rather slow FTP transfer rates from Simtel20. There are several Internet sites that maintain copies (mirrors) of the Simtel20 archives; most FTP users should go to one of the mirror sites instead. A popular USA mirror site is oak.oakland.edu (141.210.10.117); it keeps Simtel20 files in (eg) "/pub/msdos/graphics". If you have no FTP capability, you can retrieve files from Simtel20 by e-mail; see informational postings in comp.binaries.ibm.pc.archives to find out how. If you are outside the USA, consult the same newsgroup to learn where your nearest Simtel20 mirror is. Microsoft Windows: There are several Windows programs capable of displaying JPEG images. JView is the newest kid on the block. It's freeware, quite fast, has good on-line help, and can write out the decompressed image in Windows BMP format; but it can't create new JPEG files. JView lacks some useful features of the shareware viewers (such as brightness adjustment), but it's an excellent basic viewer. The current version, 0.9, is available from ftp.cica.indiana.edu (129.79.20.84), file pub/pc/win3/desktop/jview090.zip. (Mirrors of this archive can be found at some other Internet sites, including wuarchive.wustl.edu.) WinJPEG can display GIF, Targa, and BMP files as well as JPEG; it can write all of these formats too, so it can be used as a converter. It has some other nifty features including color-balance adjustment and slideshow. On the minus side, it lacks on-line help, the current version is a little slower than the current version of JView, and it's shareware (only $15 though). The current version is 1.2, available from Simtel20 and mirror sites (see NOTE above), file msdos/windows3/winjp120.zip. ColorView is another shareware entry ($30). This was an early and promising contender, but it has not been updated in some time, and at this point it has no real advantages over WinJPEG. If you want to try it anyway, the current version is 0.97, available from ftp.cica.indiana.edu, file pub/pc/win3/desktop/cview097.zip. The DOS conversion programs described above will run inside a Windows DOS window. Note that Windows viewers are generally slower than non-Windows viewers on the same hardware, due to Windows' system overhead. Macintosh: Most Mac JPEG programs rely on Apple's JPEG implementation, which is part of the QuickTime system extension; so you need to have QuickTime installed. To use QuickTime, you need a 68020 or better CPU and you need to be running System 6.0.7 or later. (If you're not running System 7, you must also install the 32-bit QuickDraw extension.) You can get QuickTime from ftp.apple.com, file dts/mac/quicktime/quicktime.hqx. Apple has released a free program called PictPixie that can convert the Usenet-standard JFIF JPEG format to and from QuickTime's internal JPEG format. PictPixie can also be used as a viewer for JFIF, QuickTime JPEG, and GIF files. You can get PictPixie from ftp.apple.com, file dts/mac/quicktime/pictpixie.hqx. Requires QuickTime. PictPixie is fast but requires lots of memory, and it has a relatively unfriendly user interface (it was intended as a developer's tool). PictPixie is an unsupported program, meaning it has some minor bugs that Apple does not intend to fix. (The QuickTime Starter Kit includes a less-buggy descendant of PictPixie called PICTCompressor. PICTCompressor is reputed to have a cleaner but less flexible user interface.) Another good choice is JPEGView, a free program for viewing both JFIF and QuickTime JPEG files, as well as converting between the two formats. The current version, 1.1, is much improved over the initial release (0.9). Get it from sumex-aim.stanford.edu, file /info-mac/app/jpeg-view-11.hqx. Requires System 7 and QuickTime. JPEGView doesn't do quite as much as PictPixie, but it can open multiple pictures and has a slide-show feature; so it is much handier than PictPixie for skimming through lots of images. JPEGView also needs less memory than PictPixie to view large images. On 8-bit-color displays, JPEGView's image quality is inferior to PictPixie (see below for details), but that will be fixed in the next release. Storm Technology has released a free JPEG viewer/converter called Picture Decompress. This is much inferior to PictPixie or JPEGView in speed, features, and memory demands, but it will run without System 7 or QuickTime, so you may be forced to use it on older systems. (You'll still need 32-bit QuickDraw.) This program can be FTPed from sumex-aim.stanford.edu, file /info-mac/app/picture-decompress-201.hqx. Make sure you get version 2.0.1 or later; earlier versions are not compatible with JFIF file format. You'll also need a tool for adjusting file type codes; you must set the type of a downloaded image file to 'JPEG' to allow Picture Decompress to open it. On 8-bit-color Macs, PictPixie produces visibly better results than the other two programs, since it uses QuickTime's slower 2-pass color-reduction algorithm, instead of the quick-and-dirty 1-pass method. Many people think that the IJG JPEG quantizer does better color reduction than either of QuickTime's methods. Real Soon Now there should be at least one Mac program based on the IJG JPEG code, which will be slower but higher quality than QuickTime. The shareware image viewer/converter GIFConverter will support JPEG in its next release. There is already a beta version out (2.3b1), but it is pretty flaky, so I recommend waiting for the real release. Once stable, this program should offer very nice viewing and format-conversion features. More and more commercial Mac applications are supporting JPEG, although not all can deal with the Usenet-standard JFIF format. Adobe Photoshop, version 2.0.1 or later, can read and write JFIF-format JPEG files (use the JPEG plug-in from the Acquire menu). You must set the file type of a downloaded file to 'JPEG' to allow Photoshop to recognize it. Amiga: The shareware program HamLab Plus is probably the best inexpensive JPEG viewer/converter for Amigas. It's cheap ($20) and can read several formats besides JPEG. The current version is 2.0.8. A demo version is available by FTP from amiga.physik.unizh.ch (130.60.80.80) and mirror sites (including wuarchive.wustl.edu in the USA), file amiga/gfx/hamlab208d.lha. The demo version will crop images larger than 512x512, but it is otherwise fully functional. If you're willing to spend real money, the commercial program Art Department Professional is a very nifty piece of software that handles JPEG. The Amiga world is heavily infested with quick-and-dirty JPEG programs, many based on an ancient beta-test version of the free IJG JPEG software (thanks to a certain magazine that published same on its disk-of-the-month, without so much as notifying the authors). Among these are "AugJPEG", "NewAmyJPEG", "VJPEG", and probably others I have not even heard of. In my opinion, anything older than IJG version 3 (March 1992) is not worth the disk space it's stored on; if you have such a program, trash it and get something newer. Acorn Archimedes: !ChangeFSI, supplied with RISC OS 3 version 3.10, can convert from and view JPEG JFIF format. Provision is also made to convert images to JPEG, although this must be done from the CLI rather than by double-clicking. There's also a product called !JPEG which provides JPEG read/write functionality and direct JPEG viewing, as well as a host of other image format conversion and processing options. Contact: DT Software, FREEPOST, Cambridge, UK. Tel: 0223 841099. Portable software for almost any system: If none of the above fits your situation, you can obtain and compile the free JPEG conversion software described in 6B. You'll also need a viewer program. If your display is 8 bits or less, any GIF viewer will do fine; if you have a display with more color capability, try to find a viewer that can read Targa or PPM 24-bit image files. If you are not reasonably handy at configuring and installing portable C programs, you may have some difficulty installing the free source code. Steve Davis (strat@cis.ksu.edu) has volunteered to maintain an archive of pre-built executable versions of the free JPEG code for various machines. His FTP archive is at ftp.cis.ksu.edu (129.130.10.80); look under /pub/JPEG to see what he currently has. (The administrators of this system ask that FTP traffic be limited to non-prime hours.) This archive is not maintained by the Independent JPEG Group, and files in it may not represent the latest free source code. (Actually, Steve has gotten pretty lax about maintaining his archive. Any volunteers to set up a new one?) There are numerous commercial JPEG offerings, with more popping up every day. I recommend that you not spend money on one of these unless you find the available free or shareware software vastly too slow. In that case, purchase a hardware-assisted product. Ask pointed questions about whether the product complies with the final JPEG standard and about whether it can handle the JFIF file format; many of the earliest commercial releases are not and never will be compatible with anyone else's files. 6B) If you are looking for source code to work with: Free, portable C code for JPEG compression is available from the Independent JPEG Group, which I lead. A package containing our source code, documentation, and some small test files is available from several places. The "official" archive site for this source code is ftp.uu.net (137.39.1.9 or 192.48.96.9). Look under directory /graphics/jpeg; the current release is jpegsrc.v3.tar.Z. (This is a compressed TAR file; don't forget to retrieve in binary mode.) You can retrieve this file by FTP or UUCP. Folks in Europe may find it easier to FTP from nic.funet.fi (see directory pub/graphics/programs/jpeg). The source code is also available on CompuServe, in the GRAPHSUPPORT forum (GO PICS), library 10, as jpsrc3.zip. If you have no FTP access, you can retrieve the source from your nearest comp.sources.misc archive; version 3 appeared as issues 1-18 of volume 29. (If you don't know how to retrieve comp.sources.misc postings, see the FAQ article "How to find sources". This appears regularly in news.answers, or you can get it by sending e-mail to mail-server@rtfm.mit.edu with "send usenet/news.answers/finding-sources" in the body.) The free JPEG code provides conversion between JPEG "JFIF" format and image files in GIF, PBMPLUS PPM/PGM, Utah RLE, and Truevision Targa file formats. The core compression and decompression modules can easily be reused in other programs, such as image viewers. The package is highly portable; we have tested it on many machines ranging from PCs to Crays. We have released this software for both noncommercial and commercial use. Companies are welcome to use it as the basis for JPEG-related products. We do not ask a royalty, although we do ask for an acknowledgement in product literature (see the README file in the distribution for details). We hope to make this software industrial-quality --- although, as with anything that's free, we offer no warranty and accept no liability. The Independent JPEG Group is a volunteer organization; if you'd like to contribute to improving our software, you are welcome to join. 7) What's all this hoopla about color quantization? Most people don't have full-color (24 bit per pixel) display hardware. Typical display hardware stores 8 or fewer bits per pixel, so it can display 256 or fewer distinct colors at a time. To display a full-color image, the computer must map the image into an appropriate set of representative colors. This process is called "color quantization". (This is something of a misnomer, "color selection" would be a better term. We're stuck with the standard usage though.) Clearly, color quantization is a lossy process. It turns out that for most images, the details of the color quantization algorithm have MUCH more impact on the final image quality than do any errors introduced by JPEG (except at the very lowest JPEG quality settings). Since JPEG is a full-color format, converting a color JPEG image for display on 8-bit-or-less hardware requires color quantization. This is *always* the case: even if you feed a 256-or-less-color GIF into JPEG, what comes out of the decompressor is *not* 256 colors, but thousands of colors. JPEG's lossiness affects each pixel a little differently, so two pixels that started as identical colors will probably come out as slightly different colors. Each original color gets "smeared" into a group of nearby colors. Therefore quantization is always required to display a color JPEG on a colormapped display, regardless of the image source. (Incidentally, because of this effect it's pretty much meaningless to talk about the number of colors used by a JPEG image. I occasionally see posted images described as "256-color JPEG". This tells me that the poster (a) hasn't read this FAQ and (b) probably converted the JPEG from a GIF.) The only way to avoid quantization is to ask for gray-scale output. On the other hand, a GIF image by definition has already been quantized to 256 or fewer colors. For purposes of Usenet picture distribution, GIF has the advantage that the sender precomputes the color quantization, so recipients don't have to. This is also the *disadvantage* of GIF: you're stuck with the sender's quantization. If the sender quantized to a different number of colors than what you can display, you have to re-quantize, resulting in much poorer image quality than if you had quantized once from a full-color image. Furthermore, if the sender didn't use a high-quality color quantization algorithm, you're out of luck. For this reason, JPEG offers the promise of significantly better image quality for all users whose machines don't match the sender's display hardware. JPEG's full color image can be quantized to precisely match the user's display hardware. Furthermore, you will be able to take advantage of future improvements in quantization algorithms (there is a lot of active research in this area), or purchase better display hardware, to get a better view of JPEG images you already have. With a GIF, you're stuck forevermore with what was sent. It's also worth mentioning that many GIF-viewing programs include rather shoddy quantization routines. If you view a 256-color GIF on a 16-color EGA display, for example, you are probably getting a much worse image than you need to. This is partly an inevitable consequence of doing two color quantizations (one to create the GIF, one to display it), but often it's also due to sloppiness. JPEG conversion programs will be forced to use high quality quantizers in order to get acceptable results at all, and in normal use they will quantize directly to the number of colors to be displayed. Thus, JPEG is likely to provide better results than the average GIF program for low-color-resolution displays as well as high-resolution ones! Finally, an ever-growing number of people have better-than-8-bit display hardware already: 15-bit "hi-color" PC displays, true 24-bit displays on workstations and Macintoshes, etc. For these people, GIF is already obsolete, as it cannot represent an image to the full capabilities of their display. JPEG images can drive these displays much more effectively. Thus, JPEG is an all-around better choice than GIF for representing images in a machine-independent fashion. 8) How does JPEG work? The buzz-words to know are chrominance subsampling, discrete cosine transforms, coefficient quantization, and Huffman or arithmetic entropy coding. This article's long enough already, so I'm not going to say more than that. For a good technical introduction, see: Wallace, Gregory K. "The JPEG Still Picture Compression Standard", Communications of the ACM, April 1991 (vol. 34 no. 4), pp. 30-44. (Adjacent articles in that issue discuss MPEG motion picture compression, applications of JPEG, and related topics.) If you don't have the CACM issue handy, a PostScript file containing a revised version of this article is available at ftp.uu.net, graphics/jpeg/wallace.ps.Z. The file (actually a preprint for an article to appear in IEEE Trans. Consum. Elect.) omits the sample images that appeared in CACM, but it includes corrections and some added material. Note: the Wallace article is copyright ACM and IEEE, and it may not be used for commercial purposes. An alternative, more leisurely explanation of JPEG can be found in "The Data Compression Book" by Mark Nelson, published by M&T Books (Redwood City, CA), 1991, ISBN 1-55851-216-0. This book provides excellent introductions to many data compression methods including JPEG, plus sample source code in C. The JPEG-related source code is far from industrial-strength, but it's a pretty good learning tool. (When you are ready to look at a real implementation, see section 6B above.) 9) What about lossless JPEG? There's a great deal of confusion on this subject. The JPEG committee did define a truly lossless compression algorithm, i.e., one that guarantees the final output is bit-for-bit identical to the original input. However, this lossless mode has almost nothing in common with the regular, lossy JPEG algorithm. At present, very few implementations of lossless JPEG exist, and all of them are commercial. Saying "-Q 100" to the free JPEG software DOES NOT get you a lossless image. What it does get rid of is deliberate information loss in the coefficient quantization step. There is still a good deal of information loss in the color subsampling step. (There should be a command line switch to disable subsampling, but as of today, there isn't one.) Even with both quantization and subsampling turned off, the regular JPEG algorithm is not lossless, because it is subject to roundoff errors in various calculations. The maximum error is a few counts in any one pixel value; it's highly unlikely that this could be perceived by the human eye, but it might be a concern if you are doing machine processing of an image. At this minimum-loss setting, regular JPEG produces files that are perhaps half the size of an uncompressed 24-bit-per-pixel image. True lossless JPEG provides roughly the same amount of compression, but it guarantees bit-for-bit accuracy. If you have an application requiring lossless storage of images with less than 6 bits per pixel (per color component), you may want to look into the JBIG bilevel image compression standard. This performs better than JPEG lossless on such images. JPEG lossless is superior to JBIG on images with 8 or more bits per pixel; furthermore, it is public domain, while the JBIG techniques are heavily covered by patents. 10) Why all the argument about file formats? Strictly speaking, JPEG refers only to a family of compression algorithms; it does *not* refer to a specific image file format. The JPEG committee was prevented from defining a file format by turf wars within the international standards organizations. Since we can't actually exchange images with anyone else unless we agree on a common file format, this leaves us with a problem. In the absence of official standards, a number of JPEG program writers have just gone off to "do their own thing", and as a result their programs aren't compatible with anybody else's. The closest thing we have to a de-facto standard JPEG format is some work that's been coordinated by people at C-Cube Microsystems. They have defined two JPEG-based file formats: * JFIF (JPEG File Interchange Format), a "low-end" format that transports pixels and not much else. * TIFF/JPEG, aka TIFF 6.0, an extension of the Aldus TIFF format. TIFF is a "high-end" format that will let you record just about everything you ever wanted to know about an image, and a lot more besides :-). TIFF is a lot more complex than JFIF, and may well prove less transportable, because different vendors have historically implemented slightly different and incompatible subsets of TIFF. It's not likely that adding JPEG to the mix will do anything to improve this situation. Both of these formats were developed with input from all the major vendors of JPEG-related products; it's reasonably likely that future commercial products will adhere to one or both standards. A particular case that people may be interested in is Apple's QuickTime software for the Macintosh. QuickTime uses a JFIF-compatible format wrapped inside the Mac-specific PICT structure. Conversion between JFIF and QuickTime JPEG is pretty straightforward; in fact Apple has released a utility program for the purpose (see PictPixie in section 6A). I believe that Usenet should adopt JFIF as the replacement for GIF in picture postings. JFIF is simpler than TIFF and is available now; the TIFF 6.0 spec has only recently been officially adopted, and it is still unusably vague on some crucial details. Even when TIFF/JPEG is well defined, the JFIF format is likely to be a widely supported "lowest common denominator"; TIFF/JPEG files may never be as transportable. 11) And what about arithmetic coding? The JPEG spec defines two different "back end" modules for the final output of compressed data: either Huffman coding or arithmetic coding is allowed. The choice has no impact on image quality, but arithmetic coding usually produces a smaller compressed file. On typical images, arithmetic coding produces a file 5 or 10 percent smaller than Huffman coding. (All the file-size numbers previously cited are for Huffman coding.) Unfortunately, the particular variant of arithmetic coding specified by the JPEG standard is subject to patents owned by IBM, AT&T, and Mitsubishi. Thus *you cannot legally use arithmetic coding* unless you obtain licenses from these companies. (The "fair use" doctrine allows people to implement and test the algorithm, but actually storing any images with it is dubious at best.) At least in the short run, I recommend that people not worry about arithmetic coding; the space savings isn't great enough to justify the potential legal hassles. In particular, arithmetic coding *should not* be used for any images to be exchanged on Usenet. There is some small chance that the legal situation may change in the future. Stay tuned for further details. 12) Does loss accumulate with repeated compression/decompression? It would be nice if, having compressed an image with JPEG, you could decompress it, manipulate it (crop off a border, say), and recompress it without any further image degradation beyond what you lost initially. Unfortunately THIS IS NOT THE CASE. In general, recompressing an altered image loses more information, though usually not as much as was lost the first time around. The next best thing would be that if you decompress an image and recompress it *without changing it* then there is no further loss, i.e., you get an identical JPEG file. Even this is not true; at least, not with the current free JPEG software. It's essentially a problem of accumulation of roundoff error. If you repeatedly compress and decompress, the image will eventually degrade to where you can see visible changes from the first-generation output. (It usually takes many such cycles to get visible change.) One of the things on our to-do list is to see if accumulation of error can be avoided or limited, but I am not optimistic about it. In any case, the most that could possibly be guaranteed would be that compressing the unmodified full-color output of djpeg, at the original quality setting, would introduce no further loss. Even such simple changes as cropping off a border could cause further roundoff-error degradation. (If you're wondering why, it's because the pixel-block boundaries move. If you cropped off only multiples of 16 pixels, you might be safe, but that's a mighty limited capability!) The bottom line is that JPEG is a useful format for archival storage and transmission of images, but you don't want to use it as an intermediate format for sequences of image manipulation steps. Use a lossless format (PPM, RLE, TIFF, etc) while working on the image, then JPEG it when you are ready to file it away. Aside from avoiding degradation, you will save a lot of compression/decompression time this way :-). 13) What are some rules of thumb for converting GIF images to JPEG? As stated earlier, you *will* lose some amount of image information if you convert an existing GIF image to JPEG. If you can obtain the original full-color data the GIF was made from, it's far better to make a JPEG from that. But if you need to save space and have only the GIF to work from, here are some suggestions for getting maximum space savings with minimum loss of quality. The first rule when converting a GIF library is to look at each JPEG, to make sure you are happy with it, before throwing away the corresponding GIF; that will give you a chance to re-do the conversion with a higher quality setting if necessary. Some GIFs may be better left as GIFs, as explained in section 5; there are images for which a JPEG file of reasonable quality will be *larger* than a GIF. (So check the sizes too.) Experience to date suggests that large, high-visual-quality GIFs are the best candidates for conversion to JPEG. They chew up the most storage so offer the most potential savings, and they convert to JPEG with least degradation. Don't waste your time converting any GIF much under 100 Kbytes. Also, don't expect JPEG files converted from GIFs to be as small as those created directly from full-color originals. To maintain image quality you may have to let the converted files be as much as twice as big as straight-through JPEG files would be (i.e., shoot for 1/2 or 1/3rd the size of the GIF file, not 1/4th as suggested in earlier comparisons). cjpeg's default Q setting of 75 is appropriate for full-color input, but for GIF inputs, Q settings of 85 to 95 often seem to be necessary to avoid image degradation. Many people have developed an odd habit of putting a large constant-color border around a GIF image. While useless, this was nearly free in terms of storage cost in GIF files. It is NOT free in JPEG files, and the sharp border boundary can create visible artifacts ("ghost" edges). Do yourself a favor and crop off any border before JPEGing. (If you are on an X Windows system, XV's manual and automatic cropping functions are a very painless way to do this.) Sometimes, smoothing a GIF before compression will reduce the JPEG file size *and* improve the output image quality. The theory is that smoothing reduces the dithering patterns found in most color GIFs; this helps because dithering creates high-spatial-frequency noise which JPEG doesn't handle well. If you can see regular fine-scale patterns on the GIF image, then smoothing is definitely indicated. At some point good JPEG software will probably include an input-smoothing option, but for now, you'll have to use external tools. If you have the PBMPLUS package, try giftoppm input.gif | pnmconvol weightfile | cjpeg >output.jpg where weightfile contains P2 3 3 200 101 101 101 101 192 101 101 101 101 (Thanks to Jef Poskanzer for these values.) For GIFs with very heavy-handed dithering, the stronger smoothing provided by pnmsmooth may work better. You may be able to drop down to Q 75 or less without visible quality loss if you smooth the input image this way. --------------------- For more information about JPEG in general or the free JPEG software in particular, contact the Independent JPEG Group at jpeg-info@uunet.uu.net. -- tom lane organizer, Independent JPEG Group Internet: tgl@cs.cmu.edu BITNET: tgl%cs.cmu.edu@carnegie