Here are the results of 10 tests performed in June 2000 to compare lossless compression of True Color (24-bit) images by all known programs developed for such purpose, including BMF, JPEG-LS and PNG software, 7-Zip, Pkzip, and archivers with special algorithms for 24-bit images. See Archive Comparison Test by Jeff Gilchrist for more details: http://ACT.i.am If anybody wants to start or continue such tests, or can suggest some other sets of images, or other compression programs, (not sources or algorithm descriptions, programs for DOS or Windows only) or knows we have missed something important, (some new fantastic technology, an algorithm or even a program capable of lossless compression of up to 1000:1 and lossy compression of up to 10000:1, we hear about it from time to time since our XTs) please let us know immediately: ratush@srsc-gw.sscc.ru Thank you!
The Art of lossless image compression vol. 16 [[1]] COMPRESSION QUALITY
(see also [[2]] Time [[3]] Details [[4]] Comments) The last eleventh line shows results for the sum of all ten tests (336 images) Original BMF RKim RK UHIC ArHanGeL NK Eri RAR Imp 275.95% 100% 101.59 120.20 100.55 110.52 112.65 117.81 133.15 154.65 289.11% 100% 124.50 127.28 104.88 114.61 135.54 122.36 140.60 152.88 245.22% 100% 101.38 123.16 107.93 112.92 111.48 123.50 124.25 144.52 238.64% 100% 112.66 124.20 104.20 110.53 120.45 121.23 132.94 146.59 457.28% 101.93 100% 133.41 110.74 112.34 120.22 125.22 146.11 157.63 234.93% 100% 101.72 119.42 107.07 110.08 106.85 121.92 125.63 143.10 278.95% 100% 103.17 123.14 103.40 109.60 110.16 119.57 135.88 146.37 332.49% 100% 102.12 126.63 105.92 111.58 112.18 120.18 139.49 153.56 225.19% 100% 104.47 118.81 104.00 108.82 110.47 118.20 122.22 141.08 547.50% 100% 104.25 131.46 104.84 107.76 127.75 115.26 170.38 157.56 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 286.32% 100% 102.98 122.95 105.47 110.35 111.15 120.43 133.89 147.68 JPEG-LS SPIHT BTPC { Display } SEA PngCrush 7-Zip PkZip 149.81% 115.50 164.36 157.93 156.06 153.89 150.94 150.33 159.16 169.90 156.41% 136.11 170.42 177.21 169.62 164.86 163.71 151.73 205.21 211.56 119.20% 111.39 129.21 140.98 141.56 139.53 146.28 129.83 193.86 196.55 127.45% 122.73 137.49 146.42 145.46 144.00 147.31 134.68 182.24 185.85 172.49% 224.13 201.52 187.28 156.67 156.16 160.57 145.35 185.82 188.99 112.56% 111.61 119.89 129.58 132.73 129.30 137.85 120.59 170.09 172.74 118.69% 119.12 131.29 140.90 139.04 137.13 143.73 129.64 171.53 175.85 123.33% 117.86 136.63 145.92 143.78 142.44 151.77 133.46 197.76 202.49 117.84% 109.95 122.53 133.78 135.20 132.58 139.93 121.86 179.50 181.44 151.17% 132.31 176.94 164.21 150.26 148.74 156.87 142.19 186.28 195.55 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 121.48% 117.50 132.78 140.67 139.78 137.42 144.92 128.95 179.40 183.54[[2]] Time
A set of 230 images totalling 271,331,460 bytes from ftp://ipl.rpi.edu/pub/image/still/KodakImages/color/ was used for this test and an AMD-K6-400 machine with 64M RAM and Windows98. Well-known 23 Kodak images copied ten times (see icopy.bat in artest16.zip) Programs,options Overall Average Compress Extract Compressed score, Users' time, time, size, seconds score seconds seconds bytes +7-zip a 44471 43736 817 137 156662222 +7-zip a -mx 44960 43743 1352 138 156492462 +arhangel a -mm -mz 34295 31998 2552 1557 108669974 +arhangel a -mm 34260 31989 2523 1551 108669974 +arhangel a -mm1 48908 45779 3476 2633 154074824 +arhangel a -mm3 42870 40214 2951 1879 136944664 bmf -f -s -q9 38628 30978 8500 2816 98323280 bmf -f -s 33477 30467 3345 2820 98323280 bmf -f -q9 33681 29829 4280 249 104947560 bmf -f 30324 29572 836 258 105229600 eri a -3ma8 31885 30118 1963 449 106102020 eri a -3ma8 -m3 31413 30078 1483 435 106182410 eri a -3ma8 -m2 31255 30290 1072 355 107379300 eri a -3ma8 -m1 32073 31299 860 299 111292130 +imp98 a -2 38040 37366 749 222 133449566 +imp98 a -2 -s4 38123 37453 744 218 133779179 +imp98 a -mm 43242 42363 977 110 151758474 +imp98 a -mm -s4 43204 42299 1006 109 151521544 +imp98 a -mm -m3 43528 42174 1505 119 150856163 +imp98 a -mm -m1 46105 45522 648 111 163246850 +imp-dos a -mm -m3 43565 42194 1524 137 150855710 Nk 35338 31666 4080 489 110767370 pkzip -exx 46871 46394 530 146 166302252 pkzip -a 46708 46380 365 146 166309392 pkzip -es 55426 55211 238 152 198128322 +pkzip32 -add -max 47059 46592 519 138 167047382 +pkzip32 -add 47018 46662 396 136 167349362 +WinRar a -mm -mde 37138 36685 504 259 130951080 +WinRar a -mm -m5 37149 36678 523 259 130921330 +WinRar a -mm -s- 37137 36685 503 259 130951080 +WinRar a -mm -s 37124 36648 529 258 130813550 +WinRar a -mmf 36740 36324 462 267 129638820 +rar32 a -mm -mde 37254 36731 582 297 130951080 +rar32 a -mm -m5 37263 36724 599 297 130921330 +rar32 a -mm -s- 37253 36730 581 297 130951080 +rar32 a -mm -s 37236 36692 605 294 130813550 +rar32 a -mmf 36860 36371 543 306 129638820 +rk -mx2 27554 16839 11906 12305 12035652 +rk -mx1 46827 37353 10527 11174 90452920 +rk -mf3 7011 5469 1713 1191 14784112 +rk -mf2 7863 6571 1435 1176 18906828 +rk -mf1 39048 37937 1235 887 132934712 +rkim cx 35399 30769 5144 2506 99896080 +rkim c 32861 31261 1778 2178 104059250 uhic m 36260 32373 4319 4477 98870920 uhic e 32812 30565 2496 2501 100132460 Alchemy ---n 46843 44143 3000 526 155940956 ---n 3 46602 44584 2242 648 157361419 ---n 2 46338 44670 1853 437 158572986 ---n 1 45301 43610 1879 434 154758370 SEA /png:100 42487 41480 1119 142 148414950 Display -i --same -bpng + filter=5,level=9 44739 42446 2547 159 151317180 + filter=5,level=1 44203 43805 442 168 156934080 + filter=1,level=9 44739 42537 2447 149 151714120 + filter=1,level=5 43955 43263 769 150 154930330 + filter=1,level=1 44605 44292 348 156 158763080 +pngcrush -q 49026 42265 7512 455 147813040 +pngcrush -brute 165325 53632 124103 460 146744530 +btpc 100 45618 45209 454 273 161606980 +lsp -g 42530 42119 456 375 150115060 +lsp 32507 32034 526 434 113570440 +locoe -N -c1 41544 41271 303 323 147305210 +locoe -N -c2 41732 41406 362 342 147701630 -PhotoJazz ????? ????? ??? ??? 111673670 + indicates new lines in current volume of ARTest - indicates oldest lines, that will not be present in next volume. Overall score is calculated by adding compression time, extraction time, and time it would take to transfer the compressed file over a 28,800bps network: (compressed_size)/3600 , because 28800 bits_per_second is 3600 bytes_per_second Average Users' score is calculated by adding (compress_time/10)+ extract_time + time it would take to transfer the compressed file over a 28,800bps network. Compression time is divided by 10 here, because more than 90% of people would never compress anything during their life (with compression programs), but they use compressed data almost _every_ time they use computers and/or Internet. That's why compression time is not so actual for them.[[3]] Details
are no longer put to this main text (380 lines reporting 10080 results on 336 files in 10 sets), but can be found in FULL version with DETAILS.DAT and *.BAT at http://geocities.com/SiliconValley/Bay/1995/artest16.zip or http://artest1.tripod.com/artest16.zip 1,2,3,4,9 image sets were taken from ftp://ipl.rpi.edu/pub/image/ and /image2/ converted from Sun Raster format to Windows Bitmap, (this conversion changes image headers only, 32 bytes are replaced by 54 bytes, but image data is not modified) This site ipl.rpi.edu is described in comp.compression FAQ by Jean-loup Gailly, in 29 lines of "Subject: [55] Where can I find Lenna and other images?" Eight images in the 5th set are from the famous Waterloo Color Set, ( losslessly compressed with Eri32 4.5fre to 2,894,045 bytes, it can be found at http://geocities.com/eri32 ) 6,7,8th image sets are from ftp://download.intel.com/pressroom/images/ 10th set was downloaded from ftp://ftp.kodak.com/www/images/dental/ They were converted from TIFF format to Windows Bitmap. Some images were slightly renamed. See DETAILS.DAT from artest16.zip for more details. Some new images (from download.intel.com) were not added yet.[[4]] Comments
Links to download programs:
7-Zip 2.11 :W http://www.7-zip.com/dl/7zip211.exe 493K ArHanGeL 1.40 :a http://geocities.com/SiliconValley/Lab/6606/arh140.zip 50K BMF 1.1 :e ftp://ftp.simtel.net/pub/simtelnet/win95/graphics/bmf_1_10.zip 132K ERI32 4.5fre :e http://geocities.com/eri32/eri45fre.zip 90K Imp-dos 1.1 :e http://www.winimp.com/imp110d.zip 266K Imp 1.12 :W http://www.winimp.com/imp112.exe 122K PkZip 2.50 :a ftp://ftp.simtel.net/pub/simtelnet/msdos/arcers/pk250dos.exe 202K PkZip32 2.50 :W ftp://ftp.simtel.net/pub/simtelnet/win95/compress/pk250c32.exe 300K RK 1.02a5 :W http://malcolmt.tripod.com/downloads/rk102a05.exe 191K RKim 1.06 :W http://malcolmt.tripod.com/downloads/rkim106.zip 57K RAR32 2.71 :e ftp://ftp.netlab.sk/public/rarsoft/rar/rarx271.exe 257K WinRAR 2.71 :W ftp://ftp.netlab.sk/public/rarsoft/rar/wrar271.exe 588K UHIC 1.0 :e ftp://ftp.elf.stuba.sk/pub/pc/pack/uhic10.zip 52K NK Nov'99 :W ftp://tu.tusur.ru/pub/ImageTools/compress.zip 81K JPEG-LS 1.00 :W http://www.hpl.hp.com/loco/jlsrefV100.zip 325K PngCrush 1.4.4:e http://download.sourceforge.net/pmt/pngcrush-1.4.4-dos.zip 123K SPIHT Nov'96 :W http://www.cipr.rpi.edu/research/SPIHT/EW_Code/lsp.exe 72K BTPC 4.1 :W http://www.engr.mun.ca/~john/btpc4.1.zip 715K Display 1.90b5:e http://www.freenet.edmonton.ab.ca/~crnelson/disp.exe 811K Alchemy 1.10 :a ftp://ftp.elf.stuba.sk/pub/pc/graph/alch110.zip 1462K SEA 1.3 :e ftp://ftp.elf.stuba.sk/pub/pc/graph/sea13.zip 705K :a - DOS programs, will run under pure DOS or in a Windows DOS box :e - DOS programs using DOS extenders like DOS/4GW or CWSDPMI :W - Windows95/98/NT programs If direct link doesn't work-most probably newer version of the program appeared at the same site: visit web page, or read the whole directory from ftp server (i.e. try the same URL, but without filename).Homepages:
Arhangel : http://geocities.com/SiliconValley/Lab/6606 Eri32 : http://geocities.com/eri32 mirror : http://artest1.tripod.com RK, RKim : http://malcolmt.tripod.com Imp,WinImp : http://www.technelysium.com.au mirror : http://www.winimp.com PkZip : http://www.pkware.com Ufa,777,7-Zip: http://www.7-zip.com RAR,WinRAR : http://www.rarsoft.com JPEG-LS : http://www.hpl.hp.com/loco PngCrush : http://pmt.sourceforge.net/pngcrush SPIHT : http://www.ipl.rpi.edu/research/SPIHT BTPC : http://www.engr.mun.ca/~john/btpc.html Display : http://fn2.freenet.edmonton.ab.ca/~crnelson/display.html Alchemy : http://www.handmadesw.com/Products/Image_Alchemy.htm SEA : http://www.photodex.com/products/dosWhat's new:
14 new programs were tested: 7-Zip 2.11, ArHanGeL 1.40, JPEG-LS 1.00, RK 1.02a5, RKim 1.06, PkZip32 2.5, Imp 1.1 for DOS, Imp 1.12 for Windows95/98/NT, BTPC 4.1, PngCrush 1.4.4, WinRAR 2.70, RAR32 for DOS and OS/2 2.70, SPIHT, Display 1.90b5. 33 new images from ftp://download.intel.com/pressroom/images were added to 6th, 7th and 8th sets. Two of them turned out to be 8-bit grayscale. UHIC was the only program unable to compress grayscale images, and sizes of uncompressed files were added, as if it could store them without compression.WARNINGS:
ESP 1.92 has some bugs and was unable to correctly uncompress some test images: they are (re)named in our tests as AYMAR,GELSNGR,NACHTSH,OTELLIN,VADASZ,YASSO,YU1, DP08189,DP2179D,L217C-3,CONCEP3,PS09157,V42798C. ESP doesn't even report something like "CRC fails"...as if everything is OK. Rkive 1.92b1 fails to losslessly uncompress most images from 6th, 7th and 8th tests, and also Clegg, Frymire, Serrano from 5th. It reports ERROR: File fails CRC check!. Archive failed integrity check. ESP, Rkive and some other programs are not tested any more, their results and links can be found in previous versions: ESP - artest3...artest12 Rkive - 3...12 UFA - 3...12 777 - 13...14 BOA - 7...14 UHARC - 5...10 CompuShow 2000! - 3...11 The LATEST RELEASE, and thirteen previous versions of these tests can be found at http://geocities.com/SiliconValley/Bay/1995/ and http://artest1.tripod.com/The FINAL PART
> [[5]] PLEASE read THIS before replying to this article was removed from this text, but can be easily found at http://geocities.com/SiliconValley/Bay/1995/artest10.html http://artest1.tripod.com/artest10.html Send your suggestions, comments to ratush@srsc-gw.sscc.ru With best kind regards, RAO Inc.