The Art of lossless image compression vol. 15
Here are the results of 10 tests performed in November-December 1999 to compare lossless compression of True Color (24-bit) images by HP Labs' software implementation of JPEG-LS, by popular PkZip, 7Zip, Boa, BMF, UHIC, by archivers with _special_ algorithms for 24-bit images: Eri32,ArHanGeL,Ufa32,777,ESP,Rkive,RKim,RK,UHarc,Imp,RAR,WinRAR, and results on same images of the famous PNG algorithm and format created by PngCrush, Display 1.89a, SEA 1.3, Image Alchemy 1.10 See Archive Comparison Test by Jeff Gilchrist for more details: http://act.by.net/ If anybody wants to start or continue such tests, or can suggest some other sets of images, or other compression programs, (not sources or algorithm descriptions, programs for DOS or Windows only) or knows we have missed something important, (some new fantastic technology, an algorithm or even a program capable of lossless compression of up to 1000:1 and lossy compression of up to 10000:1 , we hear about it from time to time since our XTs) please let us know immediately: ratush@srsc-gw.sscc.ru Thank you![[1]] COMPRESSION QUALITY
(see also [[2]] Time [[3]] Details [[4]] Comments) The last eleventh line shows results for the sum of all ten tests (303 images) Original BMF RKim RK UHIC ArHanGeL NK Eri RAR Imp 275.95% 100% 101.71 120.20 100.55 110.52 112.65 117.81 135.03 154.65 289.11% 100% 124.97 127.28 104.88 114.61 135.54 122.36 140.58 152.88 245.22% 100% 101.61 123.16 107.93 112.92 111.48 123.50 124.40 144.52 238.64% 100% 113.35 124.20 104.20 110.53 120.45 121.23 134.57 146.59 448.61% 100% 138.97 130.88 108.64 110.21 117.94 122.84 155.31 154.64 237.91% 100% 101.95 120.45 105.20 110.49 106.65 121.87 128.47 144.18 283.29% 100% 104.03 122.94 104.06 110.26 110.86 120.60 137.53 147.66 309.74% 100% 102.03 127.03 106.25 111.90 112.06 120.65 139.49 154.26 225.19% 100% 104.61 118.81 104.00 108.82 110.47 118.20 123.11 141.08 547.50% 100% 104.05 131.46 104.84 107.76 127.75 115.26 174.60 157.56 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 288.02% 100% 103.94 123.65 105.03 110.73 111.84 120.60 136.45 148.95 JPEG-LS { Display } SEA PngCrush 7Zip PkZip 149.81% 154.30 158.67 157.93 156.06 153.89 150.94 150.53 159.16 169.14 156.41% 153.58 186.20 177.21 169.62 164.86 163.71 151.56 205.21 210.53 119.20% 145.60 151.16 140.98 141.56 139.53 146.28 129.55 193.86 196.15 127.45% 142.67 148.25 146.42 145.46 144.00 147.31 134.10 182.24 185.48 169.22% 157.12 156.38 183.73 153.70 153.19 157.52 141.51 182.29 185.19 114.03% 140.20 136.69 131.69 135.10 131.44 139.37 122.31 173.38 176.24 121.09% 147.00 144.58 142.62 142.07 140.13 146.61 131.22 176.27 180.97 121.86% 153.19 150.82 144.91 143.49 142.10 152.03 132.68 199.74 204.38 117.84% 137.05 140.75 133.78 135.20 132.58 139.93 121.89 179.50 181.40 151.17% 159.42 153.92 164.21 150.26 148.74 156.87 141.66 186.28 195.49 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 123.39% 147.22 146.18 142.74 141.93 139.57 146.85 130.50 182.92 187.28[[2]] Time
A SET of 230 images totalling 271,331,460 bytes from ftp://ipl.rpi.edu/pub/image/still/KodakImages/color/ was used for this test and an AMD-Pr-350 machine with 64M RAM running Windows98 Well-known 23 Kodak images copied ten times (see icopy.bat in artest15.zip) Programs,options Overall Average Compress Extract Compressed score, Users' time, time, size, seconds score seconds seconds bytes -777 a -m4 63352 48757 16216 10465 132018518 7zip a -mx 45029 43778 1390 169 156492462 7zip a 44443 43761 757 169 156662222 +arhangel a -mm -mz 33079 31471 1786 1107 108669974 +arhangel a -mm 33059 31469 1766 1107 108669974 +arhangel a -mm1 46796 44770 2251 1747 154074824 +arhangel a -mm3 41337 39555 1980 1317 136944664 bmf -f -s -q9 37166 30332 7593 2261 98323280 bmf -f -s 32252 29800 2724 2216 98323280 bmf -f -q9 32534 29731 3114 268 104947560 bmf -f 30243 29584 732 281 105229600 -boa -m15 60564 48628 13259 13872 120353098 +eri a -3ma8 31705 30076 1809 424 106102020 +eri a -3ma8 -m3 31253 30040 1347 411 106182410 +eri a -3ma8 -m2 31094 30253 934 333 107379300 +eri a -3ma8 -m1 31959 31272 763 282 111292130 -esp /m0 /mm2 /ml7 36738 35441 1438 132 126599845 -esp /m1 /mm2 /ml7 36572 35474 1218 131 126800674 -esp /m2 /mm2 /ml7 36304 35488 906 129 126968539 -esp /m3 /mm2 /ml7 38469 38033 483 133 136268163 +imp98 a -2 37890 37360 588 231 133457225 +imp98 a -2 -s4 38003 37458 605 238 133779384 +imp98 a -mm 43094 42369 805 134 151758584 +imp98 a -mm -s4 43062 42305 841 132 151521710 +imp98 a -mm -m3 43294 42174 1244 146 150856308 +imp98 a -mm -m1 46104 45545 621 137 163246816 +imp-dos a -mm -m3 43339 42202 1263 172 150855765 +Nk 35441 31720 4134 539 110767370 pkzip -exx 46766 46407 398 173 166302252 pkzip -a 46687 46399 318 171 166309392 pkzip -es 55425 55221 225 164 198128322 +WinRar a -mm -mde 37679 37190 543 283 132671570 +WinRar a -mm -m5 37723 37217 562 282 132765230 +WinRar a -mm -s- 37702 37219 536 284 132777620 +WinRar a -mm -s 37660 37170 544 279 132614830 +WinRar a -mmf 37022 36609 458 300 130552530 +rar32 a -mm -mde 37803 37244 621 329 132671570 +rar32 a -mm -m5 37844 37272 635 330 132765230 +rar32 a -mm -s- 37828 37278 611 335 132777620 +rar32 a -mm -s 37776 37225 612 327 132614830 +rar32 a -mmf 37161 36669 546 351 130552530 rkim c 33476 31271 2450 2400 103056300 rkim cx 35322 30297 5583 1958 100013340 -rkive -mt3 35073 33938 1260 1282 117108857 -rkive -mt1 35050 33930 1243 1276 117108868 -rkive -mf3 34960 33921 1153 1276 117108724 -rkive -mf1 34957 33915 1157 1270 117108780 -ufa a -m5 -mu32 47658 42322 5927 6186 127961498 -ufa a -m5 -mu4 47906 43617 4764 5087 136996108 -ufa a -m4 -mx 41496 36117 5975 1279 123269848 -ufa a -m4 -mq 36969 35707 1400 1383 123064358 -uharc a -m3 39266 33496 6411 796 115412754 -uharc a -m3 -md64 36807 33259 3941 799 115439014 -uharc a -m2 37269 33551 4131 814 116367752 -uharc a -m1 36161 33512 2942 825 116616698 uhic m 34331 31276 3393 3473 98870920 uhic e 31230 29686 1713 1701 100132460 Display -i --same -bpng filter=5,level=9 43891 42400 1654 203 151317180 filter=5,level=1 44264 43849 460 211 156934080 filter=1,level=9 43935 42492 1602 190 151714120 filter=1,level=5 43832 43290 601 194 154930330 filter=1,level=1 44634 44336 330 203 158763080 SEA /png:100 42141 41458 757 157 148414950 +pngcrush -q 47048 42442 5117 541 149004140 +pngcrush -brute 119504 49565 77710 680 148011370 Alchemy ---n 46293 44162 2365 610 155940960 ---n 3 46345 44637 1896 737 157361408 ---n 2 46073 44714 1510 515 158572993 ---n 1 44997 43647 1497 510 154758381 locoe -N -c1 41580 41296 314 347 147305210 locoe -N -c2 41823 41445 420 375 147701630 -PhotoJazz ????? ????? ??? ??? 111673670 + indicates new lines in current volume of ARTest - indicates oldest lines, that will not be present in next volume. Overall score is calculated by adding compress_time, extract_time, and time it would take to transfer the compressed file over a 28,800bps network: (compressed_size)/3600 , because 28800 bits_per_second is 3600 bytes_per_second Average Users' score is calculated by adding (compress_time/10) + extract_time+ time it would take to transfer the compressed file over a 28,800bps network. Compression time is devided by 10 here, because more than 90% of people would never compress anything during their life (with compression programs), but they use compressed data almost _every_ time they use computers and/or internet. That's why compression time is not so actual for them.[[3]] Details
are no longer put to this main text (816 lines reporting 5757 results on 303 files in 10 sets), but can be found in FULL version with DETAILS.DAT and *.BAT at http://geocities.com/SiliconValley/Bay/1995/artest15.zip or http://artest1.tripod.com/artest15.zip 1,2,3,4,9 image sets were taken from ftp://ipl.rpi.edu/pub/image/ and /image2/ converted from Sun Raster format to Windows Bitmap, (this conversion changes image headers only, 32 bytes are replaced by 54 bytes, but image data is not modified) This site ipl.rpi.edu is described in comp.copression FAQ by Jean-loup Gailly, in 29 lines of "Subject: [55] Where can I find Lenna and other images?" Eight images in the 5th set are from the famous Waterloo Color Set, ( losslessly compressed with Eri32 4.5fre to 2,894,045 bytes, it can be found at http://geocities.com/eri32 ) 6,7,8th image sets are from ftp://download.intel.com/pressroom/images/ 10th set was downloaded from ftp://ftp.kodak.com/www/images/dental/ They were converted from TIFF format to Windows Bitmap. Some images were slightly renamed. See DETAILS.DAT from artest15.zip for more details. Some new images (from download.intel.com ) were not added yet.[[4]] Comments
Links to download programs:
ArHanGeL 1.40:a http://geocities.com/SiliconValley/Lab/6606/arh140a2.zip 50K ERI32 4.5fre:e http://geocities.com/eri32/eri45fre.zip 90K RK 1.02a1:W http://members.tripod.com/~malcolmt/downloads/rk102a01.exe 247K RKim 1.03 :e http://members.tripod.com/~malcolmt/downloads/rkim103.zip 84K Rkive 1.92b1:e http://members.tripod.com/~malcolmt/downloads/rkv192b1.zip 129K Imp-dos 1.1b5:e http://www.technelysium.com.au/imp110b5d.zip 267K Imp98 1.1b5 :W http://www.technelysium.com.au/imp110b5.exe 119K PkZip 2.50 :a ftp://ftp.simtel.net/pub/simtelnet/msdos/arcers/pk250dos.exe 202K BMF 1.1 :e ftp://ftp.simtel.net/pub/simtelnet/win95/graphics/bmf_1_10.zip 132K Ufa32 0.04b1:W ftp://ftp.simtel.net/pub/simtelnet/win95/compress/ufa004b1.zip 64K 777 0.04b1:W ftp://ftp.elf.stuba.sk/pub/pc/pack/777004b1.zip 72K 7Zip 2.01 :W ftp://ftp.elf.stuba.sk/pub/pc/pack/7zip201.zip 81K Boa 0.58b :e ftp://ftp.elf.stuba.sk/pub/pc/pack/boa058.zip 74K RAR32 2.60 :e ftp://ftp.elf.stuba.sk/pub/pc/pack/rarx260.exe 255K WinRAR 2.60 :W ftp://ftp.elf.stuba.sk/pub/pc/pack/wrar260.exe 529K ESP 1.92 :a ftp://ftp.elf.stuba.sk/pub/pc/pack/esp_v192.exe 68K UHarc 0.2 :e ftp://ftp.elf.stuba.sk/pub/pc/pack/uharc02.zip 99K UHIC 1.0 :e ftp://ftp.elf.stuba.sk/pub/pc/pack/uhic10.zip 52K NK 99nov :W ftp://tu.tusur.ru/pub/ImageTools/compress.zip 81K JPEG-LS 0.90 :W http://www.hpl.hp.com/loco/locoX900.zip 375K PngCrush1.1.8:e ftp://ftp.cdrom.com/.1/png/src/pngcrush-1.1.8-dos.zip 134K :a - DOS programs, will run under pure DOS or in a Windows DOS box :e - DOS programs using DOS extenders like DOS/4GW or CWSDPMI :W - Windows95/98/NT programs If direct link doesn't work-most probably newer version of the program appeared at the same site: visit web page, or read the whole directory from ftp server (i.e. try the same URL, but without filename).Homepages:
Arhangel : http://geocities.com/SiliconValley/Lab/6606 Eri32 : http://geocities.com/eri32 mirror : http://artest1.tripod.com RK, RKim : http://malcolmt.tripod.com Imp,WinImp : http://www.technelysium.com.au PkZip : http://www.pkware.com Ufa,777,7Zip : http://compression.hypermart.net RAR,WinRAR : http://www.rarsoft.com ESP : http://esp-team.scene.hu mirror : http://banki1.banki.hu/~arpi JPEG-LS : http://www.hpl.hp.com/loco PngCrush : http://www.xnot.com/pmt/pngcrushWhat's new:
9 new programs were tested: ArHanGeL 1.40a2, Eri32 4.5fre, NK 99nov, RK 1.02a1, PngCrush 1.1.8, Imp 1.10b5 (both DOS and Windows95/98/NT versions), WinRAR 2.60, RAR32 for Dos and OS/2 2.60 . NK is new image compressor with good compression ratio and time. RK 1.02a1 with -mx1 was only 0.42% worser than RK 1.02a1 with -mx2 (304355092/303087960) and only 0.0006% worser than RK 1.01a1 with -mx1 (304355092/304353168). PngCrush worked about 3 hours with -q switch , and about 64 hours with -brute . WinRAR and RAR32 were compressing files to exactly same sizes, but RAR32 was slower, as a rule. 33 new images appeared at ftp://download.intel.com/pressroom/images/ , they were already downloded, converted to .BMP, converted to .BMF, and written to CDROM, but will be added to 6th, 7th and 8th sets later. Three lossless JPEG programs mentioned in comp.compression FAQ - > [15] Where can I get image compression programs? > ftp://havefun.stanford.edu/pub/jpeg/JPEGv1.2.1.tar.Z (has lossless mode) > ftp://ftp.cs.cornell.edu/pub/multimed/ljpg.tar.Z (lossless jpeg) > http://idt.net/~dclunie/jpegls.html (lossless jpeg by David Clunie) were not tested at all: no executables were found in these packages: C sources only, and other interesting texts (readme files, copyright and portability notices, test images, etc.) The third link reports that > It only handles one component (ie. grayscale images) for now. while the forth desirable link from that section [15] > ppm2pz: (lossless 24-bit image compression) > http://www.jyu.fi/~kuru/compression.html seems to be not valid (since 1997, by the way).Some WARNINGS again:
ESP 1.92 has some bugs and was unable to correctly uncompress some test images: they are (re)named in our tests as AYMAR,GELSNGR,NACHTSH,OTELLIN,VADASZ,YASSO,YU1, DP08189,DP2179D,L217C-3,CONCEP3,PS09157,V42798C. ESP doesn't even report something like "CRC fails"...as if everything is OK. Rkive 1.92b1 failed to losslessly uncompress most images from 6th, 7th and 8th tests, and also Clegg, Frymire, Serrano from 5th. It reports ERROR: File fails CRC check!. Archive failed integrity check. RKim 1.03 failed to compress 3 images (AIRPLANE,GIRL3,SPLASH), and to extract some images compressed with "cx" (CBARRETT,CHOU,GELSING2,DP011199,SP011199, ED50799B,ED50799E,CONCEPT4,VC42798C,LIDOCAN1,LIDOCARP,RAPIDACC,COULSON). RK 1.02a1 fails to extract set of two exactly-same bitmap files: For example, "copy kodak01.bmp kodak01a.bmp", and try "RK -mf1 Z kodak01*.bmp" , "RK -i Z" The same happens if you try RK with any of -mx1, -mx2, -mx3, -mf1, -mf2, -mf3 ERROR 303: CRC check failed. The LATEST RELEASE, and twelve previous versions of these tests can be found at http://geocities.com/SiliconValley/Bay/1995/ http://artest1.tripod.com/The FINAL PART
> [[5]] PLEASE read THIS before replying to this article was removed from this text, but can be easily found at http://geocities.com/SiliconValley/Bay/1995/artest10.html http://artest1.tripod.com/artest10.html Send your suggestions, comments to ratush@srsc-gw.sscc.ru With best kind regards, RAO Inc.