The Art of Lossless Data Compression vol. 18b

Here are the results of tests performed in September 2000 to compare lossless compression of "binary" files by all known good enough programs developed for such purpose, including RK, DC, YBS, Bzip2, IMP, RAR and 7-zip. See Archive Comparison Test by J.Gilchrist for more details: http://act.by.net If anybody wants to start or continue such tests, or can suggest some other sets of files, or other compression programs, (not sources or algorithm descriptions, executable programs only) or knows we have missed something important, (some new fantastic technology, an algorithm or even a program capable of lossless compression of up to 1000:1 etc.) please let us know immediately: artest@hotmail.ru Thank you!

[[1]] COMPRESSION QUALITY

(see also [[2]] Speed [[3]] Details [[4]] Comments) Last eleventh line shows results for the sum of all 5029 files in ten sets. PPMonstr ACB ACE32 BOA BA BEE BIX BZip2 DC ERI -o8 -m56 u -m5 -d4096 -m15 -k -50 -m3 -d3 -m9 -mdg -k -9 (none) (none) 105.10 104.79 105.11 107.89 109.60 110.61 107.68 113.02 106.49 109.43 108.07 108.11 104.85 110.78 115.42 111.83 105.63 118.24 113.91 116.91 103.36 102.41 102.46 104.29 106.27 105.96 102.89 107.74 106.01 106.97 112.64 112.95 112.65 116.18 119.09 114.57 114.22 122.72 108.57 113.77 111.93 113.23 101.79 113.29 113.26 116.93 117.09 114.41 104.91 100% 115.77 116.72 116.68 118.59 123.06 121.78 117.98 127.96 118.46 123.08 104.92 104.47 104.80 107.05 109.17 107.38 104.00 111.64 107.06 109.91 112.14 112.93 107.77 114.86 118.97 118.83 113.94 121.25 112.38 113.37 101.44 100.13 100.59 102.01 103.45 104.18 100.78 104.21 103.65 103.70 111.74 114.51 113.19 117.44 122.32 116.41 115.32 128.67 120.72 126.38 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 106.81 106.96 104.96 109.14 111.78 110.82 107.80 114.48 108.42 110.29 IMP ArHanGel PkZip RAR RK SZip UHA YBS 7zip ZZip -1-m3-mm -1-2-mm -exx -m5-mde-mm -mx2 -o0 -b41 -m3-mm -m8mu -mx -mx-mm-b20 110.73 109.52 116.94 109.61 100% 110.04 107.58 110.32 114.95 117.35 106.82 120.88 126.23 110.03 100% 115.35 104.10 112.40 124.43 117.76 103.09 105.29 107.41 105.17 100% 106.37 102.29 104.82 106.52 108.23 109.12 114.02 138.02 110.42 105.48 118.99 100% 114.37 135.13 110.55 113.81 107.13 121.43 103.86 106.18 113.73 102.74 113.17 120.28 106.23 123.07 123.40 129.54 127.73 100% 121.56 114.55 122.64 126.54 132.90 105.80 108.73 114.03 108.45 100% 109.24 103.79 105.31 112.48 108.84 119.90 119.85 124.86 118.45 100% 117.94 112.15 117.26 121.25 123.69 100.22 102.99 103.42 100.61 100% 103.77 100.48 103.65 103.02 105.90 119.69 128.50 133.47 128.44 100% 123.91 111.30 124.06 128.79 136.23 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 109.02 111.80 118.61 109.83 100% 111.85 104.40 110.49 116.64 114.32

[[2]] Speed

was not tested this time. Next release will be soon.

[[3]] Details

are no longer put to this main text (5176 lines reporting 221760 results on 5029 files in 10 sets), but can be found in FULL version with BINARIES.DAT and *.BAT at http://geocities.com/SiliconValley/Bay/1995/artest18.zip or http://artest1.tripod.com/artest18.zip

[[4]] Comments

Links to download programs:

7-Zip 2.11 :W http://www.7-zip.com/dl/7zip211.exe 493K BIX 1.00b7 :W http://www.7-zip.com/dl/ufa/bix100b7.zip 89K ArHanGeL 1.40 :a http://geocities.com/SiliconValley/Lab/6606/arh140.zip 50K ERI32 4.7fre :e http://geocities.com/eri32/eri47fre.zip 91K Imp 1.1 :e http://www.winimp.com/imp110d.zip 266K Imp-win 1.12 :W http://www.winimp.com/imp112.exe 122K PkZip 2.50 :a ftp://ftp.simtel.net/pub/simtelnet/msdos/arcers/pk250dos.exe 202K RK 1.03b1 :e http://malcolmt.tripod.com/downloads/rk103a1d.exe 478K RK 1.03b1 :W http://malcolmt.tripod.com/downloads/rk103a1w.exe 380K RAR32 2.71 :e ftp://ftp.netlab.sk/public/rarsoft/rar/rarx271.exe 257K WinRAR 2.71 :W ftp://ftp.netlab.sk/public/rarsoft/rar/wrar271.exe 588K PPMD var.F, PPmonstr v.F :W ftp://ftp.simtel.net/pub/simtelnet/win95/compress/ppmdf.zip 97K ACB 2.00c :e ftp://ftp.simtel.net/pub/simtelnet/msdos/compress/acb_200c.zip 42K BOA 0.58b :e ftp://ftp.cdrom.com/.3/sac/pack/boa058.zip 74K DC 0.98b :W ftp://ftp.cdrom.com/.3/sac/pack/dc124.zip 55K BA 1.00 beta :e ftp://ftp.cdrom.com/.3/sac/pack/ba100b.zip 60K Bzip2 1.0.1 :W ftp://sourceware.cygnus.com/pub/bzip2/v100/bzip2-100-x86-win32.exe 68K SZip 1.12a :W http://www.compressconsult.com/szip/szip_112a_win32.zip 71K UHArc 0.2b :e ftp://ftp.cdrom.com/.3/sac/pack/uharc02.zip 101K ZZip 0.35e :W http://www.via.ecp.fr/~damien/zzip/zzip-win32.zip 24K ACE32 2.0b2 :W ftp://ftp.forlangs.net/pub/windows/winace/ace20b2.exe 546K YBS 0.03d :e http://members.nbci.com/vycct/ybs003dd.zip 48K YBS 0.03d :W http://members.nbci.com/vycct/ybs003dw.zip 42K :a - any DOS - DOS programs, will run under pure DOS or in a DOS box :e - extender - DOS programs using DOS extenders like DOS/4GW or CWSDPMI :W - windoze - Windows95/98/NT/etc programs If direct link doesn't work-most probably newer version of the program appeared at the same site: visit web page, or read the whole directory from ftp server (i.e. try the same URL, but without filename).

Homepages:

Arhangel : http://geocities.com/SiliconValley/Lab/6606 Eri32 : http://geocities.com/eri32 mirror : http://artest1.tripod.com RK : http://malcolmt.tripod.com Imp,WinImp : http://www.technelysium.com.au mirror : http://www.winimp.com ACE32 : http://www.winace.com PkZip : http://www.pkware.com RAR,WinRAR : http://www.rarsoft.com BZip2 : http://sources.redhat.com/bzip2 SZip : http://www.compressconsult.com/szip ZZip : http://www.via.ecp.fr/~damien/zzip YBS : http://members.nbci.com/vycct Ufa,777, BIX,7-Zip: http://www.7-zip.com

What's new:

All contents of this file: 20 programs were tested on 5029 test files. FULL version contains all *.BAT and *.DAT files you'll need to build 10 directories with 5029 files (artest18.zip\MAKE_BIN\*.*) and to repeat all our tests (bins.bat, binaries.dat). You will also need the following to extract .ZIP, .Z, .CAB, .1, .LZH archives: ftp://ftp.cdrom.com/.3/simtelnet/msdos/arcers/lha255e.exe (64K) ftp://ftp.cdrom.com/.3/sac/pack/i5comp21.rar (95K) ftp://ftp.cdrom.com/.3/sac/pack/icomp95.zip (60K) ftp://ftp.cdrom.com/.3/sac/pack/ttcomp.zip (25K) ftp://ftp.cdrom.com/.3/sac/pack/stix.arj (21K) pkunzip (see link above). DOS prompt calculator with user def. functions (math.exe being used for ARTest) can be found at ftp://ftp.simtel.net/pub/simtelnet/msdos/calculte/mathfc24.zip (26K) Ultra Precision Command Timer 1.6 - Freeware (C) 1993 by Erik de Neve (upct.exe being used for ARTest) can be found at ftp://ftp.cdrom.com/.3/sac/utilmisc/upct16.zip (7K) MultiEdit 7.00jP-386 was used for files editing with macrocommands, blocks etc, and standard fc.exe from any DOS/Windows package - for comparing files. Latest beta versions of BEE, DC, PPMonstr are available from authors by e-mail request: BEE: Andrew.Filinsky@p11.f4.n452.z2.fidonet.org DC: EdgarBinder@t-online.de PPMonstr: shkarin@arstel.ru , dmitry.shkarin@mtu-net.ru

WARNINGS:

ACB refuses to take files shorter than 257 bytes. About 400 such files were processed with "rar a -m0 name.acb name.bin" (see do_acb_u.bat and do_acb_r.bat). BA 1.00beta can't losslessly decompress any file compressed with -f , 49 files compressed with any option (astronmy\CRLFTX~1.bin, GFEMER~1.bin, MIRAIN~1.bin, NIL_~1.bin, NIL~1.bin README~7.bin, SAOSOU~2.bin, STARVI~6.bin, UNIV00~1.bin, ZEROME~1.bin ; chem\ARCHIV~3.bin, ARCHIV~4.bin, DISK1_~1.bin, DISPOS~3.bin, KHEMCF~1.bin etc.) It says nothing like "CRC fails". DC 0.99.158b fails to decompress HLPCOPY..bin, but only if you compress with "-mb5" switch and [<output>] filename like "any.dc5". Says nothing like "CRC fails". RK 1.03b1 was unable to correctly decompress 55 files compressed with "-mf2" or "-mf3", reporting > ERROR 303: CRC check failed. All are .htm, like font\README~3.BIN , and only one byte differs after extraction, and no problems if compressed with "-mx2" in all 55 cases. Unfortunately, 64Mb RAM is not enough to run RK 1.03 with "-mx3": endless swapping (virtual memory to hard disk) when compressing some binary files (but no swapping on text files). ZZip 0.35e can't decompress only one file, mapping\DEMOMP~1.bin . With any (compress) options, "ZZip error : CRC error". UFA and 777 can't find files with long names, if you give short versions of that long names, and they can't process files with two or more extensions. BIX, third version of UFA, was tested instead. Problems in all other 15 compressors were not found. File order (in binaries.dat) is wrong in some columns: - RK and BOA understand only long filenames, not short versions, and files get different short versions of names when extracted. - in \mapping\ directory, files were not sorted by DIR/on command: "Too many files, directory not sorted". The LATEST RELEASE, and fifteen previous versions of these tests can be found at http://geocities.com/SiliconValley/Bay/1995/ and http://artest1.tripod.com/

The FINAL PART

> [[5]] PLEASE read THIS before replying to this article was removed from this text, but can be easily found at http://geocities.com/SiliconValley/Bay/1995/artest10.html http://artest1.tripod.com/artest10.html Send your suggestions, comments to artest@hotmail.ru With best kind regards, RAO Inc.