since I'm new to ROLZ, the time performance is not good at all. But it seems to make better compression ratio the a normal LZ77 compressor!
http://comprox.googlecode.com/files/...z-0.1.0.tar.gzCode:world95.txt 3005020 => 557787 bible.txt 4047392 => 811732 fp.log 20617071 => 681905
For compiling, my MinGW needs crblib, but the one provided by Charles Bloom doesn't seem to work:
dpcm.c:63:24: schwerwiegender Fehler: crbinc/inc.h: No such file or directory
Kompilierung beendet.
imppm.c:8:24: schwerwiegender Fehler: crbinc/inc.h: No such file or directory
Kompilierung beendet.
C:\MinGW\bin>gcc -O3 *.c
In file included from dpcm.c:63:0:
c:\mingw\bin\../lib/gcc/mingw32/4.6.2/../../../../include/crbinc/inc.h:23:28:
hwerwiegender Fehler: crblib/memutil.h: No such file or directory
Kompilierung beendet.
In file included from imppm.c:8:0:
c:\mingw\bin\../lib/gcc/mingw32/4.6.2/../../../../include/crbinc/inc.h:23:28:
hwerwiegender Fehler: crblib/memutil.h: No such file or directory
Kompilierung beendet.
I compiled with no problem using "gcc -O3 *.c" with MinGW 4.6.1 (Win32). Test results:
http://mattmahoney.net/dc/text.html#2158
http://mattmahoney.net/dc/silesia.html
I guess you might need pthreadGC2.dll to run.
I have also compiled it now. Thanks for the hint.
Thanks for the benchmark.Quote Originally Posted by Matt Mahoney View PostI compiled with no problem using "gcc -O3 *.c" with MinGW 4.6.1 (Win32). Test results:
http://mattmahoney.net/dc/text.html#2158
http://mattmahoney.net/dc/silesia.html
I guess you might need pthreadGC2.dll to run.
I use native threading APIs on windows, so pthreadGC2.dll is no longer needed. (I successfully compiled and ran it with mingw32 and wine under linux)
I found that all my programs have bad performance on decompression. I think I'm playing with too long context for decoding literals.
Should I pay more attention on optimal parsing and reduce context length?
I didn't look at the source, but ROLZ decompresses slower than LZ77 because the decompresser has to maintain an index.
@RichSelian:
"I use native threading APIs on windows, so pthreadGC2.dll is no longer needed."
Can you please post a win32 - binary here in the forum?
i want to compare the program with the "open source program BALZ (ROLZ-compression) from encode"
the compression seems to be not bad ..
"bad performance on decompression"
maybe because your compression-algorithm can use 2 cores but your decompression-algorithm can use only 1 core?
best regards
Joerg
Compiled with gcc -O3 *c in MinGW 4.6.1 for Win32, packed with upx.
I use o2-o1 model to encode literals, it's the main reason that make decompression slow. but for good compression ratio it is necessary.
I don't understand why some LZ compressors (like xz) can make as good compression ratio while they are using o1 model?
It seems like high order modeling of literals is not needed because they would be coded as matches instead. Also, some algorithms like LZMA use literal exclusion after matches. The first byte after a match would be poorly predicted by a model or else it would have extended the match, so it XORs it with the predicted byte.
That means you have to search for len=3 matches? But will replacing 3 literals with a pos/len pair really help the compression ratio? (Maybe you are using some optimal parsing skills to limit match pos?)Quote Originally Posted by Matt Mahoney View PostIt seems like high order modeling of literals is not needed because they would be coded as matches instead. Also, some algorithms like LZMA use literal exclusion after matches. The first byte after a match would be poorly predicted by a model or else it would have extended the match, so it XORs it with the predicted byte.
Length 3 matches aren't worth coding because these will be common even in random files with a 16 MB window, resulting in no compression. What I mean is that if you use a context models to predict literals, they will usually make a wrong prediction after a match. Your model needs to account for this.