Nice to have some new open source compressor. The source is tiny, only 11.8 kB :) BTW very readable.
One remark:
I thought you use a sliding window? Blockwise processing like this doesn't make it faster (you always clear your offset table) and looses compression.
BTW, I want to add BALZ-file decompression to PIM archiver... :D
Can the current PIM work in CLI mode ? (for scripts, .bat's)Quote Originally Posted by encode View PostBTW, I want to add BALZ-file decompression to PIM archiver... :D
No matter what switches I used, I always get the GUI firing up...
Are there any "hidden" options to do this ?:D
It might be interesting to add CLI support though.
PIM, IIRC, has no command-line switches. PIMPLE - has. With PIM I did main accent on WinRAR/WinZip-like interface - very few freeware archivers have such familiar interface... ;)Quote Originally Posted by pat357 View PostCan the current PIM work in CLI mode ? (for scripts, .bat's)
No matter what switches I used, I always get the GUI firing up...
Are there any "hidden" options to do this ?:D
It might be interesting to add CLI support though.
Nice and tidy as is all your open source. I glanced the source and added a very quick 1min hack to to replace the inefficient table update.Quote Originally Posted by encode View Post
balz_ref e acrord32.exe ref -> 3.7sec
balz_hack e acrord32.exe hack -> 3.1sec
Less than I expected, but no time right now for more.
Last edited by Sami; 8th July 2008 at 20:05.
Well, don't be so sure. I tried many variants, some of them, like yours, is really might be more efficient with "e" mode. However, all stuff is tuned for "ex" mode and with other approaches we may seriously loose too much compression speed, believe me. All this stuff developed by years, years of testing and tuning... Anyway, I'm happy to see some user attempts and suggestions. Thank you! :)Quote Originally Posted by Sami View PostNice and tidy as is all your open source. I glanced the source and added a very quick 1min hack to to replace the inefficient table update.
balz_ref e acrord32.exe ref -> 3.7sec
balz_hack e acrord32.exe hack -> 3.1sec
Less than I expected, but no time right now for more.
So you cannot reproduce the results? Did you check my attachment?Quote Originally Posted by encode View PostWell, don't be so sure. I tried many variants, ...
I tested ex mode with acrord32.exe gets 5.5sec and the hacked version is 5.0sec.Quote Originally Posted by encode View PostWell, don't be so sure. I tried many variants, some of them, like yours, is really might be more efficient with "e" mode. However, all stuff is tuned for "ex" mode and with other approaches we may seriously loose too much compression speed, believe me.
Check this out:Quote Originally Posted by Sami View PostSo you cannot reproduce the results? Did you check my attachment?
The end of discussion... ;)Code:TraktorDJStudio3.exe (29,124,024 bytes) balz_ref e - 5.2 sec balz_hack e - 5.0 sec balz e - 4.7 sec balz_ref ex - 16.7 sec balz_hack ex - 16.9 sec balz ex - 12.5 sec
The ref is a reference, compiled with same settings. Try compiling the hack source with the same compiler as your balz.Quote Originally Posted by encode View PostCheck this out:
The end of discussion... ;)
--edit--
TraktorDJStudio3.exe (ex) ref=39.3s hack=38.2sec
Last edited by Sami; 8th July 2008 at 20:26.
Timings for balz_hack compiled with the same compiler as original:Quote Originally Posted by Sami View PostThe ref is a reference, compiled with same settings. Try compiling the hack source with the same compiler as your balz.
:pCode:The same TraktorDJStudio3.exe... balz_hack e - 5.0 sec balz_hack ex - 15.4 sec
Ok, so we got an improvment.Quote Originally Posted by encode View Post:pCode:The same TraktorDJStudio3.exe... balz_hack e - 5.0 sec balz_hack ex - 15.4 sec
--edit---
no we don't, I read you incorrectly. Now I'm puzzled why the results are like that.
--edit--
TPPM coder looks suspicious to me speedwise and ratiowise, I think whoever continues now should replace it.
Don't forget about that we should work in conjunction with the compiler...Quote Originally Posted by Sami View PostOk, so we got an improvment.
--edit---
no we don't, I read you incorrectly. Now I'm puzzled why the results are like that.
--edit--
TPPM coder looks suspicious to me speedwise and ratiowise, I think whoever continues now should replace it.
Forget about BALZ - it's not so easy as might be at the first sight, better continue working on your marvelous NanoZip!
BTW, can somebody test the IA64 compile of BALZ? I'm very curious about it's timings...
I've looked at the source in a short time. It's very readable. I just surprised when I see LZMA fast mode like match-length/index based optimal parsing (of course it's more simpler) in this tiny source. Is there a chance for adding bit optimal parsing for exx mode? I wonder the results.
Also, LZH like 9 bits symbols are ok. But, using flags is more stronger (I have tested in my compressor: BIT). Also, note that I have collected some statistics on well used test files. Most of files have an interesting statistics about match lengths. <8 and <16 most probable cases. So, I divided the match length into three parts: [2,8], (8-16], (16, MaxMatchLength]. In my experiments, I also confirm that order-2 match context for ROLZ is well-suited for fast compressing modes (it's a balancing point for efficiency: fast and strong compression for me).
Overall, good job Ilia!
Last edited by osmanturan; 9th July 2008 at 12:57.
It is still possible, though I have no time currently to do it myself. Someday, somehow... ;)Quote Originally Posted by osmanturan View PostIs there a chance for adding bit optimal parsing for exx mode?
thx for LovePimple's work
I am in China where can access http://balz.sourceforge.net/ but not
http://sourceforge.net/
Do you mean that you found Ilia's BALZ project because of the mirror at my web site?
thank you and mike supply the download addressQuote Originally Posted by LovePimple View PostDo you mean that you found Ilia's BALZ project because of the mirror at my web site?
Here's a few of my own speed optimised compiles of BALZ v1.15. A Pentium 3 or later processor is required.
ENJOY! :)
I nearly forgot this. Here is the test result with ENWIK8 on my laptop (Core2 Duo 2.2GHz, 2 GB RAM, Vista Business x64 SP1). I run the test for two times for ensuring the results.Quote Originally Posted by encode View PostBTW, can somebody test the IA64 compile of BALZ? I'm very curious about it's timings...
Suprisingly, 64-bit version much worser in speed term on compression. My experiments was opposite on ROLZ (I had 20% speed gain on compression/decompression by using x64 compile rather than x86). Note that, I had used data alignment on dictionary and ROLZ offset table which increases cache efficiency. I wonder why speed was dropped on compression.BALZ
Compression:
54.869 seconds
51.856 seconds
Decompression:
11.319 seconds
11.506 seconds
BALZ64
Compression:
70.912 Seconds
70.815 Seconds
Decompression:
10.931 seconds
10.316 seconds
32-bit version is more optimized. I did many tests, tunings, etc. 64-bit version was just blindly compiled...Quote Originally Posted by osmanturan View PostSuprisingly, 64-bit version much worser in speed term on compression. My experiments was opposite on ROLZ (I had 20% speed gain on compression/decompression by using x64 compile rather than x86). Note that, I had used data alignment on dictionary and ROLZ offset table which increases cache efficiency. I wonder why speed was dropped on compression.
Mini test, balz vs freearc 0.50a
Source : xampp-linux-1.6.7 from http://sourceforge.net/project/showf...ease_id=611649
Used only GNU/linux versions
Original File size:
61209332 bytes (gzip compression)
Ungziped:
207175680 bytes
I ungziped and recompressed the tar..
Balz:
35679161 bytes (using -ex) (35mb's)
Freearc 0.50a
28629187 bytes (using -mx only) (28mb's)
Compression time?
freearc = lzma/ppmd + preprocessors
are you really expected that balz or anything else will easily outperform two the best compression algorithms ATM?
(i don't even say about preprocessors effect)
Cpu it's 2.6Ghz HT with 512 Mb's of Ram
Kernel linux used 2.6.26
Packing times:
Balz:
--
real 3m55.129s
user 3m40.806s
sys 0m2.468s
Freearc 0.50a
---
real 4m42.682s
user 6m21.600s
sys 0m6.700s
Unpacking times:
Balz:
real 0m33.908s
user 0m30.270s
sys 0m2.028s
Freearc:
real 0m15.976s
user 0m8.525s
sys 0m2.152s
Last edited by John; 30th July 2008 at 23:18.
Did you mean Itanium (VLIW) or AMD64/EM64T (long mode)? Because the latter is ubiquitous but the former is probably rare for home users! :pQuote Originally Posted by encode View PostBTW, can somebody test the IA64 compile of BALZ? I'm very curious about it's timings...
Decided to update the BALZ project:
Uploaded optimized binaries for "Classic" BALZ v1.15 - From year 2008
Released "Optimized" BALZ v1.20 - fully compatible with the v1.15, but notable faster. Also, Normal compression mode now is much faster at the cost of compression ratio. Plus, now it has a proper command-line interface, at last!
http://sourceforge.net/projects/balz/
Enjoy!
Fu Siyuan (6th March 2015),jibz (7th March 2015),Matt Mahoney (6th March 2015),Mike (6th March 2015),moisesmcardona (6th January 2019),Nania Francesco (6th March 2015),WayneD (6th March 2015)
Code:C:\Test>balz c enwik8 enwik8.z Compressing enwik8... 100000000 -> 30056097 in 6.047s C:\Test>balz cx enwik8 enwik8.z Compressing enwik8... 100000000 -> 28232824 in 19.345s C:\Test>balz d enwik8.z e8 Decompressing enwik8.z... 28232824 -> 100000000 in 2.562s C:\Test>balz c enwik9 enwik9.z Compressing enwik9... 1000000000 -> 261416611 in 52.988s C:\Test>balz cx enwik9 enwik9.z Compressing enwik9... 1000000000 -> 245218274 in 193.654s C:\Test>balz d enwik9.z e9 Decompressing enwik9.z... 245218274 -> 1000000000 in 22.407s