Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> The best known algorithms, however, are very slow, and sometimes impractical for real-world applications.

Is this in reference to paq [1]? Are there any other really good compressors that are too slow?

[1] http://en.wikipedia.org/wiki/PAQ



For any algorithm that’s heuristically searching a very large space (in this case, the space of model parameters to compress a given string), you can expect it to do better given more time. So it would actually be surprising if the best compressors, in terms of compression ratio, were fast.

In practice, the competitions that people like to use to measure which compressor is “best” have resource limits, and compressors from the PAQ family are at or near the top of most rankings.

http://mattmahoney.net/dc/text.html has some nice charts of the Pareto frontier for at least one competition. This lets you see the most efficient compressor for a given compression ratio and vice versa. The log scale gives a sense of the diminishing returns: it’s really easy to compress most structured data to 0.5 its original size, but getting from say 0.214 to 0.213 can be a hell of a lot of work.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: