6
$\begingroup$

What is the current research in artificial intelligence and machine learning in the field of data compression?

I have done my research on the PAQ series of compressors, some of which use neural networks for context mixing.

Improve this question
$\endgroup$
0

2 Answers 2

Reset to default
2
$\begingroup$

It is already combined.

Adaptive entropy techniques are already used in most of the best compression encoders. This is true for file encoders, video encoders, and audio encoders. We use it in the solar lab to optimize sample rates in data acquisition.

In fact, pattern recognition and compression are very tightly coupled if you consider autoencoders and other feature extraction schemes and compare them mathematically with what compression does. See Data Compression – A Generic Principle of Pattern Recognition?, Gunther Heidemann, Helge Ritter, VISIGRAPP 2008

Zlib and lz4 have more like hyper-parametric learning, however they don't persist what they learn. This work is interesting: Adaptive On-the-Fly Compression, Chandra Krintz, Sezgin Sucu, IEEE Parallel and Distributed Systems, v17 n1, January 2006.

**Suggested Project:

Create a theoretical framework and software POC that learns correlations between these two sets.

  1. Quickly ascertainable features of documents or audio or video streams (i.e file path components, media titles, date, file type, and first N bytes)
  2. The existing parameters that existing open source compression software learns during its existing pattern recognition algorithms

Persisting those correlations between compression invocations may considerably improve file transfer, kernel operations (since lz4 is now native in kernels like LINUX), and media streaming.

How much effort is made to persist features extracted (pattern recognition) between frames in media streaming is worth investigating too.

Improve this answer
$\endgroup$
0
$\begingroup$

Recurrent neural networks can be trained on character level data to generate sentences which are very similar to human language. Go through this link. You could experiment with them for compressing text.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .