I am interested in optimizing the memory capacity of an AGI. Given a specific complex input an AI can create a simplified model. This is a problem that can be solved using sparse coding . However, this solves only the problem of encoding and not the maintenance of online representations—in cognitive terms: the state of mind.
A default cognitive model of short-term memory can be separated in three different stages:
Encoding → Maintenance → Retrieval
One solution is to use specialized hardware , but I am interested in software approaches to this problem and I would thus like to emphasize that it is the digital representation, which I am most interested in.
With the exception of qubits, the smallest possible representation are binary digits. However, additional architecture is required to represent phase spaces (i.e. floating point precision memory) and higher-order representations maybe include arrays and dictionaries. (Optimizing these are trivial... or simply to be postponed until needed, according to Knuth).
- How can a specific connectivity pattern be stored in an optimally compact representation?
- Is there an implementation with concrete example code?
- What is the state-of-the-art?*
* I will not specify "real-time" here, but the context is humanoid AGI.
 Papyan, V., Romano, Y., & Elad, M. (2017). Convolutional Neural Networks Analyzed via Convolutional Sparse Coding. Journal on Machine Learning Research, 18(83): 1–52. arXiv:1607.08194 http://jmlr.org/papers/volume18/16-505/16-505.pdf
 LeGallo et al. (2018). Mixed-precision in-memory computing. https://www.nature.com/articles/s41928-018-0054-8
 IBM. (2018). IBM Scientists Demonstrate Mixed-Precision In-Memory Computing for the First Time; Hybrid Design for AI Hardware. https://www.ibm.com/blogs/research/2018/04/ibm-scientists-demonstrate-mixed-precision-in-memory-computing-for-the-first-time-hybrid-design-for-ai-hardware/