As mentioned, I still don’t have clear thoughts about what AGI looks like, but I think I have a decent idea of what people I like talking to extensively look like, and I think it has to do with aesthetic. And I think aesthetic has something to do with compression.
So about two months ago, when trying to mull over these ideas with a friend, he told me about the Hutter Prize, a competition about compression:
Being able to compress well is closely related to intelligence as explained below. While intelligence is a slippery concept, file sizes are hard numbers. Wikipedia is an extensive snapshot of Human Knowledge. If you can compress the first 1GB of Wikipedia better than your predecessors, your (de)compressor likely has to be smart(er). The intention of this prize is to encourage development of intelligent compressors/programs as a path to AGI.
And I was excited to learn about this because I didn’t have a concrete idea of how one might explore this idea concretely, and it was interesting to see how they’ve been able to take it.
But then I found that something didn’t quite match to my notion of it, particulary when they mentioned that it took a substantial amount of time to decompress the compressed files. And there isn’t priority in what gets unpacked first. Or what is even worth compressing. Because I think my interest in compression is to have a more efficient tool for processing data. And while I appreciate the naive compression algorithms that help keep my computer running smoothly with low latency, I also don’t appreciate the aesthetic of blind boxing up all your belongings when moving; I think it’s an excellent opportunity to prune. Perhaps I’m overimposing the results of my own constraints, but if you have any sense of impatience, you want what you’re looking for to be in the top-k of output.
I’m suspecting that even if memory and compute scale obscenely, I don’t think the concept of ‘priority’ will become obsolete.
I guess it’s because, at core, I’m thinking more of dimensionality reduction, with the implicit goal of increasing the signal to noise ratio. Compression is one approach. PCA is another dimensionality reduction approach, one that is particularly pervasive – you could say that it ‘compresses’ the likely useful stuff in the first couple of PCs, but it’s only dimensionality reduction if you do the critical next step, of actually throwing out the rest of the PCs.
And so for type of advanced functionality I’m vaguely imagining, I think there’s something about being able to compress lossily.