can you have AGI without god?

date initiated: 2024 August 13
last updated: 2024 August 13

Hello! I’ve somewhat recovered from vacation and am ready to impress my half-processed ideas upon the world.

aesthetic = god = I?

If you didn’t yet know, I am maybe a very self-indulgent person, and it seems like it wasn’t enough to extend aesthetic to god– we should now try extending aesthetic to AGI. Or in a more fun phrasing: can you have AGI without god?

(I’m still in my phase of using my hammer of ‘aesthetic’ and testing out how well it integrates.)

Something about current LLMs are less compelling to me than expected and I’ve been trying to figure out why that is, given that I supposedly think learning systems are cool.

I suspect it’s related to the impression that they are emulating and not reasoning. I don’t get the impression it learns particularly interesting models (i.e. it does not seem to learn math conceptually (is its inability to learn math well an artifact of tokenization?)). It does not seem to have the essence of what I think would be interesting about AI.

But many people who are more technically competent/adjacent to working in AI/ML than me seem to believe that these LLMs can get you most of the way to AGI/we’ll plausibly get to AGI in two years if we just continue on our trajectory.

My intuition thinks this sounds wrong, but I don’t have a rigorous framework to strongly defend it – this is a first attempt at seeing what my intuition is trying to point to, and if there’s anything it’s overlooking.

“Intelligence is what you use when you don’t know what to do” (Chollet citing Piaget)

Some people’ve panned the Dwarkesh interview with Chollet (I think it was around the interviewing style), but I think Chollet’s memorization vs intelligence framework in it is useful for structuring where my misgivings lie.

Generally, ‘intelligence’ is a broad/conflated term, and I think the framework usefully separates different kinds of competencies. He maps memorization and intelligence to system 1 and system 2 thinking, respectively, though in his elaboration that he thinks program synthesis makes me think his pointer to ‘intelligence’ here would be more usefully delineated as a ‘search’ mechanism. A similar distinction was made at an evolutionary theory workshop that was themed ‘Stress and innovation’ – stress here also gestures the context when prior heuristics are not quite sufficient to maintain homeostasis in a given environment; where a biological system ‘does not know what to do’. It was argued that this context of stress is a prerequisite for innovation, and I think ‘innovation’ maps relevantly to ‘search’. Is this the exploit vs explore paradigm?

Intelligence in the sense of learning from others well is admirable (especially considering my ability to remember facts/details is subpar), but I think the problems I’ve been grappling with are moreso relevant to intelligence in the sense of making sense of/building frameworks for something that is not yet well understood. It is closer to ‘how should we frame this’ rather than, ‘how to solve this’? (personally, I think the main bottleneck of SAT questions is framing the problem – being clear on, what is it actually asking of you? the trickier questions depend on you misframing the question)

What do you do when you don’t know what to do?

Is it a type of confirmation bias to come around to say ‘aesthetic’, again?

There are at least two ways I go about trying to make headway into problems I find interesting:

There also seems to be a correlation between having good aesthetic and asking good questions. I’d say this maps well to the hypothesis that a type of competency is navigating a possibility space efficiently. I don’t think it’s coincidence. As a first thought, maybe it’s because they tend to organize their System 1 in a way that the gaps they reveal are interesting. How do they do that?

I notice that it is automatically easier to ask good questions in a domain you have fleshed out more and have interest in – why? I think it is related to the ‘size’ of a question – generally, it seems like you need sufficient constraint/nuance to make a good question – when you are ill-informed in a domain, questions tend to be too general/broad strokes to be satisfying in that way. Perhaps the satisfaction of a good question is the sense that building a hub/lifting the map fog here would allow for better flow of thought traffic.

And where does that sense come from? Something like, loading what you think are givens, constraints/biases, and that gives you a sense of what the shape of the gap looks like, and therefore what a good answer should roughly look like (see: an elaboration on this in ‘a thousand brains’). I’d guess this would map to System 2/deliberate thinking.

Or, a slightly different framing: I currently conceptualize how I think as having two main components, a curator and a thought generator, which maps onto the structure of GANs, and maybe to System 2 and System 1, respectively. I moreso identify my’self’ with the curator, which decides if and how the thoughts generated can be used to express its ‘aesthetic’/itself. The curator is the question asker. Fundamentally, where does the curator come from?

A first thought: in the way that abstract concepts are built from metaphors upon metaphors on sensory/physical data (see: Julian Jaynes), perhaps questions are abstractions built on fundamental goals/motivations like obtaining food/warmth/physiological homeostasis (goals/desires built on top of that would be things like social acceptance/security, social skills, ability to provide social value, technical competencies, etc). Perhaps we create subgoals/questions in service of these basic goals in a fractalish way. And maybe these goals/questions are something like a cost function; expectations/wants/stress exert pressure on the existing system, and we notice the stress/differential (see: physical learning systems, andrea) and try to figure out which knobs to address to try to move back to a lower energy state (we can guess wrong, i.e. optimizing for happiness directly).

Can an artificial intelligence skip the survival and social skills aspect? though it perhaps still needs social approval in some way (we think its answers are good) Currently, it seems to be humans that prompt engineering and give LLMs goals – is it automatable? What do you need to get machine learning to ask good questions?

Does it need aesthetic?

memory/computational constraint and aesthetic – is it still relevant for artificial systems?

(good aesthetic builds knowledge graphs/selects for models that give better pointers for where to look.) The curator influences how the database is organized – given a computational/memory/etc budget, which models are integrated and prioritized? The processing and organization of information influences how it is then retrieved. For example, when trying to figure out a situation, will you load in associations that are irrelevant/happenstance/noisy correlations, or usefully causal nodes? In an emergency/urgent/immediate situation, do you know what to do first? Most questions are not urgent, but I think it is ideal if your query brings up the more relevant/interesting points first – Pasteur: “Luck favors the prepared”.

Biological systems (including ourselves) are anticipatory systems – we want/need to make solutions for a given timeframe, in conjunction with notably limited resources. Do artificial ones need to be anticipatory in this sense? Or are they less constrained by this way? How does it affect things that they can bruteforce search the solution/heuristic space, and have a lot larger memory (less need to forget)? Does beauty only have meaning when there is scarcity?

Maybe it depends on where it is applied, but I’d guess that anywhere you need to make a decision/evaluation, that is a unit of aesthetic.

Hmm but self-driving cars ‘make decisions’ – though from what I’ve been told, their models have been trained on human drivers. So perhaps what I mean is decision making at a higher/meta level – ‘is this a good heuristic/decision tree structure or not’. Maybe this is relevant to what Hofstadter references to as ‘rut-breaking’: noticing you’re in an attractor/habit and moving to a higher level. Although, he used this to illustrate the continuum of consciousness; less conscious organisms seem to follow simpler systems/routines of behavior, maybe also overlapping with ‘self-awareness’. (What is the mechanism for noticing that you’re following a rut and breaking out of it? That you can look at things from up a level? Perhaps, an awareness that there’s another option.)

But then there is also a parameter of how to do recombination, the grind size of memorized chunks. LLMs look intelligent because of lot of good problem solving involves a significant chunk of memorization. But there seems to be an intuitive ‘sweet spot’ of memorization vs recombination. For example, catch phrases/cliches (what Orwell criticizes) can be seen as copying too much. We have a sense of the degree of recombination that is adequate to express individuality. However, the search space for solutions probably increases with higher resolution though (higher resolution means higher degrees of freedom, and higher computational cost).

So the question might be reframed as, how much recombination can current NNs/LLMs do? It can do poetry/etc, which maps to some intuition of creativity/innovation. But I wonder if there’s something fundamentally different in the character of creativity in the context of poetry versus technology. Maybe something like, there is a stricter requirement for something like ‘coherency’/’consistency’ in a scientific/technical solution – there is a stronger aesthetic for functionality/interaction with physical systems.

Do we want AI to be able to identify Galileo’s heliocentric model as ‘better’ if it existed in the times when the dominant model was Earth-centric? It might be a different question to ask if what it takes to get a good industry worker (routine knowledge) vs a good AI scientist (frontier knowledge). Independent thinking/recombination is needed to do better than what is the standard, but it can easily do worse (probably why ‘mutation’ has a negative connotation, but it is the means by which evolution works at all).

De-novo problem solving/updating/adapting/antifragility is only important if the environment changes. I think evolution is better framed as optimizing for adaptability (what is optimal depends on the environment, and the environment is constantly changing). I’ve been using learning systems and adaptive systems somewhat interchangeably to describe my interests. But LLMs feel solidly like learning systems – and I think I’m more interested in adaptive systems (which, in my current mind, is a learning system plus more).

Tentative hypothesis:

I am biased in that my interest is in, what are good mechanisms to be good at problem solving? How to automate finding good models? Or maybe the tldr is that I’m disappointed/disinterested in current AI because they’re not independent thinkers. (So the next question is, what would it take to make one? human and/or machine, do we know? maybe, do they need god/aesthetic, if so what would that look like?)

[ ]