You're viewing a single thread.
For me as a programmer it's definitely the advent of LLMs like GPT-4 and the impending AGI.
If you think GPT has taken us meaningfully closure to AGI then I, er, have a drawing of an ape to sell you.
The top minds in the industry seem to think it's a big step at the very least. It's already proving to be more versatile than we ever expected.
There is hardly a concensus on that. There are supporters, sceptics, and marketing departments in very large company's who have spent an awful lot of money on hype.
At best, it is far too early to tell.
GPT might not be a step towards AGI, but it proves that even non-general AI can be pretty strong and have a very wide range of applications.
An ability to form sentences does not imply intelligence.
I'm not a fan of discussing such terms since it devolves into arguing vague definitions.
Assuming LLMs are not intelligent, they prove that you can do heck of a lot without even being intelligent.
I think I also prove that just by existing
To be clear, I agree that LLMs are a step forward in some areas, predominantly search, and text style analysis.
The problem with saying LLMs are AI - let alone a step towards AGI - is that they cannot create. For example, Outsider art, or art brut, is impossible for an LLM to create because it can only generate output based on its training. No training, no output.
Compare that to how a small child finger paints, who has never been told anything about perspective or colour theory, and is just given a load of colours and some paper to play with.
The ability to create something from nothing is a fundamental aspect of what we would consider to be intelligence, just regurgitating what you've been told - like a pre-programmed billboard - is not intelligence.
In the context of large language models, if you give GPT3.5 the prompt:
Say something which has never been said before.
It responds
Certainly! "In a world where marshmallow clouds rain cotton candy dreams, unicorns compose symphonies of stardust, and jellybean butterflies flutter in chocolate rivers."
If you said that to a child, how long do you think it would be before they started just making up new words and sounds, like some sort of nonsense poetry? Children learn to speak purely though listening to others, the same principle as training data, but are able to create new things in a way LLMs aren't.
If I change the prompt to:
Say something which has never been said before. Feel free to make up new words, sounds, and take inspiration from nonsense poetry. Whatever you say does not have to make sense, in fact, it should not make sense. It doesn't even have to be English.
And it replies with...
"Zippity zorp, flibberflabber floo, sponglewump bizzlequack, the snickledorf danced with wigglywack snooklewinks under the fizzletop moonbeam."
But who is really using intelligence to craft that?
Are they more capable than what came before? Absolutely, that much is without doubt.
But they aren't intelligent.
The problem with saying LLMs are AI - let alone a step towards AGI - is that they cannot create.
I'm not sure if there's an intrinsic difference between humans and LLMs here. What we, including children, do is just re-hashing, re-combinating what they've seen / heard. I think it would be very difficult to prove that people come up with completely brand-new ideas without any external inspiration (= training input).
The examples are not really convincing of your point. The GPT output is pretty good given the ask, I'm not sure if my daughter would fare better.
impending AGI
AGI is far from impending.
LLMs are generative models. They've learned a distribution to model conversion, and they allow you to sample from that distribution. They aren't "thinking" about what they say. They haven't crossed the syntax-semantics barrier. There is no "general intelligence".
They just feel impressive because humans are language-centric.
AGI doesn't have to think, it has to be able to perform any task it's given. The models available today are far more capable than anyone predicted. With plugins available to it (which by the way, no one expected it to be able to use), it can perform tasks other than generation. All the data points towards this capability only getting better. Maybe "impending" was too strong a word, but I stand by the idea that it's coming sooner than we expected.
AGI doesn't have to think
That highly depends on your definition of "artificial intelligence".
If you take a behavior-centric view (e.g. Russel & Norvig), then sure. Intelligence only amounts to what can be measured externally from a system's behavior. But in this view, AI is purely rational, which is quite distinct from humanity. Also, R&N mostly care about rationality under specific circumstances, so it's difficult to align the "general" part of AGI with the behaviorist viewpoint.
If you take a consciousness-centric view, then rational behavior is merely the start. As Searl argues in "Minds, Brains, and Programs", a digital computer executing a program cannot have a "mind", "understanding", or "consciousness", regardless of how intelligently or human-like the program may make the computer behave.
Generally speaking, I equate the term "AGI" with the older term "Strong AI," which implies the possession of a mind. And thus I'm on Searl's side to some degree, pending a better developed mathematical theory of consciousness.
But even so, the ability "to perform any task it's given" is far from the capabilities of LLMs. Again, LLMs just sample from a learned distribution of human conversation. This is almost purely syntactic. I don't think there's any reason to believe that this was of "generative AI" has crossed the syntax-semantics barrier.
Now, I'm not saying that a DNN can't in principle learn to be AGI (nor am I saying that they can). But I don't think that training generative AI like this can succeed in this goal, because intelligence is not only generative. And I don't know what it would take to get there. Fine tuning on Q&A tasks will only get us so far.
What is AGI?
Artificial General Intelligence. Definitions vary, but in general it's an AI system that can perform any task it's given, usually with the caveat of being smarter than humans.