Skip Navigation
Researchers claim GPT-4 passed the Turing test
  • Humans are really bad at determining whether a chat is with a human or a bot

    Eliza is not indistinguishable from a human at 22%.

    Passing the Turing test stood largely out of reach for 70 years precisely because Humans are pretty good at spotting counterfeit humans.

    This is a monumental achievement.

  • Meta pauses plans to train AI using European users' data, bowing to regulatory pressure
  • Sure, thanks for your interest. It's an incomplete picture, but we can think of LLMs as an abstraction of all the meaningful connections within a dataset to a higher dimensional space - one that can be explored. That alone is an insane accomplishment that is changing some of the pillars of data analysis and knowledge work. But that's just the contribution of the "Attention is All You Need" paper. Many implementations of modern generative AI combine LLM inference in agentic networks, with GANs, and with rules-based processing. Extracting connections is just one part of one part of a modern AI implementation.

    The emergent properties of GPT4 are enough to point toward this exponential curve continuing. Theory of mind (and therefore deception) as well as relational spatial awareness (usually illustrated with stacking problems) developed solely from increasing the parameter count describing the neural network. These were unexpected capabilities. As a result, there is an almost literal arms race on the hardware side to see what other emergent properties exist at higher model sizes. With some poetic license, we're rending function from form so quickly and effectively that it's seen by some as freeing and others as a sacrilege.

    Some of the most interesting work on why these capabilities emerge and how we might gain some insight (and control) from exploring the mechanisms is being done by Anthropic and by users at Hugging Face. They discovered that when specific neurons in Claude's net are stimulated, everything it responds with will in some way become about the Golden Gate Bridge, for instance. This sort of probing is perhaps a better route to progress than blindly chasing more size (despite its recent success). But only time will tell. Certainly, Google and MS have had a lot of unforced errors fumbling over themselves to stay in what they think is the race.

  • Is Science Fiction Inherently Hopeful?
  • I had no idea emo ducks admired humanity like that. Imma try and be better for y'all, bring that good bread. Wait, is bread bad for you now? I think I saw that in my feed while doom scrolling.

  • And here I am just wanting to spray paint "bitch" on my sub's car :'(
  • FetLife is a relatively respected kink social media platform. It's not about hooking up (though that certainly happens, just like on any online platform - hell, I knew people that later married that met in EQ). From my limited experience there, it's mostly about making everyone feel less ostracized. Of course, they have to have very explicit rules about consent, or that turns into a predator's playground - but again, that's true of any social media platform.

  • Why do we have Pride?
  • At least gay has some positive etymological history as well as negative. F-- only has two meanings, and the vastly more common one is incredibly violent. The only thing I've seen remotely close to trying to "take that word back" is maybe Martin in the Simpsons in a throw-away gag about his pure nerdy naivete. And that's not particularly close.

  • Researchers claim GPT-4 passed the Turing test
  • Aye, I'd wager Claude would be closer to 58-60. And with the model probing Anthropic's publishing, we could get to like ~63% on average in the next couple years? Those last few % will be difficult for an indeterminate amount of time, I imagine. But who knows. We've already blown by a ton of "limitations" that I thought I might not live long enough to see.

  • Researchers claim GPT-4 passed the Turing test
  • On the other hand, the human participant scored 67 percent, while GPT-3.5 scored 50 percent, and ELIZA, which was pre-programmed with responses and didn’t have an LLM to power it, was judged to be human just 22 percent of the time.

    54% - 67% is the current gap, not 54 to 100.

  • Removed
    What's the point of living for someone like me?
  • Sounds like he needs someone with training to help him through retraining his behavioral/thought patterns, something a functional social system would provide if those were as common as comment culture.

  • The Future of Large Language Model Pre-training is Federated

    Also See: Worldwide Federated Training Of Language Models

    Claude's Summary:

    The two papers, "Worldwide Federated Training of Language Models" by Iacob et al. and "The Future of Large Language Model Pre-training is Federated" by Sani et al., both propose using federated learning (FL) as a new paradigm for pre-training large language models (LLMs). The main ideas are:

    1. FL allows leveraging more data and compute resources from multiple organizations around the world, while keeping the data decentralized and private. This can enable training larger LLMs on more diverse data compared to centralized training.

    2. FL relaxes synchronization requirements and reduces communication overheads compared to data-parallel distributed training, making it feasible for geographically distributed participants with varying hardware and connectivity.

    3. The papers present systems and algorithms for enabling efficient federated pre-training of LLMs at billion-parameter scales. Key techniques include allowing participants to modulate their amount of local training based on resource constraints, and partially personalizing models to clusters of participants with related data.

    4. Experimental results show federated LLM pre-training can match or exceed centralized training performance, with the performance gap narrowing as model size increases to billions of parameters. Larger federated models also converge faster and are more robust.

    5. Challenges include data and hardware heterogeneity across participants. The papers propose techniques like adaptive aggregation and load balancing to mitigate these issues.

    In summary, the papers argue federated learning is a promising new direction for democratizing LLM pre-training by allowing many more organizations to collaboratively train large models on their combined data and compute resources. Let me know if you would like me to expand on any part of the summary or papers in more detail.

    8
    InitialsDiceBearhttps://github.com/dicebear/dicebearhttps://creativecommons.org/publicdomain/zero/1.0/„Initials” (https://github.com/dicebear/dicebear) by „DiceBear”, licensed under „CC0 1.0” (https://creativecommons.org/publicdomain/zero/1.0/)HA
    Hackworth @lemmy.world
    Posts 2
    Comments 143