Skip Navigation

InitialsDiceBearhttps://github.com/dicebear/dicebearhttps://creativecommons.org/publicdomain/zero/1.0/„Initials” (https://github.com/dicebear/dicebear) by „DiceBear”, licensed under „CC0 1.0” (https://creativecommons.org/publicdomain/zero/1.0/)HE
Posts
2
Comments
851
Joined
2 yr. ago

  • Hallucinations are an unavoidable part of LLMs, and are just as present in the human mind. Training data isn’t the issue. The issue is that the design of the systems that leverage LLMs uses them to do more than they should be doing.

    I don’t think that anything short of being able to validate an LLM’s output without running it through another LLM will be able to fully prevent hallucinations.

  • The main disadvantage I can think of would involve a situation where your email (and possibly also other personal data) was exposed without your name attached. It’d be possible for your DLN and/or SSN (or the equivalents for other countries) and email to be exposed without your name being exposed, for example. This wouldn’t have to be a breach - it could be that, for privacy purposes, certain people working with accounts simply don’t get visibility to names.

    It’s also feasible that an employee might have access to your full name but only to partially masked email addresses. So if your email is site-firstname-lastname@example.com and they see site-firstname-****@domain.com, they can make an educated guess as to your full email address.

    Also, if your email were exposed by itself and someone tried to phish you, it would be more effective if they knew your name.

  • ACLU, is this really that high a priority in the list of rights we need to fight for right now?

    You say this like the ACLU isn’t doing a ton of other things at the same time. Here are their 2024 plans, for example. See also https://www.aclu.org/news

    Besides that, these laws are being passed now, and they’re being passed by people who have no clue what they’re talking about. It wouldn’t make sense for them to wait until the laws are passed to challenge them rather than lobbying to prevent them from being passed in the first place.

    wouldn't these arguments fall apart under the lens of slander?

    If you disseminate a deepfake with slanderous intent then your actions are likely already illegal under existing laws, yes, and that’s exactly the point. The ACLU is opposing new laws that are over-broad. There are gaps in the laws, and we should fill those gaps, but not at the expense of infringing upon free speech.

  • What makes sourcehut better?

    From a self-hosting perspective, it looks like much more of a pain to get it set up and to keep it updated. There aren’t even official Docker images or builds. (There’s this and the forks of it, but it’s unofficial and explicitly says it’s not recommended for prod use.)

  • Yes, but only in very limited circumstances. If you:

    1. fork a private repo with commit A into another private repo
    2. add commit B in your fork
    3. someone makes the original repo public
    4. You add commit C to the still private fork

    then commits A and B are publicly visible, but commit C is not.

    Per the linked Github docs:

    If a public repository is made private, its public forks are split off into a new network.

    Modifying the above situation to start with a public repo:

    1. fork a public repository that has commit A
    2. make commit B in your fork
    3. You delete your fork

    Commit B remains visible.

    A version of this where step 3 is to take the fork private isn’t feasible because you can’t take a fork private - you have to duplicate the repo. And duplicated repos aren’t part of the same repository network in the way that forks are, so the same situation wouldn’t apply.

  • Misleading title.

    The title literally spells out the concern, which is that code that is in a private or deleted repository is, in some circumstances, visible publicly.

    What title would you propose?

    If my thing was public in the past, and I took it private, the old public code is still public.

    The “Accessing Private Repo Data” section covers a situation where code that has always been private becomes publicly visible.

  • The models I’m talking about that a PI 5 can run have billions of parameters, though. For example, Mistral 7B (here’s a guide to running it on the PI 5) has roughly 7 Billion parameters. By quantizing each parameter to 4 bits, it only takes up 3.5 GB in RAM, making it easily fit in the 8 GB model’s memory. If you have a GPU with 8+ GB of VRAM (most cards from the past few years have 8 GB or more - the 1070, 2060 Super, and 3050 and each better card in that generation hit that mark), you have enough VRAM and more than enough speed to run Q4 versions of the 13B models (which have roughly 13 Billion parameters), and if you have one with 24 GB of VRAM, like the 3090, then you can run Q4 versions of the 30B models.

    Apple Silicon Macs can also competently run inference for these models - for them, the limiting factor is system RAM, not VRAM, though. And it’s not like you’ll need a Mac as even Microsoft is investing in ARM CPUs with dedicated AI chips.

  • I don't see how LLMs will get into the households any time soon. It's not economical.

    I can run an LLM on my phone, on my tablet, on my laptop, on my desktop, or on my server. Heck, I could run a small model on the Raspberry PI 5 if I wanted. And none of those devices have dedicated chips for AI.

    The problem with LLMs is that they require immense compute power.

    Not really, particularly if you’re talking about the usage of smaller models. Running an LLM on your GPU and sending it queries isn’t going to use more energy than using your GPU to game for the same amount of time would.

  • Keychain flashlight would be too weak compared to a phone. Leave it at home and if you notice the need, buy a proper one.

    Depends on what’s meant by “keychain flashlight.”

    I carry a Wukkos TS10 on my keychain and it gets quite bright - over a thousand lumens at peak compared to around 50 for the iPhone 15 Pro Max (so I’m assuming most other phones’ flashlights aren’t much brighter). There are several other flashlights that are similarly competent and compact. Mine runs on a 14500 battery, which is the size of a AA battery, so it’s a bit bulkier than the AAA-powered flashlight I started carrying over a decade ago. That one capped out at 150 lumens, so the upgrade was as much of an improvement over it as it was over a phone flashlight.

    Looking that up made me curious about how good a 10440-powered flashlight could be (a 10440 is the size of a AAA battery) and apparently they can get up to 500 lumens - at least, the ReyLight Pineapple mini-Ti can. That one’s 50 USD (on sale from 60), which is more than a lot of alternatives, but none of them get quite as bright. For example, I saw that the Mankerlight E02 II is recommended a lot and it gets nearly as bright (420 lumens) for less than half the price (23 USD).

    The Acebeam UC15 is explicitly a “keychain flashlight” - it even has the same form factor as the coin cell battery powered ones. It runs on two AAA (or 10440) batteries, though, so it’s a bit larger than them. Some I just saw top out at 12 lumens; by comparison, the UC15 tops out at 1000.

  • I disagree, unless you mean nautical piracy. The difference is that people are being swindled into paying them for a service that’s less effective than they represent it as being, whereas with piracy the only “loss” anyone suffers is speculative at best. What they’re doing is more like fraud, honestly. Unfortunately that speculative loss’s value is codified into law and the fraud is probably permitted as long as they have some fine print somewhere covering their asses.

  • Thanks for that! I recommend anyone who wants to minimize risk to follow their instructions for self-hosting:

    Is the source code available and can I run my own copy locally?

    Yes! The source code is available on Github. Its a simple static HTML application and you can clone and run it by opening the index.html file in your browser. When run locally it should work when your computer is completely offline. The latest commits in the git repository are signed with my public code signing key.

  • Generally people don’t memorize private keys, but this is applicable when generating pass phrases to protect private keys that are stored locally.

    Leaving this here in case anyone wants to use this method: https://www.eff.org/dice