Why AI is in Everything Now
Why AI is in Everything Now
Source (Tumblr), alternate frontend
31 comments
-
My perspective is that research machine learning has been chugging along reasonably for years, without any fuss, until Altman went against OpenAI’s mandate and commercialized (and marketed) ChatGPT.
Now it’s enshittified. And ruining shit. Thanks for that.
One example I often cite is the utter shock in finance land at Deepseek R1 coming out when the research/tinkerer community saw that coming miles away.
-
They’re so lost in the Altman sauce they actually want to make DeepSeek illegal because it will bankrupt so many American investors if someone makes a useful, free alternative that runs locally for pennies compared to the power-sucking cloud based slop that’s already using more power than all cryptocurrencies combined
Motherfucking Microsoft was talking about re-opening three mile island to power copilot for Christ’s sake. Couldn’t get America to consider nuclear for 50 years and now suddenly we MUST have it
The rich are all gambling addicts and we’re at their mercy
-
There’s a misconception that Deepseek is locally runnable, where the “full” model is actually overly large, and the smaller variants are not the same thing
But yeah, 100% agree with the point. Altman just wants to shut them out.
-
both versions are theoretically runnable locally, it just requires extraordinary expensive GPUs with more than 100GB of VRAM, or certain integrated GPUs that can make use of system RAM and slowly chug along.
-
One can use ik_llama.cpp to run the dense layers on a 3090/4090 and offload the MoE layers to a threadripper/EPYC CPU, with full support for its MLA attention scheme, at quite reasonable speeds. In other words, the full deepseek is surprisingly usable locally if you shoot for the right setup.
And now we have something similar from Qwen, at "only" 235B.
-
-
-
-
31 comments