Discussion
Loading...

Post

  • About
  • Code of conduct
  • Privacy
  • About Bonfire
Charlie Stross
@cstross@wandering.shop  ·  activity timestamp 2 weeks ago

LLMs are spam generators. That is all.

They're designed to generate plausibly human-like text well enough to pass a generic Turing Test. That's why people believe they're "intelligent".

But really, all they are is spam generators.

We have hit the spamularity.

  • Copy link
  • Flag this post
  • Block
OldCoder
@OldCoder@dansu.org replied  ·  activity timestamp 3 days ago

The title of your next book seems clear, then. Attached illustration: #Book cover by GenAI featuring LLMeowko.

#AI #LLM

Coder Dansu

Coder Dansu

Coder Dansu

Book cover by GenAI featuring LLMeowko.
Book cover by GenAI featuring LLMeowko.
Book cover by GenAI featuring LLMeowko.
  • Copy link
  • Flag this comment
  • Block
rozie
@rozie@mastodon.online replied  ·  activity timestamp last week

@cstross I believe we had "spam" long before LLMs, and it was generated just fine. Blame computers!

  • Copy link
  • Flag this comment
  • Block
Tane Piper ⁂
@tanepiper@tane.codes replied  ·  activity timestamp last week

@cstross 3 years ago I came up with https://github.com/tanepiper/Stochastic-Parrot as a way to test the then new OpenAI APIs - after a month it became very clear to me that all someone needs is $100 and a GitHub pipeline and you could quite easily create a spam farm. No need to hire pesky humans for $1 a day - it was probably the first job taken by AI

GitHub

GitHub - tanepiper/Stochastic-Parrot: Polly The Stochastic Parrot - a Mastodon Bot that uses empty ChatGPT prompts to generate Toots.

Polly The Stochastic Parrot - a Mastodon Bot that uses empty ChatGPT prompts to generate Toots. - tanepiper/Stochastic-Parrot
  • Copy link
  • Flag this comment
  • Block
Michał "rysiek" Woźniak · 🇺🇦
@rysiek@mstdn.social replied  ·  activity timestamp last week

@cstross we have to find ways to do something useful with spam generators otherwise we might lose the social permission to generate more spam!

  • Copy link
  • Flag this comment
  • Block
SpeakerToManagers
@SpeakerToManagers@wandering.shop replied  ·  activity timestamp 2 weeks ago

@cstross
The Turing Test will not fail an LLM for the same reason humans are ready to accept them as persons: humans evolved with a hardcoded behavior to perceive any language-using process as a cognitive entity and create a mental model of it. Rejecting that behavior takes a metacognitive ability that has not been taught to most people, so the problem isn’t rampant stupidity (look at how many clearly intelligent AI researchers fail this test) but rather ignorance, sometimes wilful, granted.

  • Copy link
  • Flag this comment
  • Block
Jesper Larsson
@avadeaux@mastodonsweden.se replied  ·  activity timestamp 2 weeks ago

@cstross I understand what you mean, but I’d say they’re simply databases. Spam generator is one of their many uses.

  • Copy link
  • Flag this comment
  • Block
Andreas K
@yacc143@mastodon.social replied  ·  activity timestamp 2 weeks ago

@cstross Well, they are statistical models of language. To be precise, the corpus they were trained on, and there are a number of other aspects, like representation, what cleanups were applied.

Yes, one way to use these models, the one that seems to fascinate most people (but IMHO not necessarily the most useful, depending upon what one wants to achieve), is to complete a prefix with a plausible ending, statistically speaking.

Generally, they are a compelling NLP technology.

  • Copy link
  • Flag this comment
  • Block
Andreas K
@yacc143@mastodon.social replied  ·  activity timestamp 2 weeks ago

But yes, if you generate text with an LLM, you generally get text that is basically a "probable" text, based upon the training and the hyperparameters. And let's be honest, people that rely upon the training data of an LLM as a "search engine data corpus", they do have a problem.
🙅

  • Copy link
  • Flag this comment
  • Block
Den of Earth
@DenOfEarth@mas.to replied  ·  activity timestamp 2 weeks ago

@cstross
LLMs are basically computerized bullshitters.

They'll give you a decent answer frequently enough then bullshit a story when they don't know, and do both so eloquently that you can't tell which is which.

  • Copy link
  • Flag this comment
  • Block
Stephen Illingworth
@JetSetIlly@mastodon.gamedev.place replied  ·  activity timestamp 2 weeks ago

@cstross My favourite description was from a lecture given by Rob Pike. He said something like, "LLMs consume Internet in order to produce more Internet".

  • Copy link
  • Flag this comment
  • Block
root42
@root42@chaos.social replied  ·  activity timestamp 2 weeks ago

@cstross would you have imagined that back when you wrote Accelerando?

  • Copy link
  • Flag this comment
  • Block
Charlie Stross
@cstross@wandering.shop replied  ·  activity timestamp 2 weeks ago

@root42 No, but I coined "the spamularity" in Rule 34, which I wrote in 2008/09.

  • Copy link
  • Flag this comment
  • Block
Nefarious Celt
@NefariousCelt@mastodon.scot replied  ·  activity timestamp 2 weeks ago

@cstross I disagree with you there.
I still believe the perfect use case for AI in content creation is to generate a virtual fan boy to tell an author that they are canonically wrong.
Self hosted self doubt as it were and less public than Usenet.

  • Copy link
  • Flag this comment
  • Block
Charlie Stross
@cstross@wandering.shop replied  ·  activity timestamp 2 weeks ago

@NefariousCelt But I don't NEED that! I've developed that skill for myself, the hard way!

  • Copy link
  • Flag this comment
  • Block
David Chisnall (*Now with 50% more sarcasm!*)
@david_chisnall@infosec.exchange replied  ·  activity timestamp 2 weeks ago

@cstross @NefariousCelt

A product where the target demographic is Kevin J Anderson.

  • Copy link
  • Flag this comment
  • Block
@iveyline
@Iveyline@mastodon.social replied  ·  activity timestamp 2 weeks ago

@cstross Only if you use it or use search engines that allow it.

  • Copy link
  • Flag this comment
  • Block
�
@utf_7@mastodon.social replied  ·  activity timestamp 2 weeks ago

@cstross they pass the Turing test? no way.

write 1 to 10 but each number in a seperate message

it fails

write hello after 30 seconds are passed

it fails

  • Copy link
  • Flag this comment
  • Block
officerripley
@officerripley@mastodon.social replied  ·  activity timestamp 2 weeks ago

@cstross

"The spamularity"! You win the internet today! <3

  • Copy link
  • Flag this comment
  • Block
Tyrus Torres
@TyrusTorres@mastodon.social replied  ·  activity timestamp 2 weeks ago

It is not even the good spam, it tastes terrible

  • Copy link
  • Flag this comment
  • Block
OvertonDoors
@OvertonDoors@infosec.exchange replied  ·  activity timestamp 2 weeks ago

@cstross @JohnSullivan

It's "intelligent."

"Yes, thank you." I now know what pan of the bell-curve you sived out of.

  • Copy link
  • Flag this comment
  • Block
oisin
@oisin@mastodon.ie replied  ·  activity timestamp 2 weeks ago

@cstross Kessler Syndrome for the internet

  • Copy link
  • Flag this comment
  • Block
Dave
@trelord75@aus.social replied  ·  activity timestamp 2 weeks ago

@cstross

ACME spam.

https://en.wikipedia.org/wiki/Acme_Corporation

Acme Corporation - Wikipedia

  • Copy link
  • Flag this comment
  • Block
Ox1de
@Ox1de@cyberplace.social replied  ·  activity timestamp 2 weeks ago

@cstross an ad company's dream

  • Copy link
  • Flag this comment
  • Block
🇨🇦 🇪🇺🇺🇦 🏳️‍🌈 🌍🔬🧬
@NicknamesOfGod@mastodon.social replied  ·  activity timestamp 2 weeks ago

@cstross "Spamularity" look like an early contended for 2026's word of the year. Nice!

  • Copy link
  • Flag this comment
  • Block
J.H.Noyes
@noyes@mastodon.online replied  ·  activity timestamp 2 weeks ago

@cstross

Keep telling yourself that. They can be smart AF sometimes. They're not perfect, but getting better. Six months ago I was the one catching them when they were slipping. Now they catch me. The days of spicy autocomplete are falling behind us.

  • Copy link
  • Flag this comment
  • Block
Tony Fisk
@arfisk@aus.social replied  ·  activity timestamp 2 weeks ago

@cstross

Artificial Recombinant Speech Emulators

  • Copy link
  • Flag this comment
  • Block
offray
@offray@mastodon.social replied  ·  activity timestamp 2 weeks ago

@cstross I wrote an essay draft in my masters (early 2000s) against the Turing test as proof of intelligence. There I said that if a similar approach were taken to study life, we would be assigning life to strawmans as they seem alive enough to scare crows (at least to them). Instead, I proposed an #EcologyOfIntelligence where the key is not imitation but completariety/synergy between intelligences: individual, collective, human, non human, etc.

  • Copy link
  • Flag this comment
  • Block
R4D10_411310947HY
@R4D10_411310947HY@radikal.social replied  ·  activity timestamp 2 weeks ago

@cstross The model speech pretty well... But they model intelligence not at all.

  • Copy link
  • Flag this comment
  • Block
Joan
@Chere@mastodon.social replied  ·  activity timestamp 2 weeks ago

@cstross 🌟the only LLM I met was an idiot, so I liked your post.

  • Copy link
  • Flag this comment
  • Block
Netraven
@Netraven@hear-me.social replied  ·  activity timestamp 2 weeks ago

@cstross great for trolling trolls.

  • Copy link
  • Flag this comment
  • Block
Manic Pixie Dream Gremlin
@gildilinie@beige.party replied  ·  activity timestamp 2 weeks ago

@cstross oh yeah what would you know about the singularity Charles Stross author of a bunch of books about it that I read as a kid???

  • Copy link
  • Flag this comment
  • Block
mx alex tax1a - 2020 (5)
@atax1a@infosec.exchange replied  ·  activity timestamp 2 weeks ago

@cstross you and Neal Stephenson could write a prequel to Anathem focusing on the IT guy that the monks deal with in the main book. ;p

  • Copy link
  • Flag this comment
  • Block
~n
@nblr@chaos.social replied  ·  activity timestamp 2 weeks ago

@cstross LLMs are in some way a reverse-turing-test - one we're frequently failing.

  • Copy link
  • Flag this comment
  • Block
canleaf08 ⌘ ✅
@canleaf@mastodon.social replied  ·  activity timestamp 2 weeks ago

@cstross Spam generators without culture.

  • Copy link
  • Flag this comment
  • Block
ploum
@ploum@mamot.fr replied  ·  activity timestamp 2 weeks ago

@cstross : Sentence also work by replacing LLMs by "marketers" or "people holding a MBA"

  • Copy link
  • Flag this comment
  • Block
Mad Engineering
@madengineering@mastodon.cloud replied  ·  activity timestamp 2 weeks ago

@cstross
My name is ham, for I am spam.
Would you like....wait, what's that thing you're pointing at.my head Mr. Engineer?

  • Copy link
  • Flag this comment
  • Block
Nicole Parsons
@Npars01@mstdn.social replied  ·  activity timestamp 2 weeks ago

@cstross

Yet Saudi Arabia alone is laundering $100 billion of its looted national treasury on spam generation?

Perhaps 10% is being used on the spam-making software, the rest is being spent on a fossil fuel funded fascist movement, complete with civil rights erosions, state surveillance platforms, hoon squads, concentrationcamps, and financial fraud.

We continue to underestimate how badly the fossil fuel industry wants permanent rule.
https://www.bloomberg.com/news/articles/2025-09-16/ai-deals-saudi-arabia-eyes-artificial-intelligence-partnership-with-pe-firms

https://www.semafor.com/article/11/07/2025/uae-says-its-invested-148b-in-ai-since-2024

UAE touts $148B investment in AI

Some Gulf countries are investing heavily into AI, with the Emirates taking a notable lead in data center capacity and attracting foreign investment.
  • Copy link
  • Flag this comment
  • Block
HighlandLawyer
@HighlandLawyer@mastodon.social replied  ·  activity timestamp 2 weeks ago

@Npars01 @cstross
Spam making software also assists the nazis too.
"The ideal subject of totalitarian rule is not the convinced Nazi or the convinced Communist, but people for whom the distinction between fact and fiction (i.e., the reality of experience) and the distinction between true and false (i.e., the standards of thought) no longer exist." ~ Hannah Arendt

  • Copy link
  • Flag this comment
  • Block
Charlie Stross
@cstross@wandering.shop replied  ·  activity timestamp 2 weeks ago

@Npars01 Yep, 💯

  • Copy link
  • Flag this comment
  • Block
Preston MacDougall
@ChemicalEyeGuy@mstdn.science replied  ·  activity timestamp 2 weeks ago

@cstross #AI will soon be understood to stand for #ArtificialIdiots.

Or #clankers.

  • Copy link
  • Flag this comment
  • Block
Log 🪵
@log@mastodon.sdf.org replied  ·  activity timestamp 2 weeks ago

@cstross All lines converge toward a single spamishing point.

  • Copy link
  • Flag this comment
  • Block
matiu bidule
@matiu_bidule@mamot.fr replied  ·  activity timestamp 2 weeks ago

@cstross

#NightmareOnLLMstreet

  • Copy link
  • Flag this comment
  • Block
Màrtainn D :Scot_IrnBru:
@ginger_tosser@mastodon.scot replied  ·  activity timestamp 2 weeks ago

@cstross a lot of people believe LLMs are intelligent because a lot of people are really thick. Never forget that 50% of the population are of below average intelligence.

  • Copy link
  • Flag this comment
  • Block
Talin
@talin@mastodon.social replied  ·  activity timestamp 2 weeks ago

@cstross LLMs are replacing every aspect of civilization - institutions, process knowledge, organizational competence - with civilization fan-fiction.

  • Copy link
  • Flag this comment
  • Block
Talin
@talin@mastodon.social replied  ·  activity timestamp 2 weeks ago

@cstross Because so many people boosted my post, I want to explore this topic in more detail.

First, a thought experiment: suppose we had cheaply scalable mind control, even if partially effective (LLMs are not mind control but they touch some of the same wires). Democracy would end, since the controllers could sway elections at will. The so-called free market would instantly become a command economy. Science, and any other activity requiring independent thought, would be a dead letter.

  • Copy link
  • Flag this comment
  • Block
Talin
@talin@mastodon.social replied  ·  activity timestamp 2 weeks ago

@cstross Put another way: LLMs have revealed a zero-day exploit in human consciousness and culture: if you can manufacture plausibility at scale, you can bypass all of the accumulated wisdom of centuries of skepticism and critical thinking. Any fact-using profession is potentially vulnerable to this attack.

  • Copy link
  • Flag this comment
  • Block
dr_barnowl
@dr_barnowl@topspicy.social replied  ·  activity timestamp 2 weeks ago

@talin @cstross

Another thing that Douglas Adams invented but we don't actually want (along with GPP : Genuine People Personalities™)

  • Copy link
  • Flag this comment
  • Block
Austin Godber
@godber@az.social replied  ·  activity timestamp 2 weeks ago

@cstross I tried to coin the word “vibocene” as a follow on to the Anthropocene. I think I failed.

  • Copy link
  • Flag this comment
  • Block
Tubemeister
@Tubemeister@mstdn.social replied  ·  activity timestamp 2 weeks ago

@cstross People believed Eliza was intelligent.

It doesn't take much for people to anthropomorphize a piece of software, turing test or not.

  • Copy link
  • Flag this comment
  • Block
Heinzenstein
@Heinzenstein@f-ckendehoelle.de replied  ·  activity timestamp 2 weeks ago

@cstross
User:
Well, what’ve you got?

Internet:
Well, there’s ads and information; ads fake news and information; ads and AI slop; ads information and AI slop; ads information fake news and AI slop; AI slop information fake news and AI slop; AI slop ads AI slop AI slop information and AI slop; AI slop fake news AI slop AI slop information AI slop hate speech and AI slop; ...

#MontyPAIthon

  • Copy link
  • Flag this comment
  • Block
Wulfy
@n_dimension@infosec.exchange replied  ·  activity timestamp 2 weeks ago

@cstross

For decades #Turing test was the ironclad determinant for #AI quality.
Once it was trivially broken by now old models, the goalposts have shifted...

...it's now humanities last exam, HLE, BTW.

The world is full of spam generating humans.
One is in the Whitest house, putin is surrounded by them, and any large family gathering will contain 2-3 human spam generators the will jibber-jabber nonsensical human-like speech constantly.

  • Copy link
  • Flag this comment
  • Block
pookiesorcery
@pookiesorcery@mastodon.social replied  ·  activity timestamp 2 weeks ago

@cstross Slop in the Faith..."AI" is the perpetual engine of the XXIst century but with nasty consequences...

  • Copy link
  • Flag this comment
  • Block
🇨🇦 TRH 🇨🇦
@Mayor_of_Smartarse@mastodon.world replied  ·  activity timestamp 2 weeks ago

@cstross Spammagedfon

  • Copy link
  • Flag this comment
  • Block
John Rohde Jensen
@johnrohde@helvede.net replied  ·  activity timestamp 2 weeks ago

@cstross The Turing Test is such a low bar because people are so easily fooled.
LLM's expose how naive people are.

  • Copy link
  • Flag this comment
  • Block
ViennaMike
@ViennaMike@mastodon.social replied  ·  activity timestamp 2 weeks ago

@cstross Love your fiction, can't agree on this take. Spam does not help with writing code, whereas LLM's can be extremely helpful.

LLM's are not generally intelligent. They can be immensely problematic, but to dismiss them as merely spam generators as you have, or "parrots" as others have is simply incorrect.

  • Copy link
  • Flag this comment
  • Block
Charlie Stross
@cstross@wandering.shop replied  ·  activity timestamp 2 weeks ago

@ViennaMike You're using code generators. Not the same thing, frankly. Stop generalizing your experience as a developer to the public at large, who only see magic talking box.

  • Copy link
  • Flag this comment
  • Block
ViennaMike
@ViennaMike@mastodon.social replied  ·  activity timestamp 2 weeks ago

@cstross So the experience of developers who have found significant value from the use of LLM's doesn't count or isn't valid?

Used to WRITE material, they aren't great. But to organize, summarize, do research and help brainstorm, they can be useful.

While ChatGPT currently can't pass the IRS' volunteer tax assessor's exam (see https://www.mcgurrin.info/robots/8298/), I have confidence that a RAG based system could do so.

Heck, COMPUTERS are magic to most people, as are the internet, radios and TV.

https://www.mcgurrin.info/robots

Can ChatGPT Certify as a Tax Preparer?

  • Copy link
  • Flag this comment
  • Block
Pseudonymous :antiverified:
@VictimOfSimony@infosec.exchange replied  ·  activity timestamp 2 weeks ago

@cstross
@ViennaMike

If you like, put another way, you're taking the "good guy with a gun" side of the argument. ablobcatnod

I mean, feel free, but own it. Intelligent well-informed adults frequently take opposite sides in this. ablobcatnodfast

  • Copy link
  • Flag this comment
  • Block
ViennaMike
@ViennaMike@mastodon.social replied  ·  activity timestamp 2 weeks ago

@VictimOfSimony @cstross If you want to debate whether or not, as a whole, LLMs are a net positive or net negative, I agree. Intelligent people can legitimately disagree. One cannot, based on the evidence, legitimately claim that they only produce spam.

  • Copy link
  • Flag this comment
  • Block
Dr Power Nap, DDS ✅️
@ThePowerNap@mefi.social replied  ·  activity timestamp 2 weeks ago

@cstross @ViennaMike

There is some real utility under the hood, but goddamn.

The pushback from higher utility bills and GenAI diarrhea is justified, and a huge black eye for practical use.

  • Copy link
  • Flag this comment
  • Block
dragonfrog
@dragonfrog@mastodon.sdf.org replied  ·  activity timestamp 2 weeks ago

@ThePowerNap @cstross @ViennaMike
Sure there's some real helpful use cases for LLMs.

I reckon if the LLM industry manages to settle down into supplying only the helpful sensible use cases for LLMs, it will be about as big - in terms of annual dollar throughput - as eg the electric kettle manufacturing sector.

But to get to that sustainable level the LLM industry would have to shrink its market cap by 3 orders of magnitude. And it probably can't do that without collapsing entirely.

  • Copy link
  • Flag this comment
  • Block
Rupert V/
@rupert@mastodon.nz replied  ·  activity timestamp 2 weeks ago

@cstross @ViennaMike Rubber ducks are useful when writing code, too. Doesn't make them intelligent.

  • Copy link
  • Flag this comment
  • Block
Mighty Orbot
@mighty_orbot@retro.pizza replied  ·  activity timestamp 2 weeks ago

@cstross LLMs always remind me of the protagonists of “Accelerando” finally making contact with a whole system of alien AIs, only to learn that they’re all just jumped-up versions of the Nigerian prince scam.

  • Copy link
  • Flag this comment
  • Block
Marcus M. 滅拉 マルクス🐧
@mocm@mastodon.social replied  ·  activity timestamp 2 weeks ago

@cstross Labermaschinen

  • Copy link
  • Flag this comment
  • Block
Brian Tatosky
@virtualbri@mastodon.online replied  ·  activity timestamp 2 weeks ago

@cstross "spicy autocomplete" has been one of my favorite descriptions of it.

  • Copy link
  • Flag this comment
  • Block
Log in

Bonfire Dinteg Labs

This is a bonfire demo instance for testing purposes. This is not a production site. There are no backups for now. Data, including profiles may be wiped without notice. No service or other guarantees expressed or implied.

Bonfire Dinteg Labs: About · Code of conduct · Privacy ·
Bonfire social · 1.0.0 no JS en
Automatic federation enabled
  • Explore
  • About
  • Code of Conduct
Home
Login