The "critic psychosis" thing is tedious and wrong for the same reasons Cory's previous "purity culture" take was tedious and wrong, a transparent and honestly somewhat pathetic attempt at self-justification for his own AI tool use for writing assistance. It pairs very well with this Scientific American article, which points out that pedestrian "writing AI tools" influence their users in subtle but clearly disturbing ways. https://www.scientificamerican.com/article/ai-autocomplete-doesnt-just-change-how-you-write-it-changes-how-you-think/
Discussion
Loading...
Post
@glyph You're so extremely full of yourself that I didn't even finish reading your comment, and I no longer care about anything you have to say. Go touch grass.
Cory also correctly points out that "AI psychosis" is probably going to be gatekept by medical establishment scicomm types soon because "psychosis" probably isn't the right word and already carries an unwarranted stigma. And indeed, I think the biggest problem with "psychosis" as a metaphor is going to be that the ways in which AI can warp our minds are mostly NOT going to be catastrophic psychosis, and are not going to have great existing analogs in existing medical literature.
@glyph I tend to reserve it for when the person I'm interacting with seems to be having a legitimate psychotic break with reality.
@glyph if this was a tablet/pill taken that promised to help you write better emails, and it accidentally caused psychological disorders, it would be put through vigorous testing before being loaded onto a dishwasher or pushed into every system available. This is the Sacklers of tech, grifting every last cent out of people who are already struggling, promising all the while that it’s non addictive.
@glyph LLMs seem to use many of the same techniques as mentallists, psychics, fortune-tellers and mediums in how they manipulate their victims, like suggestions, cold reading, flattery, confidence and in the victims confirmation bias and suggestibility. People are influenced, by the politeness and the well structured text, into ignoring factual issues, and then by having a conversation they fix the glaring problems themselves, and later attribute it to the model.
@glyph Honestly - speaking as someone with a psychotic disorder, but who is not a medical professional - "AI psychosis" seems pretty appropriate, from the behaviours I've seen it result in? Even in more mild cases of people babbling inane bullshit, but not like, so far off reality that they're at risk of physical harm (to themself or others)
@glyph I'm sure the mechanism - how they got there - has more in common with emotional abuse and brainwashing/indoctrination techniques.
But the end result - that detachment from reality - is kind of the core of the psychosis experience, and trying to find ways to keep tethered and avoid drifting off into wonderland like that is a persistent part of my day-to-day life.
Which is part of *why* I avoid the chatbots like they're carrying the plague.
@glyph Similarly, “hallucination” and “delusion” are pre-poisoned for use in this scope
I have on occasion made use of “phantasmagoria” around parts of this dynamic, especially for stuff like the droll “omg the AI is learning to lie to us, we’re cooked!” type bullshit posts, but that’s still not expansive enough to include the various other mental affectations
we need other perorations, and better perseverations alongside
If I could use another inaccurate metaphor, AI psychosis is the "instant decapitation" industrial accident with this new technology. And indeed, most people having industrial accidents are not instantly decapitated. But they might get a scrape, or lose a finger, or an eye. And an infected scrape can still kill you, but it won't look like the decapitation. It looks like you didn't take very good care of yourself. Didn't wash the cut. Didn't notice it fast enough. Skill issue.
@glyph
Here's an industrial accident that's easy to miss:
A hydraulic fluid line bursts while you're working on a machine, injecting toxic and/or hot liquid under your skin at high pressure.
https://en.wikipedia.org/wiki/High_pressure_injection_injury
"Although the initial wound often seems minor, the unseen, internal damage can be severe. With hydraulic fluids, paint, and detergents, these injuries are extremely serious as most hydraulic fluids and organic solvents are highly toxic."
@dec23k okay definitely not clicking on that link, yeesh
@glyph All the possible harm is just mental, and the mental - this seems to be an unspoken tenet held by many - isn't really real. Mental health in general doesn't really seem to be taken that seriously before it manifests physically as chainsaw wielding or some other eccentricity. Nothing to see here, just move on.
@glyph I've been using "AI delusion" for these milder cases. As I understood AI psychosis it pertains only to those cases where people fully lose grasp of reality...
I've seen it used colloquially as "being wrong because of or about AI", but that always hit me like people calling someone "crazy" for doing something odd or impulsive—and that word use isn't really a good look imo.
More to the point though in this metaphor where you're getting a potentially-infected scrape at work, we are living in the pre-germ-theory age of AI. We are aware that it might be dangerous sometimes, but we don't know to whom or why. We are attempting to combat miasma with bloodletting right now, and putting the miasma-generator in every home before we know what it's actually doing.
@glyph potentially an even better metaphor is RSI, though that does lead to the "you're holding it wrong" argument which isn't applicable, but incidental injuries are in the same bucket but it's just less obvious.
For me, this is the body horror money quote from that Scientific American article:
"participants who saw the AI autocomplete prompts reported attitudes that were more in line with the AI’s position—including people who didn’t use the AI’s suggested text at all"
So maybe you can't use it "responsibly", or "safely". You can't even ignore it and choose not to use it once you've seen it.
If you can see it, the basilisk has already won.
@glyph this is why "AI" boosterism or "both sides" debate gets an instant block
my little chunk of reality is free of this slop
@glyph i like to let them sort it out - ask the same question to like 3 models, sort of crude arbitrage
@glyph This basilisk thing (great imagery) is very true in translation. Once you've seen the MT suggestion, with its wonky syntax and not quite right tone, it's very hard to dismiss it. The cognitive load is consequently enormous
@janeishly @glyph I have found this exact thing in code reviews - my company enabled automatic AI code reviews ( 🤢 ) and the cognitive load of the automated comments was *enormous*.
It often correctly flagged something to pay attention to, but the suggested solution was always incorrect - and ignoring / discarding it was hugely expensive mentally.
I finally managed to get it changed to "opt in" rather than automatic, but wow the whole experience felt like a tarpit for thinking.
1 more replies (not shown)
@glyph when teams autocorrect rewrites something it decides i misspelled, i am filled with hatred and disgust and usually delete sentence and try again regardless of if it had suggested the word i meant to write. i don't want it anymore
@glyph i can absolutely use it responsibly because i'm not new to NLP, but unfortunately it is liquified shite.
@glyph Glyph's Basilisk > Roko's
@glyph "Interestingly, the people in the study didn’t tend to think the AI autocomplete suggestions were biased or to notice that they had changed their own thinking on an issue in the course of the study. Warning the participants that they might be exposed to misinformation by the AI didn’t temper the persuasive effect either."
Also, being aware that it can fuck with your head does not make you less susceptible to it fucking with your head. So you can't really judge if you could use it safely.
@glyph oh man, that’s dangerously close to giving an abstinence-based push some fuel and whew does my rather hedonistic ass have some thoughts on that
Now, for rhetorical effect, I'm obviously putting this fairly dramatically. Cory points out that people have been doing this *to each other* mediated by technology, in emergent and scary ways, with no need for AI. He shows that people prone to specific types of delusions (Morgellons, Gang Stalking Disorder) have found each other via the Internet and the simple availability of global distributed communication has harmed them. But obviously that has benefits, too.
@glyph Comparing how people influence each other and how LLM usage influences people is a point I find interesting.
A bunch of people get influenced in a bunch of different directions by a bunch of different people. Everybody gets influenced in mostly the same direction by the tool in the hands of ghoulish billionaires.
Sure, influencing is something we do to each other all the time. But is it really the same?
1 more replies (not shown)
@glyph Wait. Does Doctorow try to soften the edge of the fact that LLMs do a bad thing to people by pointing out sometimes people do bad things to people? Isn't that just two bad things?
If his point is it's still bad but it isn't *novel*, isn't the fact of a Fortune 500 company doing it still novel?
1+ more replies (not shown)
I'm open to a future where we do some research and figure out the limits of how AI influence works, and where the safety valves are, and also the extent to which it's *fine* that AI can influence our views because honestly many different kinds of stimuli can influence our views, not least of which is each other. But it sure looks right now like it has a bunch of very dangerous feedback loops built-in, and it's not clear how to know if you're touching one.
2+ more replies (not shown)
Bonfire Dinteg Labs
This is a bonfire demo instance for testing purposes. This is not a production site. There are no backups for now. Data, including profiles may be wiped without notice. No service or other guarantees expressed or implied.
Automatic federation enabled