dammIT

A rantbox by Michiel Scholten

LLMs are a Cognitohazard


It is amazing how many people I see claiming that AI has one good use. No one seems to agree on what that one good use is, and there's always a hell of a lot of goalpost shifting and special pleading involved. For some reason a lot of folks quite reasonably follow the arguments against AI, but then partition off the one thing as immune or exempt from having to worry about any of the ethical and practical problems. - @xgranade@wandering.shop

I've been hesitant to use LLM's (or what people call 'AI'), but have not put into words exactly the (computer-science-underbelly) feelings I have about them. I do not dislike them - they are fascinating technology, but they are far from the marvel that people think they are.

One of the first things my mind always goes to is this:

I think each and every one of these things ignores the externalities of running an LLM: The wholesale scraping of data frequently without consent (and the associated pushing of costs onto the site being scraped), the massive build-out of data centers that will never be used, a global chip shortage because the AI industry can just do that, the proliferation of slop and disinformation, and the trend of people experiencing psychosis when interacting with chatbots.

Alex continues to point out other examples in a way that I hadn't put into words that well yet, ranging from how prototypes always tend to end up in production anyway, that LLM's are terrible search engines, not as helpful in code reviewing or writing code as you would like, and not even good at summarising (something that surprised me a bit, as that was the one thing that I thought they actually did well, but he makes some good points there too).

Good read, and it might provide some well-phrased counterpoints to that colleague or boss that is raving about the AI pie in the sky.

article header image
article header image