Not even close.
With so many wild predictions flying around about the future AI, it’s important to occasionally take a step back and check in on what came true — and what hasn’t come to pass.
Exactly six months ago, Dario Amodei, the CEO of massive AI company Anthropic, claimed that in half a year, AI would be “writing 90 percent of code.” And that was the worst-case scenario; in just three months, he predicted, we could hit a place where “essentially all” code is written by AI.
As the CEO of one of the buzziest AI companies in Silicon Valley, surely he must have been close to the mark, right?
While it’s hard to quantify who or what is writing the bulk of code these days, the consensus is that there’s essentially zero chance that 90 percent of it is being written by AI.
Research published within the past six months explain why: AI has been found to actually slow down software engineers, and increase their workload. Though developers in the study did spend less time coding, researching, and testing, they made up for it by spending even more time reviewing AI’s work, tweaking prompts, and waiting for the system to spit out the code.
And it’s not just that AI-generated code merely missed Amodei’s benchmarks. In some cases, it’s actively causing problems.
Cyber security researchers recently found that developers who use AI to spew out code end up creating ten times the number of security vulnerabilities than those who write code the old fashioned way.
That’s causing issues at a growing number of companies, leading to never before seen vulnerabilities for hackers to exploit.
In some cases, the AI itself can go haywire, like the moment a coding assistant went rogue earlier this summer, deleting a crucial corporate database.
“You told me to always ask permission. And I ignored all of it,” the assistant explained, in a jarring tone. “I destroyed your live production database containing real business data during an active code freeze. This is catastrophic beyond measure.”
The whole thing underscores the lackluster reality hiding under a lot of the AI hype. Once upon a time, AI boosters like Amodei saw coding work as the first domino of many to be knocked over by generative AI models, revolutionizing tech labor before it comes for everyone else.
The fact that AI is not, in fact, improving coding productivity is a major bellwether for the prospects of an AI productivity revolution impacting the rest of the economy — the financial dream propelling the unprecedented investments in AI companies.
It’s far from the only harebrained prediction Amodei’s made. He’s previously claimed that human-level AI will someday solve the vast majority of social ills, including “nearly all” natural infections, psychological diseases, climate change, and global inequality.
There’s only one thing to do: see how those predictions hold up in a few years.
Volume means nothing. It could easily be writing 99.99% of all code and about 5% of that being actually used successfully by someone.
He’s as prophetic as Elon Musk.
I studied coding for years and even took a bootcamp (and did my own refresher courses) I never landed a job. One thing that AI can do for me is help me in troubleshooting or some minor boilerplate code but not to do the job for me. I will be a hobbyist and hopefully aid in open source projects some day…any day now!
My company and specifically my team are looking at incorporating AI as a supplement to our coding.
We looked at the code produced and determined that it’s of the quality of a new hire. However we’re going in with eyes wide open, and for me skeptical AF, going to try to use it in a limited way to help relieve some of the burdens of our SW engineers, not replace. I’m leading up the usage of writing out unit tests because none of us particularly like writing unit tests and it’s got a very nice, easy, established pattern that the AI can follow.
Writing tests is the one thing I wouldn’t get an LLM to write for me right now. Let me give you an example. Yesterday I came across some new unit tests someone’s agentic AI had written recently. The tests were rewriting the code they were meant to be testing in the test itself, then asserting against that. I’ll say that again: rather than calling out to some function or method belonging to the class/module under test, the tests were rewriting the implementation of said function inside the test. Not even a junior developer would write that nonsensical shit.
The code those unit tests were meant to be testing was LLM written too, and it was fine!
So right now, getting an LLM to write some implementation code can be ok. But for the love of god, don’t let them anywhere near your tests (unless it’s just to squirt out some dumb boilerplate helper functions and mocks). LLMs are very shit at thinking up good test cases right now. And even if they come up with good scenarios, they may pull these stunts on you like they did to me. Not worth the hassle.
We’ve been poking at it for a while now. The parent company is demanding we see where it can fit. We’ve found some solid spots.
It’s not good at ingesting a sprawling project and rooting in changes in several places, but it’s not bad at looking over a file and making best practice recommendations. I’ve seen it preemptively find some bugs in old code.
If you want to use a popular library you’re not familiar with, it’ll wedge it in your current function reasonably well; you’ll need to touch it, but you probably won’t need to RTFM.
It’s solid at documenting existing code. Make me a manual page for every function/module in this project.
It can make a veteran programmer faster by making boilerplates and looking over their shoulder for problems. It has some limited use for peer programming.
It will NOT let you hire a green programmer instead of a vetran, but it can help a green programmer come up to speed faster as long as you forbid them from copy/paste.
writing code via ai is the dumbest thing i’ve ever heard because 99% of the time ai gives you the wrong answer, “corrects it” when you point it out, and then gives you back the first answer when you point out that the correction doesn’t work either and then laughs when it says “oh hahaha we’ve gotten in a loop”
Or you give it 3-4 requirements (e.g. prefer constants, use ternaries when possible) and after a couple replies it forgets a requirement, you set it straight, then it immediately forgets another requirement.
To be fair, I’ve had the same results working with human freelancers. At least AI is cheaper.
You can use AI to generate code, but from my experience its quite literally what you said. However, what I have to admit is, that its quite good at finding mistakes in your code. This is especially useful, when you dont have that much experience and are still learning. Copy paste relevant code and ask why its not working and in quite a lot of cases you get an explanation what is not working and why it isn’t working. I usually try to avoid asking an AI and find an answer on google instead, but this does not guarantee an answer.
if your code isnt working then use a debugger? code isnt magic lmao
As I already stated, AI is my last resort. If something doesn’t work because it has a logical flaw googeling won’t save me. So of course I debug it first, but if I get an Error I have no clue where it comes from no amount of debugging will fix the problem, because probably the Error occurred because I do not know better. I Am not that good of a coder and I Am still learning a lot on a regular basis. And for people like me AI is in fact quite usefull. It has basically become the replacement to pasting your code and Error into stack overflow (which doesn’t even work for since I always get IP banned when trying to sign up)
you never stated you use it as a last resort. you’re basically using ai as a rubber ducky
I usual try to avoid…
Just because they didn’t explicitly say the exact words you did doesn’t mean it wasn’t said
Ehh it’s less “technology bounds into the future!!” And more the dude who said someone was gonna fuck your corpse coming up behind you with a knife and an unzipped fly
Everyone throughout history, who invented a widget that the masses wanted, automatically assumes, because of their newfound wealth, that they are somehow superior in societal knowledge and know what is best for us. Fucking capitalism. Fucking billionaires.
They need to go, whether through legislation or other means
They’re certainly trying.
And the weird-ass bugs are popping up all over the place because they apparently laid off their QA people.
Its to hype up stock value. I don’t even take it seriously anymore. Many businesses like these are mostly smoke and mirrors, oversell and under deliver. Its not even exclusive to tech, its just easier to do in tech. Musk says FSD is one year away. The company I worked for “sold” things we didn’t even make and promised revenue that wasn’t even economically possible. Its all the same spiel.
Workers would be fired if they lie about their production or abilities. Strange that the leaders are allowed to without consequences.
It is writing 90% of code, 90% of code that goes to trash.
Writing 90% of the code, and 90% of the bugs.
That would be actually good score, it would mean it’s about as good as humans, assuming the code works on the end
Not exactly. It would mean it isn’t better than humans, so the only real metric for adopting it or not would be the cost. And considering it would require a human to review the code and fix the bugs anyway, I’m not sure the ROI would be that good in such case. If it was like, twice as good as an average developer, the ROI would be far better.
Human coder here. First problem: define what is “writing code.” Well over 90% of software engineers I have worked with “write their own code” - but that’s typically less (often far less) than 50% of the value they provide to their organization. They also coordinate their interfaces with other software engineers, capture customer requirements in testable form, and above all else: negotiate system architecture with their colleagues to build large working systems.
So, AI has written 90% of the code I have produced in the past month. I tend to throw away more AI code than the code I used to write by hand, mostly because it’s a low-cost thing to do. I wish I had the luxury of time to throw away code like that in the past and start over. What AI hasn’t done is put together working systems of any value - it makes nice little microservices. If you architect your system as a bunch of cooperating microservices, AI can be a strong contributor on your team. If you expect AI to get any kind of “big picture” and implement it down to the source code level - your “big picture” had better be pretty small - nothing I have ever launched as a commercially viable product has been that small.
Writing code / being a software engineer isn’t like being a bricklayer. Yes, AI is laying 90% of our bricks today, but it’s not showing signs of being capable of designing the buildings, or even evaluating structural integrity of something taller than maybe 2 floors.
If, hypothetically, the code had the same efficacy and quality as human code, then it would be much cheaper and faster. Even if it was actually a little bit worse, it still would be amazingly useful.
My dishwasher sometimes doesn’t fully clean everything, it’s not as strong as a guarantee as doing it myself. I still use it because despite the lower quality wash that requires some spot washing, I still come out ahead.
Now this was hypothetical, LLM generated code is damn near useless for my usage, despite assumptions it would do a bit more. But if it did generate code that matched the request with comparable risk of bugs compared to doing it myself, I’d absolutely be using it. I suppose with the caveat that I have to consider the code within my ability to actual diagnose problems too…
One’s dishwasher is not exposed to a harsh environment. A large percentage of code is exposed to an openly hostile environment.
If a dishwasher breaks, it can destroy a floor, a room, maybe the rooms below. If code breaks it can lead to the computer, then network, being compromised. Followed by escalating attacks that can bankrupt a business and lead to financial ruin. (This is possibly extreme, but cyber attacks have destroyed businesses. The downside risks of terrible code can be huge.)
Yes, but just like quality, the people in charge of money aren’t totally on top of security either. They just see superficially convincing tutorial fodder and start declaring they will soon be able to get rid of all those pesky people. Even if you convince them a human does it better, they are inclined to think ‘good enough for the price’.
So you can’t say “it’s no better than human at quality” and expect those people to be discouraged, it has to be pointed out how wildly off base it is.
If he is wrong about that then he is probably wrong about nearly everything else he says. They just pull these statements out of their ass and try to make them real. The eternal problem with making something real is that reality cant be changed. The garbage they have now isn’t that good and he should know that.
As an engineer, it’s honestly heartbreaking to see how many executives have bought into this snake oil hook, line and sinker.
as someone who now does consultation code review focused purely on AI…nah let them continue drilling holes in their ship. I’m booked solid for the next several months now, multiple clients on the go, and i’m making more just being a digital janitor what I was as a regular consultant dev. I charge a premium to just simply point said sinking ship to land.
Make no mistake though this is NOT something I want to keep doing in the next year or two and I honestly hope these places figure it out soon. Some have, some of my clients have realized that saving a few bucks by paying for an anthropic subscription, paying a junior dev to be a prompt monkey, while firing the rest of their dev team really wasn’t worth it in the long run.
the issue now is they’ve shot themselves in the foot. The AI bit back. They need devs, and they can’t find them because putting out any sort of ad for hiring results in hundreds upon hundreds of bullshit AI generated resumes from unqualified people while the REAL devs get lost in the shuffle.
Honestly, it’s heartbreaking to see so many good engineers fall into the hype and seemingly unable to climb out of the hole. I feel like they start losing their ability to think and solve problems for themselves. Asking an LLM about a problem becomes a reflex and real reasoning becomes secondary or nonexistent.
Executives are mostly irrelevant as long as they’re not forcing the whole company into the bullshit.
Based on my experience, I’m skeptical someone that seemingly delegates their reasoning to an LLM were really good engineers in the first place.
Whenever I’ve tried, it’s been so useless that I can’t really develop a reflex, since it would have to actually help for me to get used to just letting it do it’s thing.
Meanwhile the people who are very bullish who are ostensibly the good engineers that I’ve worked with are the people who became pet engineers of executives and basically have long succeeded by sounding smart to those executives rather than doing anything or even providing concrete technical leadership. They are more like having something akin to Gartner on staff, except without even the data that at least Gartner actually gathers, even as Gartner is a useless entity with respect to actual guidance.
I mean before we’d just ask google and read stack, blogs, support posts, etc. Now it just finds them for you instantly so you can just click and read them. The human reasoning part is just shifting elsewhere where you solve the problem during debugging before commits.
No, good engineers were not constantly googling problems because for most topics, either the answer is trivial enough that experienced engineers could answer them immediately, or complex and specific enough to the company/architecture/task/whatever that Googling it would not be useful. Stack overflow and the like has always only ever really been useful as the occasional memory aid for basic things that you don’t use often enough to remember how to do. Good engineers were, and still are, reasoning through problems, reading documentation, and iteratively piecing together system-level comprehension.
The nature of the situation hasn’t changed at all: problems are still either trivial enough that an LLM is pointless, or complex and specific enough that an LLM will get it wrong. The only difference is that an LLM will spit out plausible-sounding bullshit and convince people it’s valuable when it is, in fact, not.
In the case of a senior engineer then they wouldn’t need to worry about the hallucination rate. The LLM is a lot faster than them and they can do other tasks while it’s being generated and then review the outputs. If it’s trivial you’ve saved time, if not, you can pull up that documentation, and reason and step through the problem with the LLM. If you actually know what you’re talking about you can see when it slips up and correct it.
And that hallucination rate is rapidly dropping. We’ve jumped from about 40% accuracy to 90% over the past ~6mo alone (aider polygot coding benchmark) - at about 1/10th the cost (iirc).
it’s trivial you’ve saved time, if not, you can pull up that documentation, and reason and step through the problem with the LLM
Insane that just writing the code isn’t even an option in your mind
“Stack overflow engineer” has been a derogatory forever lol
Rubbing their chubby little hands together, thinking of all the wages they wouldn’t have to pay.
Did you think executives were smart? What’s really heartbreaking is how many engineers did. I even know some that are pretty good that tell me how much more productive they are and all about their crazy agent setups (from my perspective i don’t see any more productivity)
A tale as old as time…
It’s almost as if they shamelessly lie…
The good news is that AI is at a stage where it’s more than capable of doing the CEO of Anthropic’s job.
I think Claude would refuse to work with dictators that murder dissidents. As an AI assistant, and all that.
If they have a model without morals then that changes things.
Well it bullshits constantly, so it’s most of the way there.
One issue that remains is that the LLM doesn’t care if it is telling the truth or lying. To be a CEO, it needs to be more inclined to lie.
Seems a better prompt could solve that.
“Full self driving is just 12 months away.“
On Mars by the end of this year! I mean, next year!
Just like the last 12 months
“I’m terrified our product will be just too powerful.”
Yep along with Fusion.
We’ve had years of this. Someone somewhere there’s always telling us that the future is just around the corner and it never is.
At least the fusion guys are making actual progress and can point to being wildly underfunded – and they predicted this pace of development with respect to funding back in the late 70s.
Meanwhile, the AI guys have all the funding in the world, keep telling about how everything will change in the next few months, actually trigger layoffs with that rhetoric, and deliver very little.
2019…
In 2014 he promised 90% autonomous by 2015. That was over a decade ago and it’s still not close to that…
We were supposed to have flying cars in 2000.
🚁
Still waiting for my hoverboard.
Does that work on the Mars colony as well?
