The average person who has programmed something, or the average professional career programmer? I doubt the AI will be great a dealing with the really weird bugs and conflicts any time soon.
How long do you think until AI writes and debugs code better than the average programmer?
Submitted 11 months ago by PumpkinDrama@reddthat.com to nostupidquestions@lemmy.world
Comments
JohnDClay@sh.itjust.works 11 months ago
Darorad@lemmy.world 11 months ago
Yeah, and understanding the context of a massive codebase will give it a ton of challenges
Send_me_nude_girls@feddit.de 11 months ago
Can you even give it that context, looking at security reasons? I’m not sure.
Fiivemacs@lemmy.ca 11 months ago
It’ll probably just rewrite the spaghetti the human did and not even bother fixing the bug.
Deestan@lemmy.world 11 months ago
Valid answers are “never”, “soon” or “already have - decades ago”, depending on where you draw the boundaries on definitions of AI and programmer.
Because AI means something completely different every decade, and the modern programmer works at a different level every decade.
Soon: While a decent programmer can code at a level beyond the capabilities of LLM based generators (which I assume is AI in this context), some companies employ literal hordes of programmers that fail the simplest programming tasks, like the fizzbuzz test. Their output is to slowly cut and paste haphazard bits of code from StackOverflow, internet forums, and code found lying around the company, and make something that after several bounces off Quality Assurance and adjustments by senior developer review, pass. It’s not a stretch to see that it’s mostly a matter of turning current AI tech into streamlined products to take over these parts. Are these the average programmers? There are many of them, so it could be!
Already have - decades ago: An average programmer in the early 1950s spent a lot of time taking specific tasks like “this module needs this specific pattern of input and must produce this specific pattern of output” and painstakingly turning them into machine code. They would pen and paper out logic like
LDA 10; JSR FFD2; RTC;, turn it intoA9 10 20 D2 FF 60by referencing the manual and storing this byte sequence in a punch card or directly into machine memory addressC000. Programs were much larger than this of course, and the skill lied in doing it correctly and making optimizations to the program would run properly. Assemblers took over parts of this work, compilers took over more of it, and with optimized compilers and high level languages took the work away completely. This software fits the 1960 era definition of AI. Now all people had to do was write a prompt to the Ai likevoid main() { while(true) println(“yes”); }and it would do the programmer job for you.Never: Programmers will, as they always do, use this tool for everything it’s worth and work on a higher level to make bigger things faster.
asdfasdfasdf@lemmy.world 11 months ago
My take is that it’s somewhat of a gimmick and will continue to be for a very long time.
It can write functions which do the thing you want a lot of the time, yes. But for the entire codebase it’s very important to write it in a maintainable way. E.g. It’s super important to name variables things which the developers understand, and that requires a very solid understanding of the specific domain or business the dev is working in, as well as understanding the way the dev’s mind works too. AIs don’t know anything about that and aren’t psychic.
There are numerous other psychological aspects of coding like this which differentiate a good developer from bad ones. Even when AIs write entire programs, the human dev is going to be the one maintaining it and getting pages at 2am when something goes wrong. I don’t think anyone will trust that much in an AI for a very, very long time.
Valmond@lemmy.mindoki.com 11 months ago
C000
Wouah rich-pants with 64KB of memory!
meekah@lemmy.world 11 months ago
I pick never. AI just means increased productivity. Not less jobs.
Deestan@lemmy.world 11 months ago
I agree. Software remains a growing field.
Bishma@discuss.tchncs.de 11 months ago
Individual scripts/modules and even simple microservices: not long… provided the AI isn’t actively poisoning itself right now.
Writing, securing, and maintaining complex applications: We’d need another breakthrough.
Since my role is often solutions architecture I’ve been worried about cloud systems engineering being something that’s immediately vulnerable. But after working with AWS’s Q for a couple hours, I am less worried. But if someone made an AI to create a cloud provider that is well (and accurately) documented, consistent in functionality and UX, and which actually has all the features that get announced in its own blog posts; then AI might be able to run it.
slazer2au@lemmy.world 11 months ago
In order for an AI to know what code to scrape from stack overflow a user must be able to articulate what they want the program to do, now we all know they can’t so I doubt AI can for quite some time.
Blue_Morpho@lemmy.world 11 months ago
In order for an AI to know what code to scrape from stack overflow
It’s a common assumption that gpt is only cutting and pasting what it found on Google. But it’s not true. I spent hours trying to find help with vba for Excel ( because I know neither) with no results other than function definitions. Gpt gave me working code that wasn’t anywhere on the Internet. It had to have pieced together the code based on the well documented function definitions.
I see it like Dalle and those other ai art programs. You can see the style they are copying to create the picture. But the ai generated pictures are not cut and pasted from images already on the Internet.
slazer2au@lemmy.world 11 months ago
You do have to be careful though. Sometimes it gives functions that don’t exist.
PowerShell has a well established naming scheme of get-[function] or set-[function] so when you ask GPT to create a powershell code to set the name of a file it will use set-filename but that doesn’t exist.
I do believe that you can use LLM to assist in program creation but doubt an end user can articulate in full what they want a program to do.
gravitas_deficiency@sh.itjust.works 11 months ago
If my company were to fire me and try to replace me with a LLM, I’d simply wait a month or two and then offer to do my old job at a contract rate of at least 5x of what my current salary works out to. And I’d get it.
superfes@lemmy.world 11 months ago
From some of the code I’ve had to review, we may already be there…
SpaceNoodle@lemmy.world 11 months ago
All that means is that somebody else has written better code than your colleagues.
sxan@midwest.social 11 months ago
This can’t be understated. Like many professions, software development is full of mediocre professionals.
JackLSauce@lemmy.world 11 months ago
Remember when Elon made all driving jobs obsolete with AI in 2012, 2014, 2015 and 2017?
Pepperidge Farms remembers
ezchili@iusearchlinux.fyi 11 months ago
I’ve had 100% failure rate on simple requirements
“Make a pathfinding function for a 2d grid” - fine
“Make a pathfinding function for a 2d grid, but we can only move 15 cells at a time” - fails on lesser models, it keeps clinging to pulling you the same A* as the first one
“Make a pathfinding function for a 2d grid, but we can only move 15 cells at a time, also, some cells are on fire and must be avoided if possible, but if there is no other path possible then you’re allowed to use fire cells as fallback” - Never works
There for that last one, none of the models give a solution that fits the very simple requirement
themusicman@lemmy.world 11 months ago
Huh that’s a neat problem. My instinct was to use a (fire, regular) tuple for cost, but then what A* heuristic can you use…
I guess run it once with no cost for regular tiles and remove fire from any tiles it used. Then run with normal tile costs, but block fire tiles. That doesn’t break ties nicely of course and I’m not convinced the first pass has a good A* heuristic either…
ezchili@iusearchlinux.fyi 11 months ago
It works
I think the best option is to make sure to have ‘sorted’ the calls to the fire tiles, you can do that by having them in a separate grid or just stash them to a small local array on stack when you encounter them, and investigate those at the end of the loop
If there’s no result that’s been found under the cost limit without the fire at each point of the algorithm, you do do the recursive calls for the fire
Then once you’re back to the top level call, if there was a non-fire path you will get that result, if there wasn’t you will get that instead
dQw4w9WgXcQ@lemm.ee 11 months ago
I’ve had a lot more success in debugging than in writing code. I had a problem with adjusting the sample rate of a certain metrics framework in a java application, and stackoverflow failed me, both when searching for an aswer and when asking the question. However, when I in some desperation asked GPT 3.5, I received a great answer which pinpointed the necessary adjustment.
However, asking it to write simple code snippets, i.e. for migrating to a different elasticsearch client framework, has not been great. I’m often met with the confident wrong answers.
PumpkinDrama@reddthat.com 11 months ago
HumanEval achieved 74.4%, surpassing GPT-4 at 67%. It successfully solves 43% of problems in the latest Codeforces rounds with 10 attempts. The evaluation considered the time penalty, and it still ranks in the 85th percentile or higher. AlphaCode 2 already beats 85% of people in top programming competitions (which are already better than 99% of enginners out there). So I believe AI already writes better code than the average programmer, but I don’t think it can debug a large codebase yet. I’d say it will need a platform to test and iteratively rewrite the code, and I don’t see that happening earlier than 3 years.
PM_Your_Nudes_Please@lemmy.world 11 months ago
I actually have used it to debug code before. Not an entire program (yet) but it’s great for snippets where you’re just missing a semicolon or bracket, or need advice on how to properly call a weird function. It also writes small things like batch files incredibly well. Just like with regular language, it’s great for a few paragraphs, then begins to drift as it struggles to parse longer conversations. So if you only need it for a few “paragraphs” of code, it’s great.
SnuggleSnail@ani.social 11 months ago
I also like to give AI my code and just ask to rewrite it, implementing a cleaner solution and upholding best practices. Most times, there are things that are really an improvement!
lung@lemmy.world 11 months ago
Yeah not soon, 3-10 years I’d guess. The latest research tracking AI growth says that our best models can solve entry level CS problems at 85% success. That’s not good
It’s obvious that humans do more than just pattern matching. I think I would rate the current systems as a 25% speedup to my workflow, not bad, but only for menial tasks
YoBuckStopsHere@lemmy.world 11 months ago
Probably already does.
gravitas_deficiency@sh.itjust.works 11 months ago
This doesn’t say what you think it does. It simply indicates that you have no idea what you’re talking about.
stinerman@midwest.social 11 months ago
The code has always been the easy part. The hard part is getting requirements that make sense. It will continue to be the case with bots doing the coding.
2xsaiko@discuss.tchncs.de 11 months ago
Yes. This is the new “visual programming will make executives be able to write programs themselves” but this time the technology (assuming OP means LLMs since both the ones in his image post seem to be LLMs) is completely unsuitable from the start.
poo@lemmy.world 11 months ago
you’re making me shiver remembering all the “NO/LOW CODE SOLUTION!” banners I saw at the last tech conference I went to…
Tar_alcaran@sh.itjust.works 11 months ago
The bane of humanity is the inability of anyone to actually tell you what they want.
Bonehead@kbin.social 11 months ago
Funny...of all the jobs that will be taken over by AI, this guy is actually safe...