Interesting fact from this article: 12% of vulnerabilities in human-written code are high or critical severity. For LLM code slop, that number is 32% and has apparently been pretty constant for a while rather than improving
Infosec professionals sour on automated pentesting tools
Submitted 4 hours ago by supersquirrel@sopuli.xyz to technology@lemmy.zip
reluctant_squidd@lemmy.ca 2 hours ago
I’m not a pen tester, but I think if I was, letting an AI tell me what’s wrong and being satisfied with the answer and fixes it provides wouldn’t sit right with me.
I would want to know exactly where a problem is and how it cascades throughout an entire stack. Then how any fix also affects the same.
The article ends suggesting a hybrid solution, which I believe to be a sound assessment, but it would have to be done correctly. As in, under strict and meticulous protocol and tracking/documentation. Especially whenever the LLM is utilized.
It’s like an article I read a few days ago about companies having to hire back their sys admins a few months after replacing them with AI. The AI understands the code structure, but rarely the full context of its use. Only the expertise of experienced admins can fully understand and derive dependable results. At least for now…