Sahwa@reddthat.com to Technology@lemmy.worldEnglish · 4 months agoClaude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes roguewww.tomshardware.comexternal-linkmessage-square118linkfedilinkarrow-up1817arrow-down114
arrow-up1803arrow-down1external-linkClaude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes roguewww.tomshardware.comSahwa@reddthat.com to Technology@lemmy.worldEnglish · 4 months agomessage-square118linkfedilink
minus-squarePommes_für_dein_Balg@feddit.orglinkfedilinkEnglisharrow-up64arrow-down1·4 months agoIt’s so weird how these chatbots always pretend they learnt something after they fuck up. They literally can’t.
minus-squarefrongt@lemmy.ziplinkfedilinkEnglisharrow-up19arrow-down2·4 months agoThey’re not even pretending. The algorithm says the most likely response to “you fucked up” is “I’m sorry”, so that’s what it prints. There’s zero psychological simulation going on, only statistical text generation.
minus-squarethisbenzingring@lemmy.todaylinkfedilinkEnglisharrow-up2·4 months agothe next ingestion cycle will probably pick it up but how do we know it’ll use the information in any relevant way 😶
It’s so weird how these chatbots always pretend they learnt something after they fuck up.
They literally can’t.
They’re not even pretending. The algorithm says the most likely response to “you fucked up” is “I’m sorry”, so that’s what it prints. There’s zero psychological simulation going on, only statistical text generation.
the next ingestion cycle will probably pick it up but how do we know it’ll use the information in any relevant way 😶