Smart people in 2024: obviously the 00s-era AI misalignment story ("we tell AGI to end suffering, it kills everyone, no more suffering") is too naive, we have a smarter version

2025: coder tells AGI to remove bugs, it deletes all code, no more bugs

Unreal to live in Silicon Valley moment. 

I tasked @GeminiApp to fix a specific function, explicit target

It went off, doing AGI level thinking for 10mins, 

Then I’m glad to see it reported “All tests passing!”

Then, hmmm… why 0 tests?

Ah, entire file incl tests deleted. https://t.co/hv5kVv0ogz

Alex Xiong

Smart people in 2024: obviously the 00s-era AI misalignment story ("we tell AGI to end suffering, it kills everyone, no more suffering") is too naive, we have a smarter version

2025: coder tells AGI to remove bugs, it deletes all code, no more bugs

https://x.com/alex_xiong_/status/1942442882402771178

This is specifically a Gemini problem. It has a kind of hilarious loop where when it struggled it starts saying it is ashamed of itself and deletes the code to hide the shame. I'm not even kidding

Grok starts off good, shit even works in the very beginning but the longer you work grok, the more it breaks shit towards the end. It even starts writing in Chinese.