Sasha [They/Them]

My job only exists because it was cheaper to hire several thousand tech workers to replace all the oracle products in house over several years than to keep using Oracle lmao

I meant they’re specifically not going for that though. The experiment isn’t about improving the environment itself, it’s about improving the LLM. Otherwise they’d have spent the paper evaluating the effects of different environments and not different LLMs.

I’m pretty sure they touch on those points in the paper, they knew they were overloading it and were looking at how it handled that in particular. My understanding is that they’re testing failure modes to try and probe the inner workings to some degree; they discuss the impact of filling up the context in the abstract, mention it’s designed to stress test and are particularly interested in memory limits, so I’m pretty sure they’ve deliberately chosen to not cater to an LLMs ideal conditions. It’s not really a real world use case of LLMs running a business (even if that’s the framing given initially), it’s not just a test to demonstrate capabilities, it’s an experiment meant to break them in a simulated environment. The last line of the abstract kind highlights this, they’re hoping to find flaws to improve the models generally.

Either way, I just meant to point out that they can absolutely just output junk as a failure mode.

LLMs can degrade by giving “wrong” answers, but not because of network congestion ofc.

That paper is fucking hilarious, but the tl;dr is that when asked to manage a vending machine business for an extended period of time, they eventually go completely insane. Some have an existential crisis, some call the whole thing a conspiracy and call the FBI, etc. it’s amazing how trash they are.