The tradeoff between safety and usefulness has been a theoretical aspect of the development of AI for a very long time, since before we even had serious powerful systems or agents that could do any real work.
There was this paper, back in GPT-3.5 and GPT-4 days, where they were basically red teaming and blue teaming to see if they could create a system prompt to misalign GPT-4 enough to fool GPT-3.5, or to fool the humans, and be able to be deployed without being caught as being misaligned.
They didn't know whether it was going to be the regular GPT-4 or the misaligned version. They had this gamified kind of red team blue team situation, and came up with a bunch of protocols for basically how you could trade off between getting the full utility of a model and being able to use it safely.
Robert Miles has a very interesting video on that paper and discusses a lot of those ideas.
Now those ideas are becoming actually used in real systems, because people use Claude Code and OpenAI's Codex and depend on them every day.
You want to get as much utility out of the model as you can, but you don't want it hacking anybody or deleting your database when it makes a mistake.
They've got these auto modes that check the actions of the AI in context to approve them for you. There's a tradeoff between if it's going to over-block requests and push them for human review, which is going to be super annoying and sometimes will stop a job that you meant to run overnight, and allowing some dangerous actions through.
I haven't used auto mode in a long time. I've heard a lot of stories of it doing that when you put it overnight. But I personally have been using complete YOLO mode for a while, and it hasn't really bit me yet.
I personally have gone all the way to the utility end of the tradeoff, since I trust the models largely enough to not do terrible things like delete my database, and I'm not exposing them to a whole lot of prompt injection risk by sending them loose on the internet as much.
I don't let them just monitor my inbox in an open-ended sandbox that can do anything, because that would be kind of dangerous. Anybody could just email me with a prompt injection.
But I just have to be a little bit more careful about how I use it, and then I leave everything in full YOLO mode, no auto mode, because I can't have it getting blocked when I want it to do something.
Maybe if they had a slider for it, so like leniency, if I could set it all the way on auto mode to only ever block it if it's literally about to delete a database or destroy my root or something, and never for any other reason, then I would do that.
If I can't configure auto mode myself and decide what goes through and what doesn't, then I'm not going to ever use it ever again.
It's literally just a text box where I can tell it how I want it to behave: only cause human review for absolutely catastrophic actions, like deleting my root folder or deleting a database in an irrecoverable way.
Anything else, even if it looks strange, like going to my website, publishing something, even if it's deleting a folder that isn't my main home directory, should have full leeway except for those two cases or something that's obviously malicious.
I'd leave that up to the model's judgment.
I say these models are smart, you don't need auto mode. If you're a company, and you don't have extremely sensitive stuff that people will die on if they don't have reliability, then I would say let them use YOLO.
It's going to block them less often. But if you do have something where someone's going to die or you're going to lose a million dollar deal if anything goes wrong, then I suppose you could use auto mode.
Probably auto mode is for developers whose companies don't let them use full YOLO mode, and so they need to tell their manager that they can have peace of mind by using auto mode. If they kind of have to use it for their company to allow them to, then I guess that would be better than having it in always approve.