A nice bonus is that sysadmin tasks tend to be light in terms of token usage, that’s very convenient given the increasingly strict usage limits these days.
No need for any AI-specific tool, this is exactly what devcontainer is for! Just tell your agent to use devcontainer up (and docker compose down the other way).
Devcontainers always disappointed me. The sales pitch is that everyone uses the same container, but that's not accurate. Everyone builds a container from the same config and it'll be similar, but it takes a ton of effort to make sure it's identical.
The idea that a devcontainer gets built on-demand instead of checked out like 'docker pull ..." has always felt weird to me. It's so close to being awesome, but ends up being barely useful.
Or maybe I'm wrong. Is there a way to checkout an immutable devcontainer?
The thing is, with manual coding, you spot a view in the distance, you trek your way for a few hours, and you realize when you get there that the view isn’t as great as you thought it was.
With LLM-assisted coding, you skip the trek and you instantly know that’s not it.
I have Hermes agent run a cron each hour to check if the Steam Controller if finally on sale. I don’t if that resonates with you, I quite like that use case personally :)
Must be some ironic comment that I didn't get, but a simple script (curl calling steam API and sending a notification if the price is set) + cron handle the job perfectly.
EDIT: Giving the keys to an agent for such a trivial work is ... I got your sarcasm I think ^^.
You would have had to run `jj edit` in order for this to happen, so I think it's a stretch to say you didn't ask for the edit?
This is the main difference though: in git files can be `staged`, `unstaged` or `committed`, so at any one time there are 3 entire snapshots of the repo "active".
In `jj` there is only one kind of snapshot (a change) and only one is "active" (the current working directory). When you make changes to the working directory you are modifying that "change".
As others have mentioned, the equivalent to `git checkout` would be `jj new`, which ensures a new empty change exists above the one you are checking out, so that any changes you make go into that new change rather than affecting the existing one.
Using `jj edit` will edit a commit you specify, and `jj new` will make a new empty commit after the one you specify. These work exactly the same whether you specify a commit by branch or by the hash. I'd argue that you're getting exactly what you ask for with these commands, and by comparison, what "checkout" is asking for is much less obvious (and depends on context). We've just internalized the bad behavior of git for so long that it's become normalized.
But even then, the agent can still exfiltrate anything from the sandbox, using curl. Sandboxing is not enough when you deal with agents that can run arbitrary commands.
If you're worried about a hostile agent, then indeed sandboxing is not enough. In the worst case, an actively malicious agent could even try to escape the sandbox with whatever limited subset of commands it's given.
If you're worried about prompt injection, then restricting access to unfiltered content is enough. That would definitely involve not processing third-party input and removing internet search tools, but the restriction probably doesn't have to be mechanically complete if the agent has also been instructed to use local resources only. Even package installation (uv, npm, etc) would be fine up to the existing risk of supply-chain attacks.
If you're worried about stochastic incompetence (e.g. the agent nukes the production database to fix a misspelled table name), then a sandbox to limit the 'blast radius' of any damage is plenty.
That argument seems to assume a security model where the default prior is « no hostile agent ». But that’s the problem, any agent can be made hostile with a successful prompt injection attack. Basically, assuming there’s no hostile agent is the same as assuming there’s no attacker. I think we can agree a security model that assumes no attacker is insufficient.
Code is not the only thing the agent could exfiltrate, what about API keys for instance? I agree sandboxing for security in depth is good, but it’s not sufficient and can lull you into a false sense of security.
This is what emulators and separate accounts are for. Ideally you can use an emulator and never let the container know about an API key. At worst you can use a dedicated account/key for dev that is isolated from your prod account.
VM + dedicated key with quotas should get you 95% there if you want to experiment around. Waiting is also an option, so much of the workflow changes with months passing so you’re not missing much.
That depends on how you configure or implement your sandbox. If you let it have internet access as part of the sandbox, then yes, but that is your own choice.
Internet access is required to install third party packages, so given the choice almost no one would disable it for a coding agent sandbox.
In practice, it seems to me that the sandbox is only good enough to limit file system access to a certain project, everything else (code or secret exfiltration, installing vulnerable packages, adding prompt injection attacks for others to run) is game if you’re in YOLO mode like pi here.
reply