There are a lot of ways things can go wrong. So you need a lot of levels of safety.
If your production site goes down, it’s always a system failure. Even if the entire system is a bunch of LLMs running on one person’s machine. There’s no reason why anything on your local machine should be able to make changes in production. We’ve had CI/CD pipelines with authentication and secrets management for long enough that it’s a solved problem.
I have an electric lawnmower. It plugs in via a circuit breaker, into a grounded 3-pin plug, attached to a mains unit with another circuit breaker. To start it, after plugging it in (and testing its circuit breaker), I need to push a button with one hand and pull a lever with the other. The motor will cut out, even in that configuration, if the resistance is higher than a threshold. The blades are not visible in use, because the cover completely surrounds them.
None of this prevents me from chopping my toes off with the lawnmower, but it makes it a lot harder.
So a good security system will have multiple fail-safe protections between the developer’s machine and the production environment, and ideally a mix of human and machine checks. Anti-virus, firewalls and network monitoring on the developer machine itself, as well as policies to keep everything patched. Review and limitations on the supply chain via container and package registries. Refusal to run code in a CI build if conditions aren’t met, and refusal to sign and publish until build, test and human review all pass. Refusal to deploy unless there’s a signed build. Refusal to accept any credentials except those in the secure store. Refusal to accept developer credentials to change the database.
Security isn’t easy. You have to win every hour of every day. The attacker only needs to win once. But every failsafe is another barrier they need to pass, and they only win if they can pass them all.
And for your own sanity, keep development and production isolated.
Top comments (0)