Costs under control

Costs: what can be capped

The real risk of an autonomously working system is not the single expensive run, but continuous operation that goes unnoticed. That is why a cap measures the month rather than the single run, and the check happens before the next one starts.

Three levels

One cap for the organisation, one per project, one per run. Of the three levels the tightest one stops the run. A cap applies as soon as you set it; without an amount nothing is limited.

Before the start: the month's bill

Before a run begins, the service adds up what the organisation and the project have already cost in the current month. If that lies above the cap, no further run starts. The run that breaches the cap itself is halted along the way: steps not yet begun are blocked, running ones may finish.

Volume limits per model tier

Independently of the amount, you set per model tier how many runs an organisation may start and how many of them may run at the same time.

Costs per project, not per month

What a project consumed is written at the project and in the run per step, not as a monthly total someone has to work back from. For local models you set a price per unit of compute time so that their runtime shows up in the bill as well.

The service speaks up

Four occasions report on their own: a failed run, a completed run, a requested approval and a run that stalls. Each member decides per occasion whether the message appears in the cockpit, arrives by e-mail, or both; in the cockpit it defaults to on, by e-mail to off. The owner of an organisation additionally switches on a Slack message per occasion, sent to a webhook of their own.

Settings and channels are described in thedocumentation (in German).

Metrics and tracing

Anvil Coder exposes its metrics in the usual format; Prometheus and Grafana are the intended chain. Tracing reaches across the queue: a job remains one coherent operation from the coordinator into the worker, even though a message sits in between.

Self-monitoring

A monitor observes the running operation, compiles findings, restarts what hangs and repairs what can be repaired automatically. A recurring check run establishes whether the chain as a whole still holds, instead of relying on the absence of alarms: a hermetic test run in the same namespace, on its own channels and without a model.

Both complement an operations team, they do not replace it. A system that monitors itself still needs someone who reads the findings.

Idle: the fleet scales to zero

Without work the worker fleet scales down to zero. The trigger is the backlog in the queue: when work is waiting, the workers come back. The base load, that is coordinator, databases, message bus, login and monitoring, keeps running. Idle is therefore cheaper than operation, but not free.

Rented or self-hosted

What is switched on in the rented service, and what you switch yourself when you run Anvil Coder on your own:

Metrics and tracing

Switched on in the rented service. Self-hosters enable the metrics endpoint and the trace export.

Monitor and check run

Rolled out in the rented service, with restart and automatic repair armed. Self-hosters roll out both and decide which hand is armed.

Fleet to zero

In the rented service the worker fleet drops to zero without work. Self-hosters set the lower bound.

Cost caps and quotas

Set by you in both cases, per organisation, project and run. Without an amount none applies.

What the service runs on