Secrets & orchestration

The contract

This is the agreed target behavior. Publication of this contract does not establish that Gimbal implements it. Implementation and validation are recorded separately below. This page supersedes the earlier planning proposals, including the proposal that a service launcher exits to signal readiness.

The October 19 product anti-goal supersedes earlier fixed shutdown deadlines and retained-cleanup obligations. The workflow orchestrator has no strict timing requirements or cancellation-completeness journal. Optional timing observations do not create requirements or gates.

Workflows declare what their commands and services yield and what their consumers require. Producers and consumers use plain names such as DATABASE_URL. The values themselves remain outside workflow definitions and source code. Treat every value in this channel as confidential, including transient addresses that contain no credentials.

A service is one foreground program that remains running until its scope tells it to stop. It establishes readiness, atomically publishes an output file, and remains responsible for reporting resource failure. Gimbal publishes the outputs into a store before admitting consumers. On shutdown, Gimbal requests termination; the executing node and resource owner eventually release owned resources.

Service boundaries

Orchestration establishes execution environments, connectivity, resource ownership, consumer access, and scoped cleanup. It observes the service program’s lifetime. Resource-specific readiness and ongoing health checks belong to the service program supplied by the workflow author.

The secrets/runtime-value service publishes, stores, and delivers named confidential values. It confirms publication or reports failure. It can be part of orchestration or an injected dependency; this contract does not require a separate process, package, or public secrets-service interface.

Messaging delivers execution requests, cancellation requests, and outcome events. Transport owns delivery behavior; executing nodes respond to cancellation. The workflow orchestrator does not add a second cancellation ledger or release-confirmation protocol. Messaging’s concrete implementation may need more detailed discovery and connection information than the public orchestration interface exposes. Supply that capability through construction. Do not recover a concrete orchestration implementation through type assertions or expose provider handles to workflow code.

Workflow code has a semantic relationship to orchestration through declarations, scopes, and completion promises. It does not need to hold an orchestration object. The executor establishes prerequisites before starting a consumer and reports ordinary outcomes.

Provider clients, credentials, placement, and transport configuration are constructed outside the workflow. Messaging’s bootstrap credentials also come from external construction: they cannot depend on a producer whose execution already requires that messaging connection.

Names, stores, and scope

A producer yields DATABASE_URL; a consumer requires DATABASE_URL. The consumer does not identify the producing operation or its machine. Authors can distinguish application values with names such as PRIMARY_DATABASE_URL and REPORTING_DATABASE_URL. No producer-qualified paths or structured key language are required.

Names are compile-time constants. There is one namespace across stores, following ordinary scope visibility: a child can see ancestor values and can shadow them with its own definition. There is one definition of a name in a scope instance. Stores do not disambiguate conflicting definitions or change name lookup. A consumer must require the store containing the visible definition; store access does not make a hidden ancestor definition visible instead.

Each producer writes one ENV_FILE and selects one logical destination store for all its declared outputs. Omitting the store selects the default store. A producer cannot publish the same name into two stores. These declaration restrictions belong to lint and author responsibility.

A store name denotes a logical collection; deployment construction selects its physical backend. Requiring a store implies access to all its scope-visible contents. Required key names identify what must be ready before execution; they are not per-value permissions. Access need not eagerly copy the entire store into the consumer’s environment.

Put shared producers in the scope enclosing their consumers, and establish shared values before starting parallel consumers. Values do not implicitly escape into parents or sibling scopes. Repeated work gets fresh child scopes and fresh output locations. Retained storage from an earlier invocation cannot satisfy a new invocation’s setup obligation.

Externally supplied values must be identified in workflow declarations and made available by deployment construction before their consumers execute. They follow the same visibility, access, and delivery promises. Workflow definitions contain their names, not their contents.

Producing values

Gimbal supplies a private output location through ENV_FILE. Before starting the producer, it ensures the final path is absent, removing any existing file at that invocation’s location. Removal failure fails setup. Live invocations do not share an output path. Temporary files and final files receive confidential handling and are removed when their scoped use ends; they are not ordinary run artifacts.

Finite commands write and close ENV_FILE, then exit zero. Gimbal reads the file only after successful exit. A nonzero exit fails the command regardless of file contents. Command completion includes confirmed publication of its declared outputs.

Services stay running and use the final file’s appearance as their readiness signal:

1. Establish the resource and the promised readiness.
2. Write the complete output to a temporary file in the same directory as ENV_FILE.
3. Close the temporary file and atomically rename it to ENV_FILE.
4. Leave the final file unchanged and keep running.

A service with no yielded values publishes an empty file to signal readiness. The temporary and final paths must support atomic rename; a cross-filesystem copy is not a substitute. No callback, partial-file reading, repeated publication, or streaming protocol is needed.

The executor waits for the final file while also observing service exit and cancellation. It then parses the file, checks that required declared outputs are present, and publishes them through the selected backend. This can happen in the producer’s execution environment; secret bytes need not pass through the central workflow runner.

File appearance declares resource readiness. Store acknowledgement completes value publication. Consumers proceed only after publication and their delivery requirements succeed. Gimbal trusts the producer’s readiness promise without introducing a second resource-specific health probe. The service remains subject to failure handling during publication and afterward.

A missing readiness file remains pending while the service runs, until cancellation or an applicable execution deadline. This does not authorize a retry or an implicit scheduler for future producers.

Output format

One NAME=value entry per line, split at the first =:

DATABASE_URL=postgres://...
CERTIFICATE_BASE64=LS0tLS1CRUdJTi...

• Names begin with a letter or underscore and contain only letters, digits, and underscores.
• Values are literal. Quotes remain characters. There is no shell evaluation, expansion, escape processing, or export syntax.
• Empty values are allowed. Blank lines are ignored. LF and CRLF line endings are accepted; the final newline is optional.
• Malformed entries and duplicate names fail parsing. Required declared outputs must be present. Extra entries are ignored and do not create workflow outputs.
• Multiline or awkward data can be base64 encoded. Producers and consumers agree on encoding, and consumers decode it. Base64 does not provide confidentiality.

Parsing and output-presence checks concern runtime data. They do not justify duplicate runtime machinery for authoring mistakes that lint can largely prevent.

Service lifetime

The service program stays in the foreground. It may run the resource directly or act as a wrapper around another resource, but it must not detach itself or exit after merely starting that resource.

The wrapper owns the application-specific setup and health contract. When the resource becomes unavailable, the wrapper exits unsuccessfully. A wrapper that stays alive while ignoring a dead resource violates that contract. Gimbal observes process lifetime rather than implementing a catalog of database and application health checks.

Any unexpected exit fails the scope and workflow, even if its status is zero. This applies before readiness, while publishing values, and after consumers start. There are no automatic service restarts or scope retries in this initial contract.

Shutdown

Normal scope completion collects final results and removes scoped context. Cancellation requests that owned work stop, including after partially completed startup or failed value publication. The executing node requests service termination, using SIGINT where supported, and resource owners eventually release their resources. The orchestrator does not enforce a shutdown deadline or precise UI-delivery cutoff.

Cleanup stops or releases what the program owns. Releasing use of an existing shared resource must preserve unrelated users. Normal requested shutdown is not unexpected service failure. Report shutdown errors plainly; storage failure must not prevent termination requests or release of local handles. Final output arriving after cancellation may be lost; unfinished output must not be presented as a successful complete result.

The executor may terminate a program through its supported execution boundary. Terminating a wrapper does not prove that an external resource was released. An unreachable external operation may require manual intervention. The central orchestrator releases its own resources without retaining the terminal run for cleanup retries, collecting release receipts, or journaling cancellation completeness. Timings may be logged in place, but must not become requirements, enforced thresholds, or acceptance gates.

Future retry can rerun the same command if its author makes setup safe to repeat. Idempotence remains an author obligation, not a property Gimbal can infer from the command text. This contract does not introduce retries or rollback of arbitrary side effects.

Access and delivery

The backend must provide required store access during the consumer’s scoped lifetime. Programs may receive declared values through environment variables or files. Agent integrations may expose session files or tools. Selecting an agent delivery mechanism does not automatically authorize copying secrets into prompt context; a tool that prints a secret exposes it to the model when that output enters its context.

Producers and consumers agree on names, formats, and encodings. The workflow declares the relationship without knowing the storage provider. Gimbal does not infer application meaning or decode values automatically. The ordinary scoped context channel is not a substitute for this confidential channel when its contents are recorded or appended to agent prompts.

A producer must emit a complete connection value usable by all known consumers. For example, DATABASE_URL already accounts for any routing, forwarding, or address translation required by the deployment. The value subsystem treats it as opaque and delivers it unchanged. Consumer-specific address resolution is outside this contract.

Gimbal’s ordinary events and UI show names, dependencies, publication state, execution, and failure without value contents. Producers and consumers must avoid printing secrets into captured output. A general redaction system for arbitrary application disclosures is not required.

Failure behavior

A backend write may have happened even when its acknowledgement was lost. Gimbal still treats that publication as failed. Exactly-once side effects and reconciliation of uncertain writes are not prerequisites. Provider retries may remain below the interface without changing the workflow’s completion promise.

Consumers do not continue with a successful subset of a failed producer’s outputs. Replacing a stored value does not update an already running process, and live refresh is not supported initially. New scope invocations establish current resources and values through their normal setup.

Lint and authorship

Lint owns supported checks for constant names, duplicate declarations, scope visibility, permitted ordering, and explicit joined concurrency. Keep declarations readable and available to the source-derived workflow graph. Shared values precede parallel consumers; no general dependency-planning system is necessary.

Do not design the architecture around complete runtime prevention of author misuse. When lint can prevent most of an invalid pattern, the remaining responsibility can stay with the workflow author. Incomplete static coverage is not a reason to duplicate those checks throughout execution.

Runtime checks remain appropriate for facts source inspection cannot establish: actual exit status, readiness-file contents, missing produced values, publication acknowledgement, access and delivery errors, cancellation, and cleanup failure.

Workflow code and consumers should not repeat prerequisite checks already promised by lower-level services. Handle errors those services report. Application-specific validation outside those promises remains the application’s responsibility.

Design limits

Permanent exclusions: Gimbal will never add per-value access-control lists, roles, permission delegation, or independently revocable grants to this workflow model. Named stores are the only access-control-like abstraction admitted here: requiring a store implies scoped access. Provider-native credentials may implement that promise privately.

Orchestrator anti-goal: all strict timing requirements, cancellation-completeness journals, cleanup receipts, release-confirmation ledgers, and retained retry-cleanup lifecycles are excluded. Optional timing observations are not requirements and must not be enforced as gates.

The initial contract also excludes:

• Producer-qualified keys, structured key paths, and competing store-specific namespaces.
• Mandatory distinctions between secret and non-secret runtime values.
• Consumer-relative network address rewriting by the value service.
• Incremental output publication, live rotation, and transparent hot reload.
• Automatic service restart, scope retry, failover, or native agent-session recovery.
• Partial-success continuation, generalized rollback, and distributed publication transactions.
• A general dependency scheduler, universal health-check language, or exhaustive static/runtime correctness proof.
• Support for every combination of transport, provider, and network topology.

Implementation and proof

This page establishes the contract. The complete contract has not been validated against a Gimbal implementation. Do not treat its promises as delivered features or mark existing inventory claims as passing because this page exists.

At the reviewed source baseline, Gimbal 4a8d8153, foreground services already have scope ownership and fail on premature exit. They return after process start without establishing readiness, and shutdown uses SIGTERM followed by SIGKILL. That implementation does not fulfill this contract’s atomic readiness-file and SIGINT rules. Its non-secret HTTP service-binding channel also differs from this confidential value contract.

Implement runtime values alongside the first concrete orchestration implementation. The first slice is one real path: a foreground database program establishes readiness and publishes DATABASE_URL; Gimbal confirms storage; a web consumer receives access and uses the URL; scope cleanup stops the service and releases use. Begin with one simple store backend. Vault, Doppler, broader delivery adapters, and agent-specific syntax can follow.

The new runtime-value declarations are the intended replacement for the workflow-facing WithServices / ServiceBinding delivery path. Retire the separate URL-returning FulfillService operation when the new service path covers its responsibilities. Preserve necessary scoped acquisition and release inside backends. Messaging bootstrap remains externally constructed. Exact Go signatures and provider wiring are implementation choices, not a new workflow permission language.

Demonstrate startup and publication, actual consumer use, cancellation with eventual resource release, unexpected service exit before and after readiness, missing required output, and publication failure. No fixed shutdown timing is an acceptance condition. Report the revision and behavior exercised. A manually supplied database URL does not establish this path.

Filesystem messaging, River/Postgres, and HTTP/SSE compositions must each inform subsequent implementation and evidence. The first backend demonstration is not proof of all three. Messaging prerequisites must remain available for cleanup that uses messaging, and releasing one consumer must preserve unrelated active use.

Preserve secret-free lifecycle observations in the first slice. UI rendering and live verification may follow separately. Human legibility remains unverified until the running UI shows dependencies, waiting, publication, execution, and failure clearly. Delivery status and delivery evidence remain the places to record what was actually observed.