Tenant Lifecycle

This guide explains what happens when tenants are created, attached, or restored, and what steps you need to take in each scenario.

Creating a New Tenant

Creating a new tenant is the most common operation. It sets up a fresh, fully configured environment.

What Happens Automatically

1. Asset Repository creates a new MongoDB database and registers the tenant
2. A PosCreateTenant event is published to the distribution event hub
3. All services receive the event and run their tenant setup:
   • Import their Construction Kit models
   • Run data migrations
   • Create MongoDB indexes
4. Identity Service creates:
   • 10 default roles (TenantManagement, UserManagement, etc.)
   • TenantOwners group with all default roles
   • API scopes (octo_api, octo_api.read_only)
   • API resource (octoAPI) and identity resources
   • Default identity providers (Google, Microsoft — disabled; system tenant only). Child tenants receive an OctoTenant provider pointing to the parent tenant instead.

Steps

1. Ensure you are in the system tenant context:

   octo-cli -c Config -tid "octosystem" -isu "https://localhost:5003/" -asu "https://localhost:5001/"

   octo-cli -c LogIn -i

2. Create the tenant:

   octo-cli -c Create -tid "my-project" -db "my_project_db"

3. Grant yourself access to the new tenant (from system tenant):

   octo-cli -c ProvisionCurrentUser -ttid "my-project"

4. Switch to the new tenant context:

   octo-cli -c Config -tid "my-project" -isu "https://localhost:5003/"

   octo-cli -c LogIn -i

5. Configure identity providers (optional):

   octo-cli -c AddAzureEntryIdIdentityProvider -n "Corporate Azure AD" -t "<azure-tenant-id>" -cid "<client-id>" -cs "<client-secret>" -e true

6. Create groups and assign roles (optional):

   octo-cli -c CreateGroup -n "Operators" -rids "DashboardViewer,ReportingViewer"

7. Import your Construction Kit model:

   octo-cli -c ImportCk -f "./my-model.yaml" -w

info

Step 3 (ProvisionCurrentUser) is critical. Without it, you cannot access the new tenant because you have no allowed_tenants claim for it.

Scratch Tenants for Agents

Agents and automation create throw-away child tenants with a client-credentials client of the parent. Without help such a client has no roles in the child, so a human would have to sign in to every child to work in it and to clean it up. A scratch tenant removes that login: the client that creates the tenant becomes administrator of it.

Prerequisites. The client is a client-credentials client of the parent tenant, has the role TenantManagement there and is flagged AutoProvisionInChildTenants (AddClientCredentialsClient -apic), so the child gets a mirror of it.

1. Create the tenant as a scratch tenant (log in with the client: OCTO_CLI_CLIENT_ID / OCTO_CLI_CLIENT_SECRET, LogInClientCredentials):

   octo-cli -c Create -tid "scratch1" -db "scratch1_db" --scratch

   MCP: create_tenant with scratch: true. The tenant records its creator, and the client's mirror in the new tenant joins the TenantOwners group. Only that client, in only that tenant, is affected; GetTenants shows isScratch, scratchCreatorClientId, scratchCreatedAtUtc and scratchAccessRevoked. Without --scratch nothing changes. A user token, a token of another tenant or a client without TenantManagement is refused with HTTP 403.

2. Work in the child: request a token for the child with the mirror's credentials (client_credentials, acr_values=tenant:<child>, scope octo_api; with octo-cli a context for the child and LogInClientCredentials) and use the existing commands and MCP tools. The mirror's own secret comes from RotateMirrorSecret; while the inherited secret is still accepted, the parent client's secret works there too.

3. Clean up: switch off the capabilities and delete the tenant in one step:

   octo-cli -c Delete -tid "scratch1" --disable-features -y

   MCP: delete_tenant with disableFeatures: true, confirm: true. The command uses your token for the child, prints the capabilities it disabled (AI Services, Communication, Reporting, Stream Data) and then deletes the tenant. It never deletes archives: if Stream Data stays enabled because archives are still activated, the archives are listed, the tenant is not deleted and the command fails — disable them with DisableArchive (data is kept) or remove them with DeleteArchive. The command gets the child token from OCTO_CLI_CLIENT_ID and OCTO_CLI_CHILD_CLIENT_SECRET (the secret of the mirror in the child); without that variable it tries OCTO_CLI_CLIENT_SECRET, which only works while the child still accepts the inherited parent secret. The MCP delete_tenant with disableFeatures needs a user session (the token exchange needs a user); client-credentials agents use the CLI.

   Which secret where. For direct work in the child (a child context: Restore, Dump, feature switches) the mirror's secret goes into OCTO_CLI_CLIENT_SECRET, together with OCTO_CLI_CLIENT_ID. OCTO_CLI_CHILD_CLIENT_SECRET is only read by commands that run in the parent context and fetch the child token themselves (Delete --disable-features). The mirror's own secret is issued with POST …/clients/<clientId>/mirrors/<childTenantId>/secret on the identity service (there is no CLI command for it); it is returned once. The inherited parent secret also works for the mirror.

Restore. There is no --scratch flag on Restore. To restore a dump into a scratch tenant, create the tenant with Create --scratch, log in to it with the client's mirror (child token) and restore into that tenant. The scratch record lives outside the tenant database, so the restore keeps it and the creator stays administrator; a plain restore writes or changes no scratch record.

Persistence and revoke. The grant is re-applied from the record every time the tenant is set up and every time the mirror is provisioned, so it survives an identity restart, a restore and a re-provisioned mirror. Removing the client from TenantOwners (RemoveClientFromGroup) revokes it for good: the revoke is stored with the tenant, the setup does not grant it again, and GetTenants shows scratchAccessRevoked. Grant and revoke are written to the event log of the tenant and of the system tenant. The record is removed with the tenant (Delete, Detach).

Inspecting and Recovering Tenant Provisioning

Provisioning a tenant — during Create, and the automatic setup that runs on every service start — is a durable, self-healing process: its state is persisted in the system database, survives service restarts, and a background reconciler drives any incomplete setup to completion. Two operator commands let you inspect and, if ever needed, nudge that state.

Inspecting the lifecycle state

octo-cli -c GetTenantLifecycle -tid "my-project"

Returns the tenant's provisioning state as JSON:

• State — Creating (setup in progress), Active (fully provisioned and usable), Deleting (deletion in progress), or Failed (setup gave up after the retry budget and needs an operator).
• Phase, AttemptCount, LastError — how far setup progressed, how many attempts it has made, and the last error observed while the tenant was not yet Active.

A tenant with no lifecycle record (for example one created before this feature) reports "No lifecycle record found" — treat it as a normal, already-active tenant.

Recovering a stuck tenant

In the rare case a tenant is left in Creating or ends up Failed — for example if the identity service was briefly unavailable during a burst of tenant activity — re-open its setup so the reconciler completes it:

octo-cli -c ReRunTenantSetup -tid "my-project"

This resets the tenant to Creating, clears the attempt budget, and lets the background reconciler finish provisioning it. Watch it return to Active with GetTenantLifecycle.

tip

You normally never need this — setup completes on its own and self-heals after restarts. ReRunTenantSetup is a safety valve for the rare case a tenant is left half-provisioned.

Attaching an Existing Database

Attach registers an existing MongoDB database as a tenant. This is used when:

• A database was previously detached
• A database was created outside of OctoMesh
• Moving a database from one OctoMesh instance to another

What Happens Automatically

1. Asset Repository registers the existing database as a tenant
2. Unlike create, attach does not trigger full initialization — the database is assumed to be already set up

Steps

1. From the system tenant context:

   octo-cli -c Config -tid "octosystem" -isu "https://localhost:5003/"

   octo-cli -c LogIn -i

2. Attach the database:

   octo-cli -c Attach -tid "restored-tenant" -db "existing_database_name"

3. Clear the tenant cache (forces all services to re-initialize):

   octo-cli -c ClearCache -tid "restored-tenant"

4. Grant yourself access (if not already present):

   octo-cli -c ProvisionCurrentUser -ttid "restored-tenant"

warning

If you attach a database from a different OctoMesh installation, you may need to update the system CK model:

octo-cli -c UpdateSystemCkModel -tid "restored-tenant"

Restoring from a Backup

Restoring recovers a tenant's data from a backup file. The backup includes all MongoDB data (CK models, runtime entities, identity data).

Important Considerations

• The restore operation overwrites the target database
• You must create or attach the tenant first
• After restore, you need to clear the cache so services pick up the restored data
• If the backup came from a different OctoMesh version, CK model migrations may run on first access
• Identity data (users, roles, groups, providers) is included in the backup. A dump and restore is the only way to copy a tenant with its users: ImportRt refuses users since System.Identity 2.28.0 (see Moving Users to Another Tenant)
• Passwords and two-factor keys are stored encrypted with the key ring of the environment: they keep working after a restore in the same environment; from another environment they arrive unreadable and the users reset their password and set up two-factor authentication again, unless the source key is added to the target ring (see Restore Across Environments)

Steps

1. From the system tenant context:

   octo-cli -c Config -tid "octosystem" -isu "https://localhost:5003/"

   octo-cli -c LogIn -i

2. Create the tenant first (sets up infrastructure):

   octo-cli -c Create -tid "restored-tenant" -db "restored_tenant_db"

3. Grant yourself access before restore (identity data will be overwritten):

   octo-cli -c ProvisionCurrentUser -ttid "restored-tenant"

4. Restore from backup (overwrites the database created in step 2):

   octo-cli -c Restore -tid "restored-tenant" -db "restored_tenant_db" -f "./backup.tar.gz" -w

5. Clear tenant cache (critical — services must reload the restored data):

   octo-cli -c ClearCache -tid "restored-tenant"

6. Re-provision yourself (restore may have overwritten your mapping):

   octo-cli -c ProvisionCurrentUser -ttid "restored-tenant"

caution

If the backup's original database name differs from the target, specify the old name:

octo-cli -c Restore -tid "restored-tenant" -db "new_database_name" -f "./backup.tar.gz" -oldDb "original_database_name" -w

Restore from a Different OctoMesh Instance

When restoring from a backup taken on a different OctoMesh installation:

# After the standard restore steps above:

# Update system CK model to current version
octo-cli -c UpdateSystemCkModel -tid "restored-tenant"

# Clear cache again
octo-cli -c ClearCache -tid "restored-tenant"

# Verify identity providers are correct (may need reconfiguration)
octo-cli -c Config -tid "restored-tenant" -isu "https://localhost:5003/"
octo-cli -c LogIn -i
octo-cli -c GetIdentityProviders

Creating a Backup

# From the system tenant context with Bot Services configured
octo-cli -c Config -tid "octosystem" -isu "https://localhost:5003/" -bsu "https://localhost:5009/"
octo-cli -c LogIn -i

# Dump the tenant
octo-cli -c Dump -tid "my-project" -f "./my-project-backup.tar.gz"

The backup is created as a background job. The CLI waits for completion and downloads the file.

Deleting a Tenant

danger

Deleting a tenant permanently removes the MongoDB database and all its data. This action cannot be undone. When Stream Data is enabled on the instance, the CrateDB tables of the tenant's archives are dropped as well — best-effort: if CrateDB is unreachable the delete still succeeds, an error is logged naming the tables that have to be dropped manually. Only the tenant's own archive tables are touched, never other tables in the same CrateDB schema. Detach keeps both the database and the tables; a Restore over an existing tenant replaces only the database and keeps the archive tables as well.

octo-cli -c Delete -tid "my-project"

warning

A tenant can only be deleted while none of its optional capabilities are enabled. If Stream Data, Communication, Reporting, or AI Services is still enabled on the tenant, the delete is rejected with HTTP 409 and the error message names the capabilities that are still enabled.

Disable them first. The disable commands take no -tid argument — they act on the tenant of the active octo-cli context, so switch to the context of the tenant you want to delete (UseContext) or pass --context:

octo-cli --context <child-context> -c DisableStreamData
octo-cli --context <child-context> -c DisableCommunication
octo-cli --context <child-context> -c DisableReporting
octo-cli --context <child-context> -c DisableAi

Enabling and disabling a capability (the Enable… / Disable… commands, the Studio panel and the MCP tools alike) requires the role TenantManagement in that tenant, and the token must be issued for that tenant. A token without the role is refused with HTTP 403 and a message that names the missing role (or the missing octo_api scope); a token of another tenant, also of the parent tenant, is refused with HTTP 403 as well. A client-credentials client that is only mirrored into a child tenant therefore cannot switch the child's capabilities until it is given the role there; the creator of a scratch tenant has it through the TenantOwners group.

All four capabilities can alternatively be disabled in Refinery Studio under Settings > Tenant (the Features and backup part) of that tenant. The panel reads the same enabled state this precondition evaluates (the aggregate GET {tenantId}/v1/features/status of the asset repository), so a capability the panel shows as disabled never blocks the delete. A capability whose service is not part of the installation (its URL in the _configuration discovery document is empty) is shown as Not installed without toggle buttons; if its enabled flag is still set anyway, the panel shows a warning — delete and detach keep refusing until the flag is disabled with the octo-cli commands above.

DisableStreamData has a precondition of its own: it is rejected with HTTP 409 while any archive of the tenant is still activated — the error names them. Disable them first with DisableArchive (data is kept) or remove them with DeleteArchive (rollups before their source archive), or in Refinery Studio under Data > Stream Data archives. The command only switches the tenant flag off: the System.StreamData model, the archive definitions and the stored stream data stay in the tenant (see Stream Data Archives and the Studio archives page).

octo-cli --context <child-context> -c DisableArchive -id <archiveRtId>
octo-cli --context <child-context> -c DisableStreamData

DisableCommunication has a precondition of its own: it is rejected with HTTP 409 while any deployment site or workload (Adapter or Application) of the tenant is still deployed — the error names them with their deployment state. Undeploy them first with UndeployWorkload (one call per workload) and then UndeployPool (both act on the context tenant, like the disable commands), or in Refinery Studio under Communication > Adapters / Applications / Deployment Sites. Pipelines and pipeline triggers need no action — disabling Communication takes care of them.

octo-cli --context <child-context> -c UndeployWorkload -id <workloadRtId> -y
octo-cli --context <child-context> -c UndeployDeploymentSite -id <deploymentSiteRtId> -y
octo-cli --context <child-context> -c DisableCommunication

DisableReporting and DisableAi have no precondition of their own: both only remove the enabled flag. Report definitions and stored reports (Reporting) and the AI configuration and session data (AI Services) stay in the tenant and are usable again after EnableReporting / EnableAi (EnableAi requires Communication to be enabled on the tenant).

If the tenant's data is still needed, take a backup with Dump before disabling the capabilities (see Creating a Backup) — Dump is not affected by this precondition.

Delete --disable-features -y does the disabling for you: it switches off every capability the tenant has enabled and deletes the tenant afterwards (MCP: delete_tenant with disableFeatures: true). It acts with your token for that tenant — the capability endpoints belong to the tenant's own administrator — which is why it fits the creator of a scratch tenant and a user who is administrator of the tenant. Activated archives are reported and never deleted; the tenant then stays undeleted.

info

Delete followed immediately by a new Create of the same tenant id (for example when replacing a demo tenant) is safe: while the previous deletion is still completing its database drop, the Create returns a retryable "deletion still in progress, retry later" (HTTP 409) instead of a confusing error — simply retry and it succeeds once the drop finishes.

Detaching a Tenant

Detach unregisters a tenant from OctoMesh but keeps its MongoDB database. Use it to move a database to another OctoMesh instance or to take a tenant out of service without losing its data. The database can be registered again later with Attach.

octo-cli -c Detach -tid "my-project"

• The tenant must be a child of the tenant in your current context (as for Delete); otherwise the detach is rejected with HTTP 404.
• The same precondition as for Delete applies: Stream Data, Communication, Reporting, and AI Services must be disabled on the tenant first, otherwise the detach is rejected with HTTP 409 naming the capabilities that are still enabled. See the warning in Deleting a Tenant for the disable commands.

Cleaning a Tenant

Cleaning resets a tenant to factory defaults by removing all CK models (except system) and all runtime entities, while keeping the database:

octo-cli -c Clean -tid "my-project"

Summary of Operations

Operation	Creates DB	Initializes Config	Preserves Data	Use When
Create	Yes	Yes	N/A	Setting up a fresh tenant
Attach	No	No	Yes	Re-registering an existing database
Restore	No (needs Create first)	Overwritten	Backup data	Recovering from a backup
Clean	No	Partial reset	No	Resetting to factory defaults
Delete	Drops DB (+ CrateDB archive tables)	N/A	No	Permanently removing a tenant
Detach	No (keeps DB)	N/A	Yes	Unregistering a tenant while keeping its database