Skip to main content

Pipeline Method Handlers

A method declared with execution.handler: pipeline is implemented by a pipeline instead of C# code. A tenant administrator or solution builder can add a business operation to a data model without writing a service: the model declares the method, a pipeline implements it, and callers use it like any other method. This page describes the contract between the platform and the pipeline. The general rules for handlers (consistency, errors, timeouts) are in Implementing Method Handlers and the Method Consistency Model.

Where it runs

The communication controller handles pipeline methods. It already starts pipelines on command (the Execute action of the user interface and the ExecutePipeline API), so it starts the pipeline for a method call the same way, with the caller's identity, and waits for the result. No change to the adapter is needed.

Declare the method​

types:
- typeId: Device
methods:
- methodId: UpdateFirmware-1
parameters:
- name: version
valueType: String
result:
valueType: Int
errors:
- code: FIRMWARE_LOCKED
execution:
timeoutSeconds: 120 # mandatory for a pipeline method
handler: pipeline

The qualified method id of this method is Acme.Assets/Device.UpdateFirmware-1 (model, type, method with version). A pipeline method must declare timeoutSeconds and must not have parameters or a result marked sensitive: the compiler refuses the combination with message 132 (parameter) or 134 (result), see Limits.

Bind the pipeline​

There is no binding element. The pipeline that implements a method is the deployed pipeline in the call's tenant whose well-known name equals the qualified method id:

entities:
- rtId: 'a1b2c3d4e5f60718293a4b5c'
ckTypeId: System.Communication/Pipeline
rtWellKnownName: Acme.Assets/Device.UpdateFirmware-1
associations:
- roleId: System/ParentChild
targetRtId: '...' # a DataFlow
targetCkTypeId: System.Communication/DataFlow
- roleId: System.Communication/Executes
targetRtId: '...' # the adapter that runs it
targetCkTypeId: System.Communication/Adapter

Deploy it like any pipeline (DeployPipeline). A changed pipeline takes effect after it is deployed again. The controller announces the method as soon as a deployed pipeline of that name exists and withdraws it when the pipeline is undeployed or deleted; no pipeline means the call is answered METHOD_HANDLER_NOT_AVAILABLE (reason pipeline-not-deployed). If a method is declared handler: pipeline but no pipeline exists when the controller starts, the controller logs an error that names the method and the tenant and keeps serving; set PipelineMethods:FailStartupOnInvalidMethod to true to make the startup fail instead.

What the pipeline receives​

The pipeline starts with FromExecutePipelineCommand@1. The data root contains:

{
"parameters": { "version": "2.4.1" },
"target": { "ckTypeId": "Acme.Assets/Device", "rtId": "65f0c0ffee0000000000dead" },
"method": { "id": "Acme.Assets/Device.UpdateFirmware-1", "invocationId": "...", "correlationId": "..." }
}
PathContent
$.parameters.<name>The method parameters, camelCase, as the caller sent them.
$.target.ckTypeId, $.target.rtIdThe entity the method is called on. rtId is null for a static method.
$.method.*Method id, invocation id and correlation id; use them in log messages.

The caller is not part of the data. The pipeline runs as the user who called the method, exactly as for a manual execution: nodes that read the verified caller see that user, and nodes that act for the caller against another service use the forwarded token. Set callerBinding: BindingRequired on the trigger so that the pipeline is refused when no caller was carried instead of running as the service account.

Trust of the caller

The method queue is not authenticated per message, so the caller data of a request proves nothing: anyone who can publish to the queue could name an administrator there. The controller therefore validates the forwarded access token before it looks at the method or starts a pipeline, with the same trust as its HTTP API (issuer, signature, expiry, tenant, scope octo_api; see Caller token validation). The pipeline caller is built only from the validated token, with trust level strong, and the caller data of the request is never used. A user token runs the pipeline as the token's subject; a service token (client credentials, no sub) runs it with the client id as subject.

A call without a valid token is answered METHOD_FORBIDDEN with a reason (token-missing, token-expired, token-invalid, tenant-not-allowed) and no pipeline is started. If the controller cannot load the signing keys of the identity service, the answer is METHOD_HANDLER_FAILED: the caller did nothing wrong, and a token is never accepted without a verified signature. After token-expired, retry with a fresh token.

What the pipeline returns​

The pipeline ends with SetPipelineExecutionResult@1. The value at its path is stored as the execution's OutputData and becomes the answer:

OutputDataAnswer to the caller
{"value": 1}, the value has the type of the method's resultSuccess. The caller receives the bare value 1 as the result, exactly as from a service handler (a string result is "abc", a record {"name": "x"}); the {"value": ...} wrapper is only the shape of the pipeline output. Other members are dropped.
empty, for a method without resultSuccess without result.
{"error": {"code": "FIRMWARE_LOCKED", "parameters": {"reason": "maintenance-window"}}}The declared error with its parameters. The code must be declared in the method's errors, or be one of METHOD_VALIDATION_FAILED, METHOD_TARGET_NOT_FOUND, METHOD_FORBIDDEN. Parameter values are strings, numbers or booleans and must not contain method parameter values.
anything else: not JSON, no value, wrong type, an undeclared codeMETHOD_HANDLER_FAILED with correlationId, field and reason invalid-result. Never a malformed success.
the pipeline failed on a write the database did not acknowledge (for example a write-concern timeout, or a commit whose result the driver reports as unknown)METHOD_OUTCOME_UNKNOWN with correlationId: the write may be on the primary, so "nothing happened" would be wrong. The adapter classifies the exception and reports a structured failure kind (PipelineExecution.FailureKind = OutcomeUnknown); the controller never reads the error message for this. A write the database refused (for example a duplicate key) stays METHOD_HANDLER_FAILED. Read the state before you call again (R5).
the pipeline failed for another reason, was interrupted or cancelledMETHOD_HANDLER_FAILED with correlationId. The error message of the execution stays in the log of the controller. An interrupted execution, and an execution that an adapter reports after it was offline, carry no failure kind and are answered METHOD_HANDLER_FAILED even if a write was in flight.
the pipeline is not finished when the deadline passesMETHOD_TIMEOUT. The pipeline is not cancelled and may still finish; see Limits.
the pipeline cannot be started, for example the adapter is offlineMETHOD_HANDLER_NOT_AVAILABLE with reason pipeline-start-failed.

See the failure kind of an execution​

The failure kind is not only the answer to a method caller. An operator reads it on the execution itself: the REST pipeline execution (GET /{tenantId}/v1/PipelineDebug/{pipelineRtId} for the history and .../latest for the newest execution) carries failureKind, and so do the octo-cli commands GetPipelineExecutions and GetLatestPipelineExecution.

{ "Status": 2, "ErrorMessage": "...", "FailureKind": "OutcomeUnknown" }
failureKindMeaning for the operator
OutcomeUnknownThe pipeline failed on a write the database neither acknowledged nor refused. The write may have taken effect: read the state before you run the pipeline or call the method again.
FailedAn ordinary failure. The database refused the write, or the pipeline failed for another reason.
nullThe execution did not fail, or it carries no kind (see the limitation below).

The field is additive: clients and controllers from before this change ignore it or leave it out. Limitation: an execution that was interrupted (for example by a controller restart or a lost adapter) and an execution that an adapter reports after it was offline carry no failure kind. They show failureKind: null, and a method caller gets METHOD_HANDLER_FAILED even if a write was in flight.

Sample​

This pipeline implements Acme.Assets/Device.UpdateFirmware-1: version 0.0.0 is locked and refused with the declared error FIRMWARE_LOCKED, any other version succeeds with the result 1. The pipeline is also the definition used by the integration tests of the communication controller (Methods/SamplePipelines/Device.UpdateFirmware-1.pipeline.yaml).

triggers:
# BindingRequired: the pipeline never runs anonymously
- type: FromExecutePipelineCommand@1
callerBinding: BindingRequired
transformations:
# success path: the result record {"value": <Integer>}
- type: If@1
path: $.parameters.version
operator: NotEqual
value: "0.0.0"
valueType: String
transformations:
- type: SetPrimitiveValue@1
targetPath: $.result.value
value: 1
valueType: Int
# business refusal: FIRMWARE_LOCKED is declared in the errors list of the method
- type: If@1
path: $.parameters.version
operator: Equal
value: "0.0.0"
valueType: String
transformations:
- type: WriteJson@1
targetPath: $.result
jsonString: '{"error": {"code": "FIRMWARE_LOCKED", "parameters": {"reason": "maintenance-window"}}}'
- type: SetPipelineExecutionResult@1
path: $.result

A caller of the method receives 1, or the error FIRMWARE_LOCKED for version 0.0.0.

Consistency and limits​

  • Each call is its own unit of work. The nodes of a pipeline commit as they run; a later failing node does not roll back earlier ones. Put write nodes last or model compensation as nodes. The call runs outside the caller's request transaction like every method (consistency model).
  • Creating entities of the method's own type. If the method replaces the generic create of a type, the pipeline of that method is the only pipeline run in which ApplyChanges@1, ApplyChanges@2 and ImportFromExcel@1 may insert entities of that type; in every other run (including a manual execution of the pipeline and a pipeline chained with ToPipelineDataEvent) the insert is refused with GENERIC_CREATE_REPLACED_BY_METHOD. See Pipelines that insert entities.
  • A timeout does not stop the pipeline. After METHOD_TIMEOUT the pipeline may still finish; the outcome is unknown. A pipeline method that must not run twice is the author's responsibility. A request that the bus redelivers is not started again; the caller receives METHOD_OUTCOME_UNKNOWN.
  • No sensitive parameters or results. A pipeline execution stores its input and its output, so a method with a parameter marked sensitive: true cannot use handler: pipeline: the compiler refuses it with message 132, which names the method, the type and the sensitive parameters (decision OD-9). Implement such a method as a service handler. As a second gate, the controller also refuses a pipeline method with a sensitive parameter (or one of type Secret) at runtime, for a model that did not pass the compiler: it is not announced, the controller logs an error that names the method and the parameter names (never values), and a call is answered METHOD_HANDLER_NOT_AVAILABLE with reason sensitive-parameters-not-supported. The same holds for a result that is a secret: a result marked sensitive: true is refused with message 134, because the execution store keeps the output of SetPipelineExecutionResult (Sensitive results). The runtime gate covers results as well: a pipeline method whose result is marked sensitive (only possible with a hand-crafted or older definition that did not pass the compiler) is not announced either, and a call is answered METHOD_HANDLER_NOT_AVAILABLE with reason sensitive-result-not-supported, without starting a pipeline. If a method has a sensitive parameter and a sensitive result, the reason stays sensitive-parameters-not-supported. The platform redaction of sensitive results (Sensitive results) therefore never has to deal with a pipeline execution that stored the output.
  • Dedicated adapters. A pipeline on a leased adapter is queued until a pool member is leased, like any execution; with a short method timeout the call can end in METHOD_TIMEOUT.
  • One handler kind per method id. The method queue is per method id, not per tenant. Do not declare the same method id as service in one tenant and pipeline in another.
  • Synchronous only. An asynchronous method (execution.mode: async, operation handle) needs a service handler; the compiler refuses async together with handler: pipeline (message 142). See Asynchronous handlers.

Configuration of the controller​

Section PipelineMethods (environment variables OCTO_PIPELINEMETHODS__...):

SettingDefaultMeaning
Enabledtruefalse: the controller serves no pipeline methods.
HeartbeatInterval / Ttl15 s / 45 sHow often the controller re-checks the pipelines and re-announces itself, and how long an announcement is valid.
EndpointReadyTimeout30 sHow long it waits until a method queue accepts messages before it skips the method until the next heartbeat.
PollInterval / MaxPollInterval100 ms / 750 msHow often it reads the execution of a started pipeline while it waits.
FailStartupOnInvalidMethodfalsetrue: the startup fails if a pipeline method has no deployed pipeline or has sensitive parameters or a sensitive result.

See Also​