Skip to main content

FromPipelineDataEvent@1

The FromPipelineDataEvent@1 node is used to receive data from a pipeline data event and trigger the execution of the transformation pipeline.

This node automatically registers two consumers:

  • Pub/sub consumer: Binds to the DataFlow topic exchange using the pipeline's own runtime entity ID as the routing key, receiving fire-and-forget messages from upstream ToPipelineDataEvent@1 nodes.
  • Command consumer: Listens for command/response requests from upstream ToPipelineDataEvent@1 nodes that have awaitResult: true. Executes the pipeline and returns the result to the caller.

No configuration is needed to enable either mode — the node handles both automatically.

Inter-Pipeline Communication Architecture​

The DataFlow entity establishes a shared topic exchange in the event hub. When pipelines within the same DataFlow need to communicate:

Fire-and-Forget Mode​

  1. The sending pipeline uses ToPipelineDataEvent@1 with a targetPipelineRtId to publish data to the DataFlow's topic exchange
  2. The receiving pipeline uses FromPipelineDataEvent@1 which automatically binds to the exchange using its own pipeline runtime entity ID as the routing key
  3. The event hub routes the message to the correct pipeline based on the routing key match

Command/Response Mode (Await Result)​

  1. The sending pipeline uses ToPipelineDataEvent@1 with awaitResult: true to send a command request
  2. The receiving pipeline's command consumer executes the pipeline and captures the result
  3. The result is sent back to the sender, which places it at resultTargetPath in the data context

In command/response mode, if the target pipeline fails, the error is propagated back to the sender — the sender's execution will also fail with the target's error message.

Adapter Prerequisites​

  • General availability: All adapters support this node type.

Node Configuration​

triggers:
  - type: FromPipelineDataEvent@1

No additional configuration is needed. The node automatically:

  • Uses the pipeline's own runtime entity ID as the routing key when binding to the DataFlow's topic exchange (pub/sub)
  • Registers a command consumer at a deterministic address derived from the tenant, DataFlow, and pipeline IDs (command/response)