core module is the engine and control plane of Yukta. It takes a workflow definition (a DAG of trigger, processor and terminal nodes), runs it as a live stream of messages, lets people and programs steer it while it runs, and records enough to explain or resume it afterwards.
Last verified against commit d2424ed on 2026-10-15.
Reactive
Project Reactor (
Mono/Flux) everywhere; no blocking on the event loop.Streaming DAG
One stream per edge, with back-pressure from consumer to producer.
At-least-once
Retries and resumes may repeat a step; plugins de-duplicate on the message id.
Single JVM
Run state lives in one process; active/standby HA is optional.
Overview
Who this is for and how to read it
Scope: what is and is not in core
In core- Workflow definitions, versions, validation and compilation
- The stream engine and per-node failure and time policy
- Admission control, idempotent starts, the command pipeline
- Tracker, progress, results, metrics and tracing
- Execution log, checkpoints, recovery and resume
- Supervisor and reviewer hosts, worker registry, leader election
- API key and role model, variable allow-list
- REST controllers, SSE endpoints, Host/Origin filter:
web - MCP tools:
mcp. CLI:cli. UI:ui - Node implementations (
LLM_CALL,HTTP_REQUEST, …):plugins/*; contracts:plugin-api - PostgreSQL stores and Flyway migrations:
persistence-postgres - RSocket worker transport and agent:
workers - Application wiring and native image:
boot
Related documents. The core module guide covers the design debate (scenario replays, quality lenses, risks, decisions). This page is the stable reference: use it for what exists and how to configure it. Known bugs are tracked in the bug backlog; module build notes live in
core/CLAUDE.md. Related public pages: glossary, limits and defaults, status values and configuration.Status legend
Feature cards carry one of four statuses.Core concepts
The vocabulary used throughout. Terms appear incode style where they match a type, field or setting.
A workflow, concretely
The shape below is the real JSON model. Theengine block is the failure policy described in ENG-04; the plugin types are examples.
engine are abbreviated, and plugin option names are defined by each plugin. For complete, runnable workflows see examples and the workflow guide.
Architecture
Core splits cleanly into a data plane that moves messages through the graph and a control plane that admits, steers and records runs. Everything else (storage, workers, HA) plugs in behind interfaces.Position in the system
Component map
Solid arrows carry data. Dashed arrows carry commands (cmds) or the event feed about a run. Each box maps to a feature group in the catalog.Design principles
- Plugins hold business logic only. Back-pressure, retries, timeouts, pause/step and failure routing live in the engine once, wrapped around every node.
- One path to change a run. Humans, supervisors, REST and MCP all send the same commands through the same ordered, reviewable, audited pipeline.
- Safe defaults, explicit opt-in. Process execution is off, environment access is allow-listed, limits are off (not unbounded by accident: documented in the reference).
- Storage behind interfaces. Memory, files and PostgreSQL implement the same stores, so durability is a deployment choice, not a code path.
- Honest guarantees. At-least-once, forward-only status, FIFO per run: see Guarantees.
Run lifecycle
What happens from a start request to a finished run. Each step links to its feature.1
Start
Gateway receives the request; an
Idempotency-Key repeat returns the first id. ADM-022
Prepare
Load, validate, compile; concurrent starts share one preparation. DEF-03
3
Admit
Scheduler grants a slot, queues or refuses (429). Only then is the id handed out. ADM-01
4
Wire
Each node gets its wrapper; every child subscribes before sources start. ENG-01
5
Flow
Messages stream; policies, valves and clocks apply per node. ENG-02
6
7
Finish
Terminals complete, status set, slot released, end logged, run archived for restart. ENG-06
The node wrapper
Every node, whatever the plugin, is assembled into the same pipeline. Order matters: a pause happens before the plugin call, and a retry happens inside the clocks.1
Merge parents
Join rule decides what a failed parent means. ENG-03
2
Edge buffer
Optional bounded buffer that drops instead of pacing parents. ENG-01
3
Valve
Pause, step, skip, breakpoint. CTL-02
4
Plugin call
Attempt, element timeout, retry, onError. ENG-04
5
Clocks and status
Idle and deadline clocks, forward-only status. ENG-05
6
Broadcast
Output goes to every child; the slowest sets the pace. ENG-01
Package map
Base packagecom.infenia.yukta under core/src/main/java. Use this to find where a change belongs.
Feature catalog
Every capability of the core module, grouped by concern. Each card states what it does, how it behaves, how to configure it, where the code lives and what its limits are. Cards are headedGROUP-NN: name with a Status line; the groups are Workflow definitions (DEF), Execution engine (ENG), Admission and governance (ADM), Control plane (CTL), Observability (OBS), Durability and persistence (DUR), Scale-out and high availability (SCL), Security (SEC) and Extensibility (EXT).
Workflow definitions
Storing, versioning, validating and compiling a workflow before any plugin runs.DEF-01: Definition store and versions
Status: stable Keeps each workflow definition per session together with its newest 20 versions, so edits can be diffed, annotated and rolled back, and a run can be pinned to one version. Behaviour- Saving an identical definition adds no version. Each run records a content hash of the graph it used (
definitionVersion), carried by progress, history and the log. - A start payload with
yukta.versionruns that saved version and bypasses the prepared-workflow cache. WorkflowDiffreports a changed description, nodes added, removed or changed, and edges added or removed. A version may carry a note.- File layout:
<baseDir>/workflows/<session>/<workflow>/current.jsonandhistory/<version>.json; ids are URL-encoded; writes are atomic and fsynced.
yukta.session.store-type = in-memory | file | postgres
Surface: REST GET …/versions/diff?from=&to= (omit to for current), rollback endpoint, POST …/versions/note?version=¬e=. MCP list_workflow_versions, diff_workflow_versions, rollback_workflow, note_workflow_version. UI Versions panel.
Source: WorkflowDefinitionStore FileWorkflowDefinitionStore InMemoryWorkflowDefinitionStore DefinitionRevision WorkflowDiff WorkflowService
Limits: Newest 20 versions kept. PostgreSQL variant lives in DUR-04.
DEF-02: Validation
Status: stable Refuses a broken graph before any plugin runs, with an error naming the node and the option at fault. Behaviour- Rejects cycles, duplicate node ids, duplicate edges, an edge
sourcePortthe source plugin does not declare (erroris allowed on nodes withonError: ROUTE), and graphs over 500 nodes or 2000 edges. - Rejects malformed failure policies: unknown keys,
maxBackoffMsnot below the node timeout,retryOnandnoRetryOnsharing a class,deadlineMsnot aboveelementTimeoutMs. - Identifiers (session, workflow, node, plugin name, file path, config key) are shape-checked by Bean Validation annotations at the orchestrator boundary, e.g. no path separators in a session id.
WorkflowValidator NodePolicy TopologicalSortService. Package validation (SessionId, WorkflowId, NodeId, PluginName)
Limits: Size caps are constants, not settings.
DEF-03: Preparation and compile cache
Status: stable Turns a validated graph into a runnable form, once per definition content, and shares the work between concurrent starts. Behaviour- One assembler per node kind (trigger, processor, terminal) builds each node’s wrapper (ENG-02).
- Concurrent starts of a workflow that is not yet prepared share one preparation, keyed by definition content.
- A prepared workflow is cached only if the stored definition has not changed meanwhile, so an old compile never overwrites a newer edit. Applying a session configuration drops a stale cached entry.
- A processor with
engine.placementhas its plugin swapped for aRemoteProcessorhere (SCL-01).
WorkflowPreparator WorkflowCompiler PreparedWorkflowCache ExecutionResources PreparedWorkflow
Limits: Cost grows with graph size (hence the 500-node cap). The cache is in memory and simply rebuilt after a restart.
Execution engine
Running a prepared graph as streams of messages, with failure and time policy applied uniformly to every node.ENG-01: Streaming DAG and back-pressure
Status: stable Each edge is a Reactor stream. A slow consumer slows its producers, so memory stays bounded without queues between nodes. Behaviour- Every child subscribes before its source starts, so no early message is lost. A fan-out moves at the speed of its slowest child.
- Opt-out per node:
engine.edgeBuffer {size, overflow}puts a bounded buffer in front of a node. When the node is slower than its parents it drops messages (DROP_OLDESTdefault, orDROP_NEWEST) instead of pacing the fan-out. Drops are counted inyukta.edge.dropped. The default stays lossless. - Blocking plugins run on virtual threads so they do not hold the event loop.
engine.edgeBuffer
Source: StreamTopologyDecorator NodeStreams NodeBroadcast StreamAssemblyHelper
Limits: If a terminal on one branch fails and stops reading, siblings stall until the run ends (see ENG-06). Guarded by StreamTopologyDecoratorBackpressureTest (20-node chain, 10k messages, bounded memory).
ENG-02: Node wrapper pipeline
Status: stable Every node, whatever its plugin, is wrapped in the same pipeline so plugin authors write only business logic. Behaviour- Order: merge parents → edge buffer → valve → plugin call (attempt, timeout, retry, onError) → clocks and status → broadcast to children. See the steps in Architecture.
- The plugin gets a Reactor context holding the execution id, node id, a heartbeat
Runnable(WorkflowContext.HEARTBEAT_KEY), aPluginLogSinkand, for sub-workflows, a result collector. - With
retry,onErrororelementTimeoutMsthe plugin is called once per message and the message carries anattemptheader (1, 2, …).
NodePolicies NodeActivity ProcessorNodeAssemblerStrategy TriggerNodeAssemblerStrategy TerminalNodeAssemblerStrategy ExecutionContextBuilder
ENG-03: Join rules
Status: stable Decides what a node with several parents does when one parent fails. Behaviour
Configure: per node:
engine.trigger
Source: NodePolicy (JoinRule) ParentEdgeInfo UpstreamFailure
Limits: The rule is per node, not per edge.
ENG-04: Failure policy (engine config)
Status: stable
A reserved engine object in any node’s config sets retry, error routing and time limits. Plugins never see it.
Behaviour
- All-or-nothing output. With
retryoronErrorCONTINUE/ROUTE, what an attempt emitted before it failed is held back and dropped, so a retry or dead letter never lands on top of a partial result. One message’s output is therefore buffered until its attempt ends. - Malformed policies are rejected by validation (DEF-02) with node id and option.
NodePolicy (parsing) NodePolicies (behaviour) FailureLedger
Limits: Retry is per message, so use it with per-message processors. A retried plugin can see the same message twice: de-duplicate on the message id.
ENG-05: Clocks and timeouts
Status: stable Five independent clocks, all pause-aware, so a timeout means what you expect.
Behaviour
- Idle timeout is the node’s
timeoutSeconds, else the plugin’sgetDefaultTimeout(config). Built-ins:PROCESS_EXECUTORitstimeout(300 s) + 10 % (min 5 s);LLM_CALLevery attempt plus backoff;HTTP_REQUESTits timeout + 5 s; others 30 s.WAIT_FOR_SIGNALadds its wait + 30 s. - A timeout given as an expression is unknown at compile time: the node starts with the default and is re-timed when its first message arrives (
NodeTimeouts). Write numeric options as typed expressions, e.g.${payload.secs:int}. - Pausing freezes all clocks. A terminal’s idle timeout and
deadlineMsalso cover what it does after its last input. - Long silent work keeps its node alive by calling the heartbeat
Runnablefound underWorkflowContext.HEARTBEAT_KEYin the Reactor context.
timeoutSeconds, engine.elementTimeoutMs, engine.deadlineMs. Server: yukta.execution.queue-timeout. Per session: execution timeout.
Source: NodeActivity NodeTimeouts ExecutionResources ExecutionTimeoutException
Limits: Triggers have no input, so they keep the default idle timeout unless the plugin declares another.
ENG-06: Status model
Status: stable Run and node status only move forward, so a late or out-of-order event can never reopen a finished node. Behaviour- Order:
PENDING < QUEUED < RUNNING < any terminal status. A status that would step back is ignored, but its metadata is still merged. - A node becomes
RUNNING(and gets itsstartTime) at its first activity (NodeActivity.onFirstTouch: first input, or subscribe for a trigger), not when the graph is wired. - Nodes behind a failed node end
UPSTREAM_FAILED, so only the node that failed showsFAILURE. - The first terminal that fails ends the run at once and cancels the other branches (
CANCELLED). A caller that cancels ends the runCANCELLED. A second end of a run is ignored. - A processor skipped from the start can be unskipped later.
TaskStatus WorkflowState NodeActivity. Tests: WorkflowStateTransitionTest (thousands of random event orders)
All statuses: see Enumerations.
ENG-07: Sub-workflows
Status: stable ASUB_WORKFLOW node runs another stored workflow of the same session for each message, enabling composition and agent-style plans.
Behaviour
- The child is a normal run with its own execution id, tracked and audited like any other. Its terminal nodes feed a
ResultCollector; the node emits{executionId, results[]}. A failing child fails the node, so the parent’s failure policy applies. - The child’s start payload carries
yukta.depthandyukta.parentExecutionId. Nesting is limited to 5. - With
yukta.execution.max-concurrentset, a child runs on the slot its parent holds (ExecutionScheduler.borrowed()): children are not counted in limits or the running gauge, which prevents a parent waiting on children from starving them. - The child is subscribed inside the parent node, so cancelling the node cancels the child (
CANCELLED).
SubWorkflowRunner SubWorkflowGateway ExecutionScheduler
Limits: The parent-child link exists only in the child’s start payload and its RUN_STARTED entry; run history does not show it yet.
Admission and governance
Deciding whether, when and in what order a run may begin.ADM-01: Scheduler, limits and queue
Status: stable Grants a slot to each run, or queues or refuses it. The caller receives an execution id only after a slot is granted, so two requests racing for the last slot can never both succeed. Behaviour- Caps: total concurrent runs, per workflow, per session. All are off by default. A start that does not fit waits (
QUEUED) or is refused with HTTP 429, peron-limit. - Queue order: payload key
yukta.priority(number, higher first, default 0), then the session with the fewest running runs (oldest first on a tie), then arrival. One busy session cannot starve others. - The slot is taken after preparation and before the id is handed out. Restarting a finished run, restart-from-node and resume take their slot the same way. Restarting a run that is still going keeps the old behaviour: the old run holds its slot until it ends.
- Shutdown: nothing new is admitted, queued starts are refused, running ones get
shutdown-timeoutto finish. - Refusals are counted by reason in
yukta.executions.rejected{reason=shutdown|limit|queue_full|workflow_queue_full|queue_timeout|not_leader}.
yukta.execution.max-concurrent · …max-concurrent-per-workflow · …max-concurrent-per-session · …on-limit (QUEUE|REJECT) · …queue-capacity (100) · …queue-capacity-per-workflow · …queue-timeout · …shutdown-timeout (30s)
Surface: Status QUEUED in progress; gauges yukta.executions.running and yukta.executions.queued
Source: ExecutionScheduler SchedulerMetrics ExecutionRejectedException QueueTimeoutException WorkflowStarter
Limits: One scheduler in one JVM. Queue positions and slots are lost on a crash; the start was logged first, so the run is marked ABORTED on boot and can be recovered.
ADM-02: Idempotent start
Status: stable A client that retries a start after a network failure gets the first run back instead of starting a second. BehaviourPOST /workflow/startaccepts anIdempotency-Keyheader. A repeat with the same key, session and workflow within the TTL returns the first execution id. A failed start is not remembered.- With the file log the keys are written as
IDEMPOTENCY_KEYentries and reloaded on startup (the window restarts at the reload).
yukta.start.idempotency-ttl (10m) · yukta.start.idempotency-max-keys (100000)
Source: StartIdempotency WorkflowStarter
Limits: A key evicted before its TTL (cache full) lets a repeat start again.
ADM-03: Live limit and priority changes
Status: stable Operators change scheduler limits and a queued run’s priority without a restart. BehaviourchangeLimits(max, perWorkflow)is recorded in the log under run idscheduler; the newestretained-runsof them are kept. On startup the newest appliedSetLimits(max, perWorkflow)entry is re-applied, so the log wins over configuration from then on.changePriority(execution, priority)moves aQUEUEDstart and is recorded asSetPriority(n); rejected if the run is not waiting.- Both pass the reviewers and are audited with the actor. Over REST, limits need role ADMIN and priority OPERATOR.
/scheduler/limits, /executions/{id}/priority
Source: SetLimitsCommand SetPriorityCommand (in ExecutionControlCommand) ExecutionRecovery
Control plane
Changing a run that is already going, by people or by programs, through one ordered and audited path.CTL-01: Command pipeline
Status: stable All commands for one run enter one queue, are applied in the order sent, acknowledged to the caller, and recorded as applied or rejected with the actor. Behaviour- Commands are a sealed set (
ExecutionControlCommand): pause/resume workflow and node, step mode, step, stop node, stop workflow, skip, breakpoint, signal, restart, restart-from-node, set limits, set priority. One processor class per command. - The caller’s answer arrives only once the command has been applied (
CommandSequencer: FIFO per run; runs are independent). - Audit: every command for a known run is logged as
CONTROL_APPLIED(status = command name) orCONTROL_REJECTED(status =command: reason). Commands for unknown runs are not recorded. A restart that fails is audited as rejected. - The actor is carried in the Reactor context (
ControlActor.CONTEXT_KEY); REST uses the API key name, supervisors their id.
ControlBusGateway, used by REST, MCP and the CLI. Split internally into WorkflowStarter, NodeCommands, RestartCoordinator, ExecutionQueries (shared base ControlDispatch).
Source: CommandSequencer ExecutionControl DefaultControlBusGateway DirectiveDispatcher
Limits: A batch consumer: ample for people and supervisors, not a high-volume API.
CTL-02: Pause, step, breakpoint and skip
Status: stable A valve in front of every node implements fine-grained steering of a live run. Behaviour- Pause stops taking new messages; the one being processed finishes. All clocks freeze, so a pause never causes a timeout.
- Step lets exactly n messages through, then pauses.
- Breakpoint holds a node before the first element whose
payload,payload.some.pathormetadata.keyequals a given text. Ordered and audited like any command (SetBreakpointCommand). - Skip (processors only) passes elements through without calling the plugin from the next element on; one already in the plugin finishes. Unskip resumes. Other node kinds are rejected.
POST /api/executions/{id}/nodes/{node}/skip|retry|breakpoint (DELETE clears). MCP control_node SKIP|UNSKIP|RETRY|BREAKPOINT_CLEAR, set_node_breakpoint.
Source: ReactiveControlValve SkipNodeCommandProcessor SetBreakpointCommandProcessor StepNodeCommandProcessor
CTL-03: Stop modes
Status: stable Three strengths of stopping a run.
Surface: REST
POST /api/workflow/{session}/{workflow}/stop?mode=DRAIN|CANCEL|KILL
Source: StopMode (plugin-api) StopWorkflowCommandProcessor StopNodeCommandProcessor
CTL-04: Restart, restart-from-node and retry node
Status: stable Re-run a whole run, or one node and everything after it, replaying the recorded outputs of the nodes before it so earlier work is not repeated. Behaviour- Restart from node re-runs the node and its descendants and replays the recorded output of the nodes feeding them (
RestartPlan); independent branches are left alone. On a run still going, only parents that have finished (durable end marker) are replayed; a replayed output is marked finished under the new run. - Retry node does the same for a node that ended
FAILUREorTIMED_OUT, using the output its parents recorded. - The archive keeps recent finished runs restartable: graph, start payload and recorded outputs (
ExecutionArchive). A node that exceeds its caps makes a restart that needs it be refused rather than replayed partially, unless a spill directory is set (DUR-02). - Restart-from-node after a server restart is not supported (the archive is in memory); use resume for aborted runs.
yukta.restart.retained-executions (10) · …messages-per-node (200) · …bytes-per-node (4 MiB) · …spill-dir
Source: RestartPlan RestartCoordinator RestartFromNodeCommandProcessor ExecutionArchive ArchivedExecution
CTL-05: Signals and approval steps
Status: stable Hands a named signal (for exampleapprove) to the plugin of a node, through the same review, ordering and audit as other commands. This is how human-in-the-loop steps work.
Behaviour
- The plugin must answer
ACCEPTEDorQUEUED. A node of a live run that is not waiting yet keeps the signal for ten minutes (at most 100,QUEUED); a signal for a node that is not registered gives HTTP 409. - Waits of all plugins share
SignalWaits.shared(), so one list of open approvals (oldest first) serves REST, MCP and the workflow page. WAIT_FOR_SIGNALholds each message until its signal arrives (signal,timeoutSecondsdefault 3600,onTimeout,onReject) and tags it withsignal.name/decision/data.- An answer is recorded in the checkpoint store as a message of pseudo node
signal:<node>(a timeout is not), so a resumed run reuses it for the same message id instead of waiting again.
POST /api/executions/{id}/nodes/{node}/signal?name=&decision=&data=, GET /api/sessions/{s}/approvals. MCP signal_node, list_pending_approvals.
Source: SignalNodeCommandProcessor SignalWaits PendingApproval
Limits: The wait itself is in memory: after a restart a message produced again waits for a new signal, and a pending approval is invisible until its node waits again. Live waits are lost on HA takeover.
CTL-06: Reviewers
Status: stable A policy hook that can refuse any command, human or automated, before it runs. Behaviour- A
ControlReviewerbean implementsreview(ControlRequest). The first refusal inorder()wins; a reviewer that errors refuses. - The caller gets
ControlRejectedException(HTTP 403) and the log getsCONTROL_REJECTED. - Limit and priority changes pass reviewers too. The raw plugin-level node command refuses signal bodies (use the signal command).
ControlReviewer ControlRejectedException ControlRequest
CTL-07: Supervisors
Status: stable Programs that watch every run and steer it: a rule engine, an AI agent or your own code, inside or outside the JVM. Behaviour- A bean implements
ExecutionSupervisor:supervise(Flux<ExecutionEvent>)receives the global event feed and returnsExecutionControlCommands.SupervisorHostsubscribes each supervisor once and applies commands with the supervisor id as actor (logged asPauseWorkflowCommand [by <id>]). - Isolation: failures resubscribe with backoff; commands are limited to
yukta.supervisor.max-commands-per-second(20);yukta.supervisor.dry-run=truerecords without applying; a supervisor never sees the outcome of its own commands; events arrive through a bounded buffer (256) onboundedElastic, and a slow supervisor drops events (yukta.tracker.dropped{kind=supervisor}) instead of holding the tracker. - No JVM code needed: the feed is also an SSE stream,
GET /api/events/stream(filtersexecutionId,type); external agents steer through REST or MCP. GET /api/supervisorsreports kind (CONTROL_PLANE|SUPERVISOR), order, dry-run flag and command counts.- Reference implementation:
plugins/supervisors/rule-supervisor(JSON rules + SpEL).
yukta.supervisor.max-commands-per-second · yukta.supervisor.dry-run · yukta.supervisor.rules.file
Source: SupervisorHost ExecutionEventFeed SupervisorInfo ExecutionSupervisor
Limits: Event delivery is best effort. Commands are rate limited but a supervisor can still react to the state changes it causes.
CTL-08: Plugin-level control bus
Status: partial An older channel beside run commands, carrying heartbeats, statistics and raw signals to a node’s plugin. Behaviour- Keyed by workflow and node (not by run), it bypasses the reviewers and is not audited like run commands. Handlers:
ControlHeartbeatHandler,ControlStatisticsHandler,ControlSignalHandler.
ControlBusService DirectiveDispatcher ActivePluginRegistry
Limits: No code emits heartbeats or statistics through it today. Two sessions with equal workflow and node ids would share its state if one ever did (tracked as a latent item, see the bug backlog).
Observability
Turning what happens in a run into ordered state, live streams, results and metrics.OBS-01: Tracker and live progress
Status: stable Every change in a run becomes an event in one ordered queue; a single consumer turns them into progress snapshots, history and meters that REST, SSE and MCP all read. Behaviour- State and workflow events are never dropped: they go to an unbounded queue (gauge
yukta.tracker.queue.depth). Log lines are best effort (OBS-02). - One consumer applies events in batches (
TrackerEventApplier) to per-runWorkflowState; it resubscribes with capped backoff if its stream fails, untilshutdown(). - Every progress snapshot carries
definitionVersion(content hash of the graph). - History: the newest
status-history-max-per-run(200) snapshots per run forstatus-history-max-executions(1000) runs, always the newest; a late SSE client catches up from this cache (TTL 5 minutes). yukta.tracker.max-sessionsevicts the oldest session without a running run when more are held (off by default). Finished runs are cleaned up afteryukta.tracker.cleanup-ttl(10m).- Use
SinkEmit.offer, notSinks.Many#emitNext, for any bounded sink:emitNextterminates the sink on overflow.
yukta.tracker.max-sessions · yukta.tracker.cleanup-ttl · yukta.streaming.status-history-max-per-run · …status-history-max-executions · …status-history-ttl-minutes
Source: DefaultTaskTrackerService TrackerEventApplier WorkflowState ExecutionStreams ExecutionHistory DefaultStatusHistoryCache SinkEmit
Limits: Unbounded state queue means an event storm becomes memory growth, not data loss: alert on the depth gauge. Everything is in memory; after a restart the newest runs are restored from the log. With max-sessions set, each start scans every session while the limit is exceeded (O(sessions)): keep the value small.
OBS-02: Live logs
Status: stable Plugins write log lines while they run; they reach the feed, the stores and the logs endpoint as written, labelled stdout or stderr. Behaviour- A plugin writes through the
PluginLogSinkin the Reactor context. A node that did so (NodeActivity.hasLoggedLive) is not echoed again from its output message; other nodes log that message cut to 2,048 characters. - More than 10,000 waiting lines are dropped and counted in
yukta.tracker.dropped{type=log}(type=statecounts events lost because the tracker had ended). - A failed log write is logged and counted in
yukta.tracker.log.append.failed{type=node|run}; it never blocks a status update or the end of a run. - Engine log lines carry
executionId/nodeIdas structured key-values; there is no MDC.
yukta.logs.store (retention, capped at 24 h)
Surface: REST GET …/logs (SSE). MCP get_execution_logs with afterCursor (the previous nextCursor).
Source: PluginLogStore PluginLogStoreConfig PluginLogEntry WorkflowLogEvent
OBS-03: Execution results
Status: stable A separate channel, apart from logs, recording what reaches every terminal node of every run: payloads kept whole and typed. Behaviour- Terminals publish each input message and a
NODE_END; the orchestrator publishesEXECUTION_ENDand completes the stream. - The live stream is complete; only the replay buffer is bounded, and end events carry
truncatedwhen it dropped anything.
yukta.results.retained-events (1000 per run) · yukta.results.retention (1h after last use) · yukta.results.max-executions (200)
Surface: REST GET /api/sessions/{s}/executions/{e}/results (SSE, or JSON with Accept: application/json and afterCursor). MCP get_execution_result. CLI yukta result. UI progress panel.
Source: ExecutionResultService ResultChannel ResultEvent TerminalNodeAssemblerStrategy
Limits: In memory only: results are gone after a restart.
OBS-04: Metrics
Status: stable Micrometer meters for every governance and execution decision. The full list is in the Metrics reference. Behaviour- The tracker records node executions, durations and retries when a node reaches a terminal status, and run duration and queue wait per run.
- Scheduler gauges, rejection counts by reason, checkpoint bytes and errors, dropped events and edge drops complete the picture.
RunTelemetry SchedulerMetrics CheckpointMetrics MicrometerPluginMetricsReporter
OBS-05: Tracing
Status: optional Each run is one Micrometer observationyukta.run, the parent of one yukta.node observation per node (tags module, status).
Behaviour
- Observations become spans only when a Micrometer tracer is on the classpath.
- Node spans are opened only for a status the run state accepted, so a late
RUNNINGfor an ended node leaves no open span.
RunTelemetry
Durability and persistence
Making a run explainable and resumable after the process dies. Each backend has its own switch, so a deployment can be half durable: see the durability matrix.DUR-01: Execution log
Status: stable An append-only record of run starts (with the input payload), ends, control commands and idempotency keys. Behaviour- Event types:
RUN_STARTED,RUN_FINISHED,RUN_ABORTED,CONTROL_APPLIED,CONTROL_REJECTED,NODE_FINISHED(history only),RUN_RESUMED,IDEMPOTENCY_KEY. - Memory by default, trimmed like the file log (
LogRetention: newestretained-runs, never a run without an end, everycompact-afterevents). Withyukta.execution-log.dirit isexecutions.jsonlon disk. - File access runs on one background writer thread. Events are not fsynced unless
yukta.execution-log.fsync=true; a rewritten file (compaction, session config) is forced to disk before it replaces the old one. - Boot recovery (
ExecutionRecovery): runs that never finished are markedABORTED, the newestretained-runsare restored into history, and the log is rewritten without older ones.
yukta.execution-log.dir · …retained-runs (500) · …compact-after (10000) · …fsync (false)
Source: ExecutionLogStore InMemoryExecutionLogStore JsonlExecutionLogStore ExecutionLogEvent ExecutionRecovery
Limits: Without fsync, power loss can lose the last few events. A failure to write a run’s end no longer keeps the run from ending.
DUR-02: Checkpoints and durable end markers
Status: stable Each node’s output is recorded; when a node finishes, a small “done” marker is forced to disk with it before downstream nodes continue. A resume trusts only that marker. Behaviour- Memory store: capped by messages per node, bytes per node (strings and byte arrays by size, other payloads count 1 KiB) and retained executions. A node over a cap makes a restart that needs it refuse, unless a spill directory is set: the whole node output then moves to files (replayed as plain JSON), is deleted with the execution, and leftovers of an earlier process are removed at startup. Do not point it at a shared folder.
- File store (
yukta.restart.dir): one JSON line per message through one open channel per node untilmarkComplete; no caps.markCompleteforces the output, writes and forces<node>.doneand the directory. - PostgreSQL store: DUR-04.
yukta.checkpoint.errorscounts failed markers; that node then runs again on resume.
yukta.restart.dir · …spill-dir · …store-type · …retained-executions (10) · …messages-per-node (200) · …bytes-per-node (4194304)
Source: NodeCheckpointStore FileNodeCheckpointStore InMemoryNodeCheckpointStore CheckpointMetrics
Limits: There remains an instant between a plugin’s side effect and the marker: make plugins idempotent on the message id. A resume reads a node’s whole output into memory.
DUR-03: Resume and recovery
Status: stable Continue a run that a restart aborted: finished nodes replay their recorded output, the rest run again. Behaviour- Needs
yukta.execution-log.dir(holds the start payload) and a file or PostgreSQL checkpoint store. A run resumes once (RUN_RESUMED) and only if its definition is unchanged. The claim is kept in memory and dropped on failure or if the caller cancels. - Refused (HTTP 409
ConflictException) if the run was not aborted, was already resumed or its workflow changed. yukta.restart.auto-resume=trueresumes each interrupted run after any recovery, one after another.- Resume reads the log and checkpoint files on
boundedElastic.
yukta.restart.auto-resume (false)
Surface: REST POST /api/executions/{id}/recover. MCP control_workflow action RECOVER.
Source: ExecutionResumer ExecutionRecovery RestartPlan
DUR-04: PostgreSQL stores
Status: optional Sessions, definitions with versions, the execution log, node checkpoints, KV state and session memory can all live in PostgreSQL, chosen per store by a property. Behaviour- R2DBC at run time (
spring.r2dbc.url); Flyway migrations applied over JDBC (spring.flyway.url) frompersistence-postgres/src/main/resources/db/migration. Profilepostgres; build with-Pyukta.persistence=postgres. - Stores:
R2dbcSessionConfigStore,R2dbcWorkflowDefinitionStore(newest 20 versions, notes),R2dbcExecutionLogStore(one writer thread; trims toretained-runs; never drops a run without an end),R2dbcNodeCheckpointStore(one JSONB row per message plus a state row per node with the end marker; migration V3),R2dbcKeyValueStore(KV_STORE; V5),R2dbcSessionHistoryStore(SESSION_MEMORY; V6). Leader lease is V4. - The in-memory and file defaults step aside when the type is
postgres.
yukta.session.store-type=postgres · yukta.restart.store-type=postgres · yukta.kv.store-type=postgres · yukta.session-memory.store-type=postgres
Limits: Pool tuning, health checks and native-image metadata for R2DBC and Flyway are not finished.
DUR-05: Session configuration
Status: stable Stores each session’s configuration (workflows, execution timeout, tags) and its workflow definitions. Behaviour- Applying a session configuration stores the session and its workflows, and drops a cached prepared workflow if the stored definition has moved on.
- Default execution timeout (the run deadline of ENG-05) is 3600 s and can be set per session.
yukta.session.store-type (in-memory | file | postgres)
Source: SessionService SessionConfigStore FileSessionConfigStore InMemorySessionConfigStore SessionConfigProperties
Scale-out and high availability
Running node work on other machines and surviving the loss of the server that admits runs.SCL-01: Remote workers
Status: optional A processor node withengine.placement {pool} runs on a worker process of that pool. Retries, onError, timeouts, valves, checkpoints and status work exactly as for a local node.
Behaviour
- Registry: a registration or heartbeat renews a lease (
yukta.workers.lease, 15s); a worker that misses it is dropped. Workers connect out to the control plane over RSocket (routeworker.session), reconnecting with a growing pause. - Placement:
WorkflowPreparatorswaps the plugin for aRemoteProcessor; the local plugin still provides type, schema, timeouts and validation. The control plane picks the connected worker of the pool that allows the plugin type and has the most free slots; none means the attempt fails at once, retryably. - Protocol: one
requestChannelper attempt (node.run): START, the input messages, COMPLETE; back come the output, HEARTBEATs that restart the node’s idle clock, and COMPLETE or ERROR. Back-pressure and cancellation work end to end. Messages cross as plain JSON values. - Failure: a worker lost mid-attempt fails it with a
WorkflowExecutionExceptionnaming it; the engine’s retry can land on another worker. A stopping worker drains first (drainingon the lease frame), waiting up toyukta.workers.drain-timeout(30s). - Security: shared token; optional mutual TLS (control plane:
spring.rsocket.server.ssl.bundleandclient-auth=need; worker:yukta.workers.ssl-bundle, host name verified). yukta.workers.control-planemay list several comma-separated addresses; a standby refuses registration, so workers find the leader after a failover.
yukta.workers.token, spring.rsocket.server.port. Worker: yukta.workers.control-plane, …token, …id, …pools, …plugin-types, …capacity. Build with -Pyukta.workers=true.
Surface: REST GET /api/workers. Gauges yukta.workers, yukta.worker.slots.
Source: WorkerRegistry RemoteProcessor RemoteNodes RemoteNodeExecutor WorkerRegistration. Module workers. Design: remote workers plan.
Limits: Triggers, terminals and the stateful plugins (SESSION_MEMORY, KV_STORE, WAIT_FOR_SIGNAL, SUB_WORKFLOW) cannot be placed. Without the workers module a placed node fails to prepare. The control plane still sees every message (a data hop). Default transport is a shared token over plain TCP. Native-image worker registration is not verified.
SCL-02: Active/standby high availability
Status: optional Several servers share a PostgreSQL database; one holds a lease and admits runs, the others stand by and take over when it expires. Behaviour- A single server is always leader (
SingleServerLeader). Withyukta.ha.mode=postgresthe lease row (V4__leader_lease) has a fencing number that grows with every change of holder. It is renewed three times per lease; a graceful stop releases it. - A server counts itself leader only while its last renewal is younger than the lease, so one that cannot reach the database stops before another takes over.
- A standby refuses every start, restart and resume with
NotLeaderException(HTTP 503;yukta.executions.rejected{reason=not_leader}). - Takeover: startup recovery is skipped in this mode;
TakeoverRecoveryrunsExecutionRecovery.recover()(old leader’s unfinished runs becomeABORTED). Withyukta.restart.auto-resume=truethey are resumed from the checkpoints (use the PostgreSQL store or a shared directory).
yukta.ha.mode=postgres · yukta.ha.lease (15s) · yukta.restart.auto-resume
Surface: Gauge yukta.leader (1 on the leader)
Source: LeaderElection SingleServerLeader NotLeaderException TakeoverRecovery
Limits: Lost on takeover: live approval waits, queued starts, slots and the old tracker state. Clients must send requests to the leader (no proxy); use a load-balancer health check on the leader gauge.
Security
Who may call the API, what a workflow may read, and how much a node may run.SEC-01: API keys, roles and actors
Status: stable One key set guards both REST and MCP. Named keys carry a role and may be limited to sessions; the key name becomes the actor in the audit. Behaviour- Keys come from
yukta.api-key(one key that may do everything) andyukta.api-keys[](name,key,role,sessions). A key is plain text orsha256:<hex>; several keys at once allow rotation without downtime. A request presentsAuthorization: Bearer <key>orX-API-Key. - Roles:
VIEWERreads.OPERATORalso starts, controls and recovers runs (paths under/api/executions/and/api/workflow/, no..).ADMINeverything, including defining sessions and workflows (which can run processes, scripts and SQL) and changing scheduler limits. - The audit records the actor as
key:<name>;GET /api/control/auditlists who sent which command. - With no key set, the web layer enforces a Host and Origin check (
HostOriginFilter): unknownHostor foreignOriginon a non-GET gets 403, protecting a local server from CSRF and DNS rebinding.
yukta.api-key · yukta.api-keys · yukta.allowed-hosts · yukta.allowed-origins
Source: ApiKeys ApiRole RequestPaths. Enforcement in web and mcp.
SEC-02: Expressions, variables and secrets
Status: stable Resolves${...} placeholders in a node’s configuration just before it runs, from an allow-list, and hands out secrets by name.
Behaviour
- Sources:
${payload.x},${metadata.key}, typed values such as${payload.secs:int}(untyped stays text),${context.x}(plain values from the run’s Reactor context, outsideyukta.*). ${env.X}and${sys.x}read only:YUKTA_VAR_*variables,yukta.var.*properties, a short list of harmless names (HOME,USER,PATH,user.name,os.name, …), and names inyukta.variables.allowed/YUKTA_VARIABLES_ALLOWED. Names that look like secrets stay blocked even if listed.- Secrets are written
decrypted:name; the default provider readsYUKTA_SECRET_NAME, so values never appear in a stored definition. - Expressions (SpEL) may call only a short list of methods (
SafeMethodResolver). The evaluators live in plugin-api and the script plugin, not in core.
yukta.variables.allowed / YUKTA_VARIABLES_ALLOWED · YUKTA_VAR_* · YUKTA_SECRET_*
Source: VariableResolver SpelUtils SafeMethodResolver
Limits: A node whose configuration cannot be resolved fails that message like any plugin failure, so retry and error routing apply.
SEC-03: Execution safety limits
Status: stable Settings that bound what a node can do. Enforced by the plugins and web layer; documented here because they define core’s trust boundary.
GraalJS scripts may
Java.type only java.time.* and a few value classes and cannot reflect on Class. Scripts have no memory limit.
Extensibility
The seam between the engine and the work a node does.EXT-01: Plugin contract and registry
Status: stable Triggers produce messages, processors transform them, terminals consume them. Core discovers plugins as Spring beans and looks them up by type. BehaviourPluginRegistrymapsgetType()to the plugin and fails at startup on a duplicate type. A plugin may declare config fields, output ports, a default timeout (getDefaultTimeout(config)) and a failure mode.- The plugin runs inside the same JVM as the engine. Throwing
IllegalArgumentExceptionmeans “bad input” and is never retried. - Plugins can be implemented by
plugin-apicontracts without depending on core internals; the engine hides itsengineoptions from them.
PluginRegistry PluginCategory (plugin-api) ConfigField (plugin-api) FailureMode (plugin-api)
Limits: A heavy or untrusted plugin competes with the engine for CPU and memory until it is placed on a worker (SCL-01). Adding a command type is a plugin-api change (sealed interface).
EXT-02: Plugin logging API
Status: stable A small functional API through which plugins write structured log lines tied to their execution. BehaviourPluginLoggerFactoryis a@FunctionalInterfacewith one method,create(LoggerContext).LoggerContextis a record of execution id, session id, plugin id and plugin name; default methods for 3- and 4-argument creation keep older callers working.DefaultPluginLoggerFactoryextracts the values and builds aDefaultPluginLogger. Lines flow to live logs.
PluginLogger PluginLoggerFactory PluginLogStore
EXT-03: LLM and MCP budgets
Status: stable Per-run token, cost and call budgets that stop runaway agent loops. The nodes live in plugins; the shared ledger lives in plugin-api and the engine honours its abort signal. BehaviourLLM_CALLtakesbudget: {maxTokens, maxCost, maxCalls, hardStop, agent};MCP_TOOL_CALLtakes{maxCalls, agent, hardStop}. Use is counted per run across nodes (RunBudgets.shared(); newest 10000 runs); a budget with anagentis counted apart.- At the limit the next call fails like a failed call (so
failureModeandrouteByStatusapply). WithhardStop: trueit throwsRunAbortedException, whichNodePoliciesnever retries, drops or routes: the whole run ends. yukta.llm.rate-limit.<provider>(requests per minute) spaces calls to a provider across all nodes and runs.
llm.budget.exceeded · llm.tokens{provider,direction} · llm.cost{provider}
Limits: One call may overshoot the limit.
Reference
Lookup tables. Each row names the feature that owns it, so a change to a feature points straight at the rows to update.Configuration properties
“Module” notes say where the property is read when that is not core. Defaults are from the code; “off” means the feature is disabled until set.Metrics
All meters are Micrometer. Tags in braces.Enumerations
Node and run status (TaskStatus)
PENDING → QUEUED → RUNNING → one terminal status:
SUCCESS·FAILURE·ERRORSTOPPED·CANCELLED·TIMED_OUTSKIPPED·UPSTREAM_FAILED·ABORTED
ABORTED is given on boot to a run that never finished.
Policy values
- onError:
FAIL·CONTINUE·ROUTE - trigger (join):
ALL_SUCCESS·ALL_DONE·ONE_SUCCESS - edgeBuffer.overflow:
DROP_OLDEST·DROP_NEWEST - stop mode:
DRAIN·CANCEL·KILL - on-limit:
QUEUE·REJECT - API role:
VIEWER·OPERATOR·ADMIN - Log event:
RUN_STARTED·RUN_FINISHED·RUN_ABORTED·CONTROL_APPLIED·CONTROL_REJECTED·NODE_FINISHED·RUN_RESUMED·IDEMPOTENCY_KEY
HTTP outcomes produced by core
Client surface by capability
Defined inweb, mcp and cli; every entry calls the same ControlBusGateway or query service.
Guarantees and limits
What the module promises, and the honest edges of each promise.Durability matrix
What survives a restart, by deployment profile. “Memory” is the default; “File” sets the log and restart directories and file stores; “PostgreSQL” adds the PostgreSQL stores.Capacity ceilings
- One brain: admission, command ordering and run state are in one JVM (active/standby gives failover, not scale-out of admission).
- Graph size: 500 nodes, 2000 edges, sub-workflow depth 5.
- Memory growth: the state queue is unbounded; sessions are evicted only when
yukta.tracker.max-sessionsis set; the in-memory log and idempotency cache are bounded by their retention settings. - Large payloads are carried through memory by every node that touches them; there is no claim-check yet.
Maintaining this document
This page is built so that changing core means touching one or two small, predictable places. The code stays the source of truth; a script flags drift. The page lives atdocs/contributing/core-reference.mdx. The drift checker is core/scripts/check-core-reference.sh.
How the page is organised
What to edit when you change core
Card template:
Conventions
- IDs are permanent.
GROUP-NN, assigned in order and never renumbered. Other pages, plans and reviews link to them. - Card format. A card is a
###headingGROUP-NN: namefollowed by a**Status:**line. The status is one ofstable,optional,partial,planned. - Classes named in backticks in CamelCase (for example
ExecutionScheduler) are checked against the source tree. Use the simple class name. Names that are not in the repository (JDK or library types) are listed in the script’s allow-list. Plain backticks are also used for settings, values, endpoints and anything not a class. - Settings are written in full the first time (
yukta.execution.queue-timeout) and may be abbreviated with…inside the same card. - Defaults go in parentheses after the setting, or in the reference table; state “off” for disabled-by-default.
- One fact, one place. Describe behaviour in the card and link to it elsewhere with an in-page anchor link (the heading slug, for example
#eng-04--failure-policy-engine-config); do not restate it. - MDX safety. Put curly braces and
<in backticks, escape\|inside table cells, and use{/* */}for comments. - No SPDX header in
.mdxfiles; they are covered by thedocs/**/*.mdxannotation inREUSE.toml.
Checking for drift
Run from anywhere inside the repository:- a backticked class name cites a class that no longer exists in
core,plugin-api,workers,persistence-postgres,plugins,messaging,webormcp; - a
yukta.*setting or meter string appears incore/src/mainbut nowhere on this page; - a feature has a missing or duplicate ID, an unknown status, or a retired ID is reused;
- an in-page link (
](#…)) points at a heading that does not exist.
Pull-request checklist for core changes
- Behaviour, default or limit changed: card and reference row updated
- New setting or meter: row added to the reference tables
- New class cited: the backticked name is the real class name
- Feature finished or removed: status or card updated, retired ID recorded
-
core/scripts/check-core-reference.shpasses and--stampwas run -
core/CLAUDE.mdupdated if a build command or key location changed
Retired feature IDs
None yet. Add one line per retired feature, for example “ENG-09 removed in 2026-11 (replaced by ENG-04)”.Glossary
The wider project glossary is at /reference/glossary.