Primer
Yukta runs workflows: graphs of steps such as ‘receive a webhook, classify with a model, call an API, notify someone’. The engine (core) should not know how to call every API or model, and the people who write those steps should not need to know how the engine schedules, retries or stops things. plugin-api is the agreement between the two sides. It is a small Java library of interfaces and helpers that core, the REST layer and the built-in plugins all compile against. A new plugin is a class that implements one interface; the engine discovers it and calls it.Where plugin-api sits
The vocabulary in eight cards
One plugin, from startup to stop
A processor plugin called HTTP_REQUEST used in one workflow. Click a stage to see what happens, why, and what can go wrong.Registered
Validated
Prepared
Runs
Failure routed
Control signal
Stopped
Shutdown?
plugin-api in plain words
Describe
Register
Check
Run
Fail
Steer
Remember
Components
Plugin contract
Plugin + 3 kinds Defines what every workflow node plugin must provide and what it may override. The base interface Plugin has two required members (a unique type string and a category); each kind adds one required stream method.- Three kinds extend the base. A trigger implements start(config) and returns a Flux of messages (a Flux is a stream of zero or more items). A processor implements process(input, config) and returns a Flux. A terminal implements consume(input, config) and returns a Mono (a result that completes once) that finishes when everything was consumed. The category (TRIGGER, PROCESSOR, TERMINAL) is filled in by the kind, so an author does not write it.
- Everything else has a default. Timeout 30 seconds, buffer size 1024, one output port named ‘default’, not blocking, validation passes, initialize/prepare/shutdown do nothing, no control-signal handling. A plugin that overrides nothing but the two required members is valid. Example: the three-line uppercase processor in the primer.
- Two defaults take the node’s config: getOutputPorts(config) and getDefaultTimeout(config). They exist because the answer can depend on settings. A router whose ports come from its config returns different ports per node; an LLM call with retries returns a longer timeout than a plain HTTP call, so the engine does not cut it off at 30 seconds.
- isBlocking() tells core how to run the plugin. If true, core subscribes it on a virtual thread (a cheap thread that can wait on I/O without hogging a CPU thread). If false (default), the plugin runs on the reactive threads and must never block. The plugin does not choose the thread itself.
- Core keeps one shared instance per type. At startup the registry builds a map from getType() to the object and throws if two plugins share a type, naming both classes, so a conflict is a startup failure, not a wrong node at runtime. Because one object serves every node, per-node settings must come from the config argument on each call.
- Validation has two hooks. validateConfig(config) checks settings alone. validateInContext(context, config) also sees the node’s incoming and outgoing edges. Both return a Mono that errors on failure. initialize and prepare run once per node before a run, on the same shared object, so they must be repeatable and only warm caches.
- Guarantees. The compiler guarantees that a plugin supplies a type, a category and its stream method. Core guarantees unique types at startup. Statelessness of the shared instance is a rule in the Javadoc, not something the code checks.
- State. None in this module. Interfaces only. Whatever a plugin keeps in its own fields is lost on restart and shared across all nodes of its type.
- Ceiling. The contract has 18 members, which is a lot for a plugin author to learn, though only 3 are required. Adding a new default method is safe; adding an abstract one breaks every plugin.
- On failure. A plugin that returns a Flux that errors fails its node; engine policy then applies. shutdown(config) exists but core never calls it, and isComputationallyHeavy and suppressOptimizationHint are not read by core.
getType(), getCategory() (filled by the kind), getDefaultTimeout(config), default 30 s, getDefaultBufferSize(), default 1024, isBlocking(), default false
Metadata & schema
describe, ConfigField, UiDesign Lets a plugin describe itself and its settings without being run, so UIs, REST and agents can list and configure plugins.- describe() returns a PluginDescriptor with three parts: a description, a usage pattern (free text, such as a sample config), and an optional UiDesign. The default is an empty descriptor, so describing is optional.
- UiDesign carries a small block of HTML plus a width and height. The UI draws it as the node’s picture. Many plugins repeat nearly the same HTML (colour, icon, label), which is a known duplication in the plugins module.
- getConfigSchema() returns a list of ConfigField entries: name, type, required flag, default, allowed options and a one-line description. Types are STRING, NUMBER, BOOLEAN, ENUM, LIST, MAP, EXPRESSION and SECRET. EXPRESSION means ‘edit as a SpEL or template text in a code-style box’; SECRET means ‘the value should be a decrypted:<key> reference, not the credential itself’.
- If the schema is empty (the default) the UI falls back to a raw JSON editor and the usage pattern text. So a plugin with no schema still works, but the form is poorer.
- Three factory methods keep declarations short: required(name, type, description), optional(name, type, default, description) and choice(name, required, default, description, options…) which produces an ENUM.
- Guarantees. Field options are copied into an immutable list. Descriptor and schema are read-only data with safe empty defaults.
- State. None. Read by web (the plugin list endpoint) and by the graph service that describes workflows.
- Ceiling. The schema is flat: no nested objects, no conditional fields (show field B only if field A is X), and no validation rules beyond required and the options list.
- On failure. The schema is not checked against what the plugin really reads. If the code reads a key the schema omits, the UI form hides a real setting; the module’s CLAUDE.md asks authors to keep names in step by hand.
ConfigField.Type: STRING NUMBER BOOLEAN ENUM LIST MAP EXPRESSION SECRET, PluginCategory: TRIGGER PROCESSOR TERMINAL
Graph context, liveness
WorkflowContext, heartbeat Gives a plugin two pieces of engine information: the edges around its node during validation, and a way to report liveness during a long run.- WorkflowContext holds the node id, the list of outgoing edges and the list of incoming edges. Each edge records source, target and the optional source port. Core builds it for each node during validation and passes it to validateInContext. The constructor copies the lists, so later changes by the caller are not seen.
- A plugin can use the context to enforce a graph rule. Example: a JOIN-like plugin rejects a workflow in which it has fewer than two incoming edges, with a clear message, before anything runs.
- The heartbeat is a Reactor context entry under the key ‘yukta.heartbeat’. Its value is a Runnable. A long-running plugin reads it with Mono.deferContextual and runs it now and then. Each call restarts the node’s idle timeout. Example: a 10-minute process step that prints progress every minute calls the heartbeat each time so it is not declared dead after 30 seconds of silence.
- The key is absent outside a workflow run (for instance in a unit test), so plugins must handle a missing key. Core also puts execution, node, workflow and session ids in the same Reactor context under its own keys; those are core’s, not defined here.
- Guarantees. The context is immutable once built. The heartbeat only restarts the idle clock; it does not extend a hard overall limit.
- State. None. The heartbeat is per subscription and lives only for the run.
- Ceiling. Only direct neighbours are visible, not the whole graph, so a plugin cannot check a rule like ‘there is a terminal somewhere downstream’.
- On failure. If a plugin never beats and works silently longer than its idle timeout, the engine times the node out as if it were dead.
WorkflowContext.HEARTBEAT_KEY = yukta.heartbeat
Expression sandbox
SpelUtils (SpEL, cached) Evaluates the expressions that users write in workflow settings (a filter condition, a branch rule, a mapping) against a message, inside a restricted context.- It uses Spring’s SpEL (Spring Expression Language, a small expression syntax) with a SimpleEvaluationContext, the restricted variant. Allowed: read properties, read map keys, call a short list of safe methods (such as name.startsWith(‘x’); about 40 names plus everything in java.time), and use variables passed in (#n). Not allowed: static access T(java.lang.System), constructors (new java.io.File), and bean references (@someBean). Tests assert each of these is rejected, including ‘a’.getClass().forName(…).
- Why this matters: expressions come from workflow definitions, which come from users over REST or MCP. The original design used the full evaluation context, which let a workflow author run any Java code on the server. The restricted context closed that hole (an old review finding, now fixed).
- Parsed expressions are cached in a least-recently-used map of at most 1000 entries, so the same expression text is parsed once. The cache is guarded by one lock. preParse(text) lets a plugin parse during prepare; blank text is ignored.
- Two entry points: evaluateSync(expression, root[, variables]) returns the value directly, and evaluate(…) wraps it in a Mono that runs when subscribed. A fresh evaluation context is built per call, so variables set by one call never leak into the next (a test checks this).
- The result type is chosen by the caller through generics, with no check. A caller that expects a Boolean and gets a String fails with a ClassCastException at the call site.
- Guarantees. No static calls, no object construction, no bean access, no property writes; variables never leak between calls; the parse cache never exceeds 1000 entries.
- State. A process-wide static cache of parsed expressions, lost on restart (it only saves parsing time).
- Ceiling. One lock around the cache, and parsing happens while holding it. There is no evaluation timeout: an expression that loops through a large collection runs until it finishes.
- On failure. A bad expression throws a SpEL exception; nothing is cached for it. Callers should surface it as a config error in validateConfig by parsing early.
MAX_CACHED = 1000 (not configurable)
Failure & config helpers
PluginGuard, FailureMode Shared building blocks so every plugin handles failure, metrics and config checking the same way: PluginGuard, FailureMode, PluginMetricsReporter, ConfigValidation, ResultCollector, MessageMapper and three exception types.- FailureMode has two values. ERROR (default) lets the exception fail the node. CONTINUE turns the failure into a message. FailureMode.from(value) parses a config text case-insensitively; null gives ERROR; anything else throws an IllegalArgumentException that lists the valid values.
- PluginGuard.guard(operation, input, mode, reporter, metric) wraps one message’s work. On success it counts OK. On failure it counts ERROR and then either re-raises (ERROR mode) or returns the original input with failure details attached (the plugin’s node id, the exception class name and its message) and its source port set to ‘error’ (CONTINUE mode). A downstream edge from the ‘error’ port can then deal with it. PluginGuard.counted(flux, …) only counts, for plugins that handle failures themselves.
- PluginMetricsReporter is a one-method interface: increment(nodeId, metric, tag). The default NOOP drops everything. Core supplies a real one; the tag is OK or ERROR here, and plugin-specific words such as MATCH or DISCARD elsewhere.
- ConfigValidation gives fail-fast readers: positiveLong, positiveInt (also checks the int range) and requiredString. They reject zero, negatives, fractions, infinity and values too large to store, so a config of 1e30 cannot silently become a wrong number.
- ResultCollector is a thread-safe bucket used by sub-workflow execution to gather final messages from terminal nodes. MessageMapper is a two-method interface for converting a message payload to a domain object and back while keeping headers. The exceptions (WorkflowExecutionException, plus FilterEvaluationException and NoMatchingBranchException) give plugins a technology-neutral error type.
- Guarantees. Using guard gives the same two failure behaviours and the same metric counts in every plugin that uses it. Config readers never return an out-of-range number.
- State. None, except ResultCollector, which holds messages in memory for the life of one sub-workflow call.
- Ceiling. Adoption is by choice. A plugin that ignores these helpers is still valid; nothing forces consistency. The error port name ‘error’ is a string constant, not a type.
- On failure. In CONTINUE mode the failure message leaves through port ‘error’. If no edge uses that port, the message is not delivered anywhere useful, so a workflow author must wire it.
PluginGuard.ERROR_PORT = error, FailureMode ERROR | CONTINUE (parsed from config text), ConfigValidation: positiveLong, positiveInt, requiredString
Control commands
sealed set of 16 commands Defines every instruction an operator, API caller or supervisor can send to a live run, as a closed list of typed records.- ExecutionControlCommand is a sealed interface: the list of allowed types is fixed in the declaration, so a switch over it needs no default branch and adding a command makes every switch fail to compile until it handles the new one. Every command names its target run through executionId().
- Run-level: PauseWorkflow, ResumeWorkflow, StopWorkflow (with a reason and a StopMode), Restart and RestartFromNode (both carry a pre-generated id for the new run, so the caller can poll it at once). Node-level: PauseNode, ResumeNode, StopNode (immediate or drain, with reason), SkipNode (true or false), EnableStepMode, DisableStepMode, StepNode, and SetBreakpoint. That is 13 in total.
- StopMode has three values. DRAIN (default) stops the triggers and lets work in flight finish, cancelling it if it does not finish before the drain deadline. CANCEL cancels every node’s subscription at once so plugins can clean up. KILL cancels and also removes the execution, so later commands for it are refused.
- SetBreakpoint takes a field and a text. The field must be ‘payload’, ‘payload.some.path’ or ‘metadata.key’, and the node holds before the first element whose value’s text equals the given text. The record’s constructor rejects any other field, and rejects a field without a value. A null field clears the breakpoint. The condition is declarative on purpose: it can arrive from a remote caller, so it must not be code.
- ControlActor.CONTEXT_KEY (‘yukta.control.actor’) names the Reactor context entry that carries who sent a command. It is empty for API callers and holds the supervisor’s id for supervisor commands, which is how the audit log writes ‘PauseWorkflowCommand [by my-supervisor]’.
- SkipNode on a running run only works for processors; core rejects other node types. It takes effect from the next element, and an element already inside the plugin finishes normally.
- Guarantees. The set is closed and checked by the compiler; SetBreakpoint cannot be built with an invalid field; StopWorkflow never has a null mode.
- State. None here. Commands are values. Core orders them per run, applies them and logs them.
- Ceiling. Adding a command is a breaking change for every exhaustive switch in the repo, by design. The list is fixed at compile time, so a third-party plugin cannot define its own command type.
- On failure. A command for an unknown run is rejected by core with a not-found error. Commands whose processor throws are recorded as rejected.
StopMode DRAIN | CANCEL | KILL, ControlActor.CONTEXT_KEY = yukta.control.actor
Command processing
ControlSignalProcessor The extension point that turns a command into an action: the interface a processor implements, the priority rule, and the directive type a processor may return.- A ControlSignalProcessor answers canProcess(command) with true or false and implements process(command), which returns a Mono of a WorkflowDirective, an empty Mono, or an error. Spring beans of this type are collected by core’s directive dispatcher.
- When several processors accept a command, the one with the highest getPriority() handles it and the rest are skipped. The default is 0; the built-in processors use values from 10 to 20. To override a built-in, a custom processor must return more than that built-in’s priority (above 20 beats all). Example: a processor with priority 30 that turns Restart into RestartFromNode for specific nodes.
- A processor can act in two ways. It can do the work itself and return an empty Mono (this is what most built-ins do for pause, skip and so on). Or it can return a directive for the dispatcher to apply. The only directive today is Stop(reason), so only ‘stop this run’ can be handed back; everything else is done inside the processor.
- Ordering and acknowledgement are core’s job, not this interface’s: core runs the commands of one run strictly one after another, tells the caller when the command was applied or rejected, and records it.
- Guarantees. Exactly one processor handles each command (the highest priority among those that accept). The directive set is sealed, so the dispatcher’s switch is exhaustive.
- State. None.
- Ceiling. No chain: a second processor cannot add audit around a first. If two accept at equal priority, the winner is the first one in Spring’s bean list; the contract does not define that.
- On failure. If no processor accepts a command, core fails the call with ‘No processor registered for command’. A processor error becomes a rejection recorded in the log.
getPriority(), default 0; built-ins 10-20, WorkflowDirective.Stop(reason)
Supervisor SPI
supervise, review, decide Lets automated decision makers (rule engines, models, custom code) watch execution events, emit commands, and approve or refuse any command.- ExecutionSupervisor has three members: id() (unique, used in audit and metrics), supervise(events) which receives the event stream and returns commands, and review(request) which returns Allow or Deny for any command. The default review allows everything and the default order() is 0.
- The events are four types: NodeStatusChanged, WorkflowStatusChanged, ControlOutcome (a command was applied or rejected, with who sent it) and LogLine. The feed is best effort by design: a supervisor that cannot keep up misses events instead of slowing the runs. Core counts drops in yukta.tracker.dropped.
- Review is a veto in a line. Core asks the supervisors in order() sequence, lowest first, and the first denial refuses the command. If a supervisor’s review throws an error, that counts as a denial (‘review failed’). The caller gets a rejection (HTTP 403) and the log records it. Example: a supervisor denies StopWorkflow for runs tagged ‘billing’ unless sent by a human.
- ControlPlanePlugin is the simple form: implement decide(event) which returns a ControlDecision (a list of commands, a confidence from 0 to 1, and a source RULE or MODEL). Its supervise handles events one at a time in order, treats an error from decide as no opinion, and emits the commands. Core reads only the commands; confidence and source are for the plugins that combine decisions (a rules-first plugin consults rules before a model).
- Core isolates supervisors: it subscribes each once at startup, resubscribes after a failure with backoff from 1 second up to 1 minute, rate limits commands (20 per second per supervisor by default), supports dry-run (commands are only recorded), and does not deliver a supervisor’s own command outcomes back to it. These limits live in core’s SupervisorHost, not here.
- ControlRequest carries the command and the actor (a supervisor id, or null for an API caller). Decision is a record of allowed and a reason, with Decision.ALLOW and Decision.deny(reason).
- Guarantees. Every command, human or automated, can be vetoed; a failing reviewer denies rather than allows; a supervisor failure never stops a run.
- State. None here. Core holds subscriptions and counters in memory.
- Ceiling. decide() handles one event at a time per plugin, so the speed of one slow decision (a model call) caps the plugin’s event rate. The event feed has no replay.
- On failure. review() has no timeout in the contract or in core’s reviewer; a review that never completes blocks later commands for that run. A decide() that never completes stalls all later events for that plugin. The Javadoc asks authors to bound both.
yukta.supervisor.max-commands-per-second (core, default 20), yukta.supervisor.dry-run (core, default false), GET /api/supervisors (core)
Store SPIs
NodeCheckpointStore, Secret Two storage interfaces that core implements: NodeCheckpointStore (each node’s output per run) and SecretProvider (credential lookup).- NodeCheckpointStore.save(executionId, nodeId, message) records the last message a node emitted; get returns it. When a run is restarted from a node, core reads the checkpoint of that node’s direct parents and replays them as synthetic input, so earlier nodes do not run again.
- Three methods have defaults so older stores keep compiling. getAll returns every message a node produced (the default knows only the last). isComplete says whether nothing was dropped (a store with a size cap reports false once it has dropped, so a restart never replays half an output as if whole). markComplete and isFinished record and report that a node ended normally and all its output is safe; a durable store must make the marker survive a crash before its Mono completes.
- The engine calls markComplete once when a node’s stream completes (never for a failure or a stop) and waits for it before passing the completion on. A failure writing the marker must not fail the run: the node is simply reported as not finished.
- clear(executionId) releases a run’s checkpoints and is called when a run finishes. The implementations must be thread-safe.
- SecretProvider has one method, getSecret(key), returning a Mono of the secret text. A config value written as decrypted:<key> is resolved through it. Core’s default reads environment variables with a fixed prefix YUKTA_SECRET_, so a workflow cannot read arbitrary environment variables. A deployment can supply its own bean to use a vault.
- Guarantees. isComplete and isFinished let a restart avoid replaying partial or unfinished output. Durable stores must persist before markComplete’s Mono completes.
- State. None in this module. Core ships memory and file implementations; what survives a restart depends on which one is configured.
- Ceiling. clear, isComplete and isFinished are plain synchronous calls, so a store backed by a remote database would block a thread on each. save, get and getAll are reactive.
- On failure. Missing checkpoints give an empty result. A store failure on markComplete is swallowed by design.
yukta.secret env prefix YUKTA_SECRET_ (core default provider)
Build & test setup
dependencies, tests Declares what the library depends on and how it is tested, since every other module compiles against it.- Dependencies: reactor-core, the jakarta annotation API and the messaging module are exposed to everything that uses plugin-api (api scope). spring-expression is internal (implementation scope), so plugins do not get Spring’s expression classes for free.
- Core exposes plugin-api to its own users with api scope, so web, mcp and boot get it transitively.
- Tests are plain JUnit 5 with Reactor’s StepVerifier. They cover the defaults of the contract, PluginGuard, FailureMode, ConfigField, the sandbox (rejected expressions, cache bound), the checkpoint defaults, the command records, the supervisor defaults and the control-plane plugin.
- The java-test-fixtures plugin is applied but the module has no fixtures source, so a plugin author gets no ready-made test kit (a fake Message factory, a harness that runs a plugin as the engine would).
- pmdTest is switched off with a note until test-source PMD violations are fixed. Checkstyle and SpotBugs still apply.
- Guarantees. The tests pin the sandbox and the documented defaults, so a change in them fails the build.
- State. None.
- Ceiling. No version marker exists on the contract, so a plugin built against an older release cannot declare what it needs and can only fail at runtime.
- On failure. A change to a default method’s behaviour can silently change every plugin that relied on it; only tests in this module guard that.
./gradlew :plugin-api:test
Scenarios
Life of a plugin
A plugin from registration to the end of a run, showing which method is called when, and which one is never called.Registered at startup
Checked before the run
Initialised and prepared
Runs with engine policy
Gets a control signal
Never told to shut down
A bad config is caught early
How shared helpers and the validation hooks catch a mistake before anything runs.A user saves bad settings
The plugin validates
A graph rule is checked
The UI form helps earlier
Why not at run time
A plugin call fails
What a plugin failure becomes: counted, raised or routed to the error port, and the quiet failure of a long job that looks dead.The call throws
The guard counts it
ERROR or CONTINUE
The error port is wired or not
Silent long work
An operator stops a run
A stop command travelling through review, processor and directive, and what it leaves behind in the event feed.A command is built
Supervisors review it
A processor handles it
The directive is applied
The outcome is an event
Restart reads checkpoints
A supervisor pauses a run
How an automated watcher reads events, issues commands inside limits and can still miss events.Subscribed once
An event arrives
A decision is made
Core applies it with limits
Other supervisors may veto
Slow supervisor, missed events
A hung reviewer (edge case)
An edge case: one reviewer that never answers, and how it can block even a human stop.A reviewer waits on a call
The first command waits
A human presses stop
Other runs are fine
Ways out
Capabilities
Eighteen things the contract lets a plugin or a supervisor do, grouped by where they are used.Plugin contract
Describing
Graph and liveness
Helpers
Control
Supervision
Storage
Guarantees and limits
plugin-api is an interface, so most guarantees are promises that other modules must keep. Each row says who keeps the promise: the compiler, a test, core at runtime, or only the reader’s discipline.What is enforced and what is only convention
Defaults a plugin inherits
Semantics of control and events
What survives a restart
This module holds no state, so the question is about what its interfaces let others keep.Reference
Plugin contract: method, default and caller
Every member of the Plugin interface. The last column says who calls it in core, checked in the current code.The 13 control commands
All carry executionId. Extra fields are listed.Write a plugin: checklist
The smallest correct plugin and the rules that follow from the contract.Architecture decisions
Plugins are Reactor functions over streams, in three kinds
Plugins are Reactor functions over streams, in three kinds
Defaults for almost every hook
Defaults for almost every hook
Control commands are a closed sealed set of records
Control commands are a closed sealed set of records
Highest priority wins; one processor per command
Highest priority wins; one processor per command
Supervisors are plugins with an event feed and a veto
Supervisors are plugins with an event feed and a veto
Expressions run in a restricted SpEL context behind one helper
Expressions run in a restricted SpEL context behind one helper
Storage behind interfaces with evolvable defaults
Storage behind interfaces with evolvable defaults