Runtime Topology: YU15-eod-risk-extract
The inherited cluster tier is unchanged in shape: three Aeron Cluster members behind a stateless gateway tier. This state adds a producer alongside them, plus the two dependencies it needs; a NATS broker with JetStream and a database holding the published closing prices.
Entrypointsβ
| Entrypoint | Process | Purpose |
|---|---|---|
ClusterNodeMain | order-matcher-cluster StatefulSet | Hosts the clustered service. Serves /health and /ready; readiness now reports the consensus-log position. |
ClusterGatewayMain | cluster-gateway Deployment | Inherited unchanged: REST and FIX ingress, forwarded through the cluster client. |
RiskExtractMain | risk-extract Deployment | The EOD producer. Same image, different main. |
nats-server --jetstream | nats Deployment | Trigger stream, cut transport, announcement transport. |
mariadb | eod-price-db Deployment | The state's own schema, from the database-init-sql ConfigMap. Read-only from the producer's view; written by trade-processor. |
trade-processor | trade-processor Deployment | Consumes the ADR-048 trade bridge's /trades and persists Trade + Position; produces and publishes the closing-price version. |
price-publisher | price-publisher Deployment | Market data for equities and the listed option chain. Options are derived from their underlying's tick, never walked independently. |
position-service | position-service Deployment | Marks every account against the published version and emits eod.pnl.done; the extract's real trigger. |
Componentsβ
| Component | Role | State |
|---|---|---|
| Sequenced marker | Names the consensus sequence the extract is cut at | None; mutates nothing |
RiskExtractCut | Renders the canonical cut on every member | Pure function of replicated state |
RiskExtractCutPublisher | Leader-side cut β NATS bridge | Bounded SPSC queue, daemon thread |
RiskExtractCsv | Renders the delivered fixture | Pure function of cut + reference data |
RiskExtractMain | Orchestrates one EOD batch | Durable consumer cursor only |
RiskExtractGcsSink | Immutable object delivery | None |
| Object sink | Holds fixtures and their cuts | Write-once, keyed by the stamp |
Networkingβ
| Path | Transport | Notes |
|---|---|---|
| producer β members | Aeron Cluster client, UDP 21800β22200 | The producer's pod label must appear in the cluster NetworkPolicy ingress allowlist; without it the client silently cannot reach any member. |
| leader β producer | NATS risk.extract.cut | One message per extract, self-counting rows. |
| producer β NATS | TCP 4222 | Trigger consumption and delivery announcement. |
| producer β price DB | TCP 3306 | Read-only, one query per extract. |
| producer β object sink | Filesystem, or HTTPS to storage.googleapis.com | Write-once either way. |
Startup / Health Orderβ
- NATS becomes ready; the producer creates the EOD stream if position-service has not already.
- The price database becomes ready. The producer does not connect to it until a batch fires.
- Cluster members form and elect; readiness gates each member on its consensus-log position relative to its peers.
- The gateway connects;
/readyturns 200 once its cluster session is live. - The producer connects to NATS (retrying until it is there) and subscribes to its durable trigger. It holds no cluster session while idle; one is opened per batch.
Ordering between the producer and everything else does not matter: it retries NATS, ensures the stream idempotently, and connects to the cluster and the database only when a batch actually fires.
Degraded Behaviorβ
| Condition | Behavior |
|---|---|
| NATS unavailable at producer start | Producer retries indefinitely, logging each attempt. It does not exit; a batch producer that dies on a cold dependency simply is not there when the batch fires. |
| Cluster unreachable when a batch fires | The marker ack times out, the extract fails, the trigger stays unacked, and JetStream redelivers. Nothing partial is written or announced. |
RISK_EXTRACT_NATS_URL unset on members | The marker still sequences and every member still renders and hashes the cut; nothing is published, so the producer times out waiting for it and reports the missing configuration in the failure. |
| Cut lost or truncated in flight | A lost message is a timeout; a truncated one fails the declared row-count check. Neither can pass as a complete portfolio. |
| Trading occurs during the build | The witness marker lands beyond N + 1, the producer refuses to emit, and the trigger is redelivered. |
A security has neither a published close nor a trade at N | The whole extract aborts. No zero-filled or omitted row is ever delivered. |
| An account holds a position but has no counterparty mapping | The whole extract aborts. |
| Price database unavailable | The extract fails and is redelivered. With RISK_EXTRACT_JDBC_URL deliberately unset, every row instead marks from the cluster's last trade and says so per row. |
| Object already exists at the key | The write is refused; on a filesystem by CREATE_NEW, on GCS by if-generation-match: 0. A redelivered trigger cannot replace a fixture already scored against. |
| A contract's underlying is absent from the feed | The contract is skipped rather than quoted off a fabricated price. It is then unpriced at EOD, and the fail-safe halts any account holding it; loudly, rather than marking it wrong. |
An option fill reaches trade-processor | Persists with its full OCC symbol. Before the widened columns this failed with Data too long for column 'security' and the fill was lost from SQL while remaining booked in the cluster. |
| A member restarts during the EOD window | It replays, re-renders the identical cut for any marker in the replayed range, and rejoins the Service on its consensus-log position; it does not have to wait for trading to resume. |
| Leader changes between the two markers | The producer's session follows the new leader; if the session is lost the marker ack times out and the extract is retried rather than emitted against a partial view. |