Full developer documentation: cercuit-ola.github.io/defi-agent-pipeline2026
The DeFi space in 2026 has a research problem that isn’t about intelligence it’s about architecture.
AI agents are everywhere. Yield monitors, liquidation hunters, governance watchers, cross-chain arbitrage scouts. What separates the ones that work reliably from the ones that quietly score opportunities on stale data and fire alerts into the void is not a smarter model. It’s a structured pipeline, a system where data flows through defined stages, failures are visible, and confidence degrades explicitly rather than silently.
This article documents that pipeline. Not the agent code — the architecture that makes agent code trustworthy.
Before we get into what the pipeline is, it’s worth being precise about how agents fail without one.
WITHOUT a pipeline:
[Chain event]
│
▼
[Scoring formula] ←── oracle price from 12 blocks ago
│ CEX feed that timed out 4 minutes ago
│ liquidity depth read from a cached state
▼
[Alert fires]
confidence: ???
data age: ???
trust: none
The agent above is not broken in any obvious way. It produces output. It fires alerts. The problem is that it has no answer to the question how fresh is this signal? — and without that answer, no downstream system (human or machine) can calibrate how much to trust it.
The pipeline architecture answers that question at every stage.
┌──────────────────────────────────────────────────────────────────────────┐
│ DeFi Research Agent Pipeline │
│ │
│ ┌───────────┐ ┌───────────────┐ ┌─────────┐ │
│ │ │ │ │ │ │ │
│ │ INGESTION │───►│ NORMALISATION │───►│ SCORING │ │
│ │ │ │ │ │ │ │
│ │ on-chain │ │ schema map │ │ OSF-1 │ │
│ │ off-chain │ │ deduplication │ │ ranking │ │
│ │ oracles │ │ ts alignment │ │ filters │ │
│ │ gov feeds │ │ confidence │ │ │ │
│ └───────────┘ └───────────────┘ └────┬────┘ │
│ │ │
│ ▼ │
│ ┌──────────┐ ┌────────┐ │
│ │ │ │ │ │
│ │ OUTPUT │◄───│DECISION│ │
│ │ │ │ │ │
│ │ JSON rpt │ │ tier │ │
│ │ alerts │ │ select │ │
│ │ dashboard│ │ strat. │ │
│ │ attests. │ │ │ │
│ └──────────┘ └────────┘ │
│ │
│ Each stage has: defined input schema · defined output schema │
│ defined failure modes · explicit confidence tracking │
└──────────────────────────────────────────────────────────────────────────┘
The five stages are: Ingestion → Normalisation → Scoring → Decision → Output. Every stage has a contract with the next one. When something breaks, you know exactly where to look.
Not all DeFi data is the same. The ingestion layer must treat these four categories differently:
DATA SOURCE TAXONOMY
════════════════════════════════════════════════════════════════
ON-CHAIN OFF-CHAIN
───────────────────────── ─────────────────────────────────
Trust: cryptographic Trust: provider-dependent
Latency: 1–12s (block) Latency: 100ms–5 min
Source: RPC / indexer Source: CEX APIs, news, sentiment
• eth_call state reads • Binance / Coinbase WebSocket
• eth_getLogs events • DeFiLlama TVL & yield
• Block metadata • LunarCrush sentiment
• Mempool (Flashbots) • Gauntlet risk scores
ORACLE GOVERNANCE
───────────────────────── ─────────────────────────────────
Trust: network consensus Trust: protocol-dependent
Latency: 0–400ms Latency: minutes–hours
On-chain in location, Structured event streams
off-chain in origin requiring interpretation
• Chainlink Data Feeds • Snapshot / Tally proposals
• Pyth Network (400ms) • Collateral factor changes
• Uniswap v3 TWAP • Interest rate model updates
• Redstone • Circuit breaker triggers
════════════════════════════════════════════════════════════════
Key risk: ORACLE MANIPULATION — always cross-reference.
Key risk: GOVERNANCE BLINDSPOT — parameter changes shift risk
profiles before price moves signal anything.
The governance category is the one most agents skip. A Compound governance proposal that changes the collateral factor for WBTC is not a price event it’s a risk event. An agent that tracks only price feeds will be caught flat-footed when parameter changes alter the profitability of a strategy it has been recommending for three days.
Consider what actually arrives from a Uniswap v3 swap on Arbitrum:
RAW EVENT (what the chain gives you)
─────────────────────────────────────────────────────────────
amount0: -2500000000000000000
amount1: 8312440000
sqrtPriceX96: 1771595571142957147523985337...
tick: 202461
─────────────────────────────────────────────────────────────
AFTER NORMALISATION (what the scoring engine needs)
─────────────────────────────────────────────────────────────
{
"record_id": "evt_arb_uniswap_0x7a3b...14829401",
"chain": "arbitrum",
"protocol": "uniswap_v3",
"event_at": 1719432811000, ← block timestamp (causal order)
"ingested_at": 1719432811340, ← agent receipt (latency calc)
"asset_in": { "id": "WETH", "amount": "2.5" },
"asset_out": { "id": "USDC", "amount": "8312.44" },
"price_usd": "3324.976",
"confidence": 0.97, ← data quality signal
"data_freshness": "live",
"tags": ["price_divergence"] ← cross-ref flag
}
─────────────────────────────────────────────────────────────
The gap between those two representations is the normalisation stage’s entire job. The three operations that matter most:
NORMALISATION OPERATIONS
═══════════════════════════════════════════════════════════════
1. TIMESTAMP ALIGNMENT
┌────────────────────────────────────────────────────┐
│ event_at = block timestamp (causal ordering) │
│ ingested_at = agent receipt time (latency calcs) │
│ │
│ These diverge by 2–5 seconds under normal load. │
│ Never conflate them. Never compare CEX timestamps │
│ directly to block timestamps. │
└────────────────────────────────────────────────────┘
2. TOKEN DECIMAL PRECISION
┌────────────────────────────────────────────────────┐
│ WETH raw: 2500000000000000000 (18 decimals) │
│ USDC raw: 8312440000 (6 decimals) │
│ │
│ Store BOTH: decimal string + original raw integer │
│ Never use float arithmetic — rounding compounds │
└────────────────────────────────────────────────────┘
3. CANONICAL ASSET IDs
┌────────────────────────────────────────────────────┐
│ "WETH" = "ETH" = "0xC02aaA...4082" = "ethereum" │
│ │
│ All map to canonical ID: "WETH" │
│ Via maintained asset registry, loaded at startup │
└────────────────────────────────────────────────────┘
═══════════════════════════════════════════════════════════════
The confidence field on every normalised record is the normalisation stage's most important output. It's a [0, 1] value derived from source type, data freshness, and cross-reference agreement. It carries forward through scoring, degrading the entire score if any source is stale.
Before you can score, you group. The correlation engine assembles records that reference the same asset pairs within a 5-second window into opportunity candidates. A CEX price record + a Uniswap pool reserve read + a Chainlink oracle price = a CEX-DEX arbitrage candidate.
Then OSF-1 runs:
OSF-1: OPPORTUNITY SCORING FORMULA
═══════════════════════════════════════════════════════════════
score = (Y × U × C) / (R + L)
┌─────────────────────────────────────────────────────────┐
│ Y Yield estimate │
│ = expected_return_bps / 10000 │
│ Liquidation 8% bonus → Y = 0.08 │
│ CEX-DEX spread 25bps → Y = 0.0025 │
├─────────────────────────────────────────────────────────┤
│ U Urgency weight │
│ = 1 + (1 / max(1, opportunity_age_seconds)) │
│ Age 0s → U = 2.0 (maximum freshness bonus) │
│ Age 60s → U ≈ 1.016 │
│ Age 5m → U ≈ 1.003 │
├─────────────────────────────────────────────────────────┤
│ C Confidence factor │
│ = mean(source_confidences) × freshness_factor │
│ All sources within TTL → F = 1.0 │
│ Any source 1–2× TTL → F = 0.7 │
│ Any source 2–3× TTL → F = 0.4 │
│ Any source > 3× TTL → F = 0.0 (BLOCKS score) │
├─────────────────────────────────────────────────────────┤
│ R Risk score │
│ = 0.5×SC + 0.3×OR + 0.2×LQ │
│ SC = smart contract risk (Gauntlet/DeFi Safety) │
│ OR = oracle risk (type + deviation history) │
│ LQ = liquidity risk (pool depth vs position size) │
├─────────────────────────────────────────────────────────┤
│ L Liquidity penalty │
│ = 1 + max(0, (pos_size/pool_liq − 0.01)) × 5 │
│ ≤1% of pool liquidity → L = 1.0 (neutral) │
│ 5% of pool liquidity → L ≈ 1.2 (penalty kicks in) │
└─────────────────────────────────────────────────────────┘
═══════════════════════════════════════════════════════════════
Here’s the formula at work on real scenario inputs:
OSF-1 REFERENCE VALUES
══════════════════════════════════════════════════════════════════
Y U C R L
CEX-DEX arb, 25bps, 0s, live 0.0025 2.0 0.95 0.25 1.01 → 0.019 —
Yield arb, 180bps, 30s, live 0.018 1.032 0.92 0.40 1.00 → 0.043 —
Liquidation, 5%, 0s, low-risk 0.05 2.0 0.97 0.15 1.00 → 0.647 B
Liquidation, 8%, 0s, blue-chip 0.08 2.0 0.98 0.10 1.00 → 1.568 A
Tier thresholds: A ≥ 0.80 │ B ≥ 0.60 │ C ≥ 0.40 │ D < 0.40
══════════════════════════════════════════════════════════════════
Note: scores are relative within a single scoring cycle.
Always use tier labels (A/B/C/D) for cross-cycle comparisons.
The formula has one particularly important property: a single expired feed can zero out any score. If any source’s data exceeds 3× its configured TTL, C = 0, and score = 0. This is a design choice, not a bug — the pipeline refuses to score on data it cannot vouch for.
The most instructive thing about looking at real production architectures is that two very different approaches — event-driven and poll-and-model — converge on the same pipeline structure. They just enter at different points and move at different speeds.
ARCHITECTURE COMPARISON
════════════════════════════════════════════════════════════════════════
OpenClawD AiXT Trading Intelligence
────────────────────────── ──────────────────────────────────
Approach: Reactive / event-driven Approach: Poll-and-model
┌──────────────────────────┐ ┌──────────────────────────────┐
│ WebSocket Subscriptions │ │ Polling Loop (30s cadence) │
│ (Alchemy / Infura) │ │ REST APIs + RPC calls │
└──────────┬───────────────┘ └─────────────┬────────────────┘
│ │
▼ ▼
┌──────────────────────────┐ ┌──────────────────────────────┐
│ Event Router │ │ Feature Store │
│ routes by event type │ │ keyed by (protocol, │
│ │ │ asset_pair, chain) │
└──────────┬───────────────┘ └─────────────┬────────────────┘
│ │
▼ ▼
┌──────────────────────────┐ ┌──────────────────────────────┐
│ Correlation Engine │ │ Gradient-Boosted Tree │
│ 5s window, asset+chain │ │ retrained weekly │
│ → Candidate Assembler │ │ → OSF-1 compatible score │
└──────────┬───────────────┘ └─────────────┬────────────────┘
│ │
└────────────────┬───────────────────┘
│
▼
┌────────────────────────┐
│ Shared Decision Layer │
│ Same output JSON schema│
│ Same tier (A/B/C/D) │
└────────────────────────┘
BEST AT: BEST AT:
Sub-minute arb opportunities Multi-day yield trend signals
Liquidation racing Governance risk monitoring
Price divergence detection Protocol health deterioration
MEV-adjacent research Macro sentiment correlation
════════════════════════════════════════════════════════════════════════
These are not competing architectures. A production stack runs both.
OpenClawD’s key innovation in the ingestion layer: every swap event is cross-referenced against the most recent Pyth oracle price as it enters the normaliser. If the implied pool price deviates from the oracle by more than 0.5%, the record is tagged price_divergence: true. The correlation engine uses this tag to fast-track CEX-DEX arbitrage candidate assembly — a divergence-tagged record jumps the queue.
AiXT’s feature vector per (protocol, asset_pair, chain) triplet at each polling cycle:
AiXT FEATURE VECTOR
────────────────────────────────────────────────
Protocol / Asset / Chain triplet features:
yield_current current APR
yield_7d_avg 7-day rolling average APR
utilisation_rate current utilisation
utilisation_1h_delta 1h rate of change
tvl_usd current total value locked
tvl_24h_delta 24h TVL change (%)
governance_risk_score from Gauntlet / Chaos Labs
cex_funding_rate perpetual futures funding rate
sentiment_4h 4h rolling score from LunarCrush
────────────────────────────────────────────────
Model: gradient-boosted tree
Output: OSF-1 compatible score (retrained weekly)
The output of both systems is the same JSON schema. A downstream execution system doesn’t need to know whether a signal came from an event-driven reactive pipeline or a 30-second polling model. It sees a scored, tiered strategy candidate with data lineage attached.
A score without provenance is a black box. The pipeline’s output format solves this with three fields on every opportunity:
TRUST METADATA IN EVERY OUTPUT RECORD
══════════════════════════════════════════════════════════════
"confidence": 0.91
↑ Derived from source quality and data freshness.
Tells the consumer: how much should I rely on this score?
"status": "degraded"
↑ Set at report level if any feed is stale or down.
Tells the consumer: the scoring context was impaired.
"sources": [
{ "source_id": "chainlink_eth_usd", "ingested_at": 1719432800000, "confidence": 0.97 },
{ "source_id": "uniswap_v3_swap", "ingested_at": 1719432811340, "confidence": 0.96 },
{ "source_id": "binance_spot", "ingested_at": 1719432790000, "confidence": 0.88 }
]
↑ Full data lineage. Every feed that contributed to this
score is listed with its ingestion timestamp.
Tells the consumer: where did this data come from?
══════════════════════════════════════════════════════════════
An agent that hides its own uncertainty is worse than no agent.
This is the trust framework. It doesn’t require ZK proofs (though the pipeline spec notes that ZKML attestations are the direction the field is moving for high-stakes decisions). It requires only that the pipeline be honest about what it knew, when it knew it, and how confident it was.
PIPELINE OUTPUT FORMATS
══════════════════════════════════════════════════════════════
JSON REPORT (canonical)
├── generated every 60s or on critical event
├── full opportunity set, data lineage, confidence metadata
└── schema: report_id, generated_at, status, opportunities[]
├── fires when score exceeds configured threshold
├── webhook, Telegram, or Slack delivery
└── contains: top opportunity only + tier + confidence
DASHBOARD (real-time)
├── WebSocket stream of scoring output
├── updates on every cycle
└── for human analysts monitoring opportunity flow
SIGNED ATTESTATION (audit/on-chain)
├── cryptographically signed summary of recommendation
├── for downstream on-chain verification
└── the ZKML-ready output format for future integration
══════════════════════════════════════════════════════════════
Every pipeline has failure modes. The most dangerous ones are silent. Here are the three that kill production agents most often:
TOP 3 SILENT FAILURE MODES
══════════════════════════════════════════════════════════════
1. WEBSOCKET HEARTBEAT FAILURE
─────────────────────────────────────────────────────────
What happens: Provider drops connection silently.
Client doesn't receive a close frame.
Agent continues "running" on no new data.
Confidence values start degrading.
Eventually all scores → 0.
Symptom: Feed shows "live" but no events for 3+ minutes.
Fix: Heartbeat timer: reconnect if silent > 2×block_time
2. EXPIRED FEED DRIVING SCORES TO ZERO
─────────────────────────────────────────────────────────
What happens: One feed exceeds 3× TTL.
Confidence factor C collapses to 0.
All scores → 0.
No tier-A or tier-B candidates.
Symptom: Degraded_feeds array not empty + all scores < 0.01
Fix: Fix the feed. Not the formula.
3. DUPLICATE RECORDS FROM PARALLEL SOURCES
─────────────────────────────────────────────────────────
What happens: WebSocket and polling fallback both active.
WebSocket lags → poller fills in.
WebSocket recovers → same events arrive again.
Correlation engine creates duplicate candidates.
Symptom: Same opportunity listed twice in output report.
Fix: Deterministic record_id = sha256(tx_hash + log_index)
Deduplication bloom filter, 10-minute TTL.
══════════════════════════════════════════════════════════════
I structured this project using the Diataxis framework a documentation architecture that separates content into four types, each serving a different reader in a different mode:
DIATAXIS DOCUMENTATION FRAMEWORK
══════════════════════════════════════════════════════════════
PRACTICAL THEORETICAL
┌──────────────┐ ┌──────────────┐
STUDY │ TUTORIALS │ │ CONCEPTS │
│ (learning) │ │(understanding)│
│ │ │ │
│ step-by-step │ │ mental models│
│ guided action│ │ why it works │
└──────────────┘ └──────────────┘
┌──────────────┐ ┌──────────────┐
WORK │ GUIDES │ │ REFERENCE │
│ (doing) │ │(information) │
│ │ │ │
│ task-focused │ │ specs, schemas│
│ assumes skill│ │ formulas, APIs│
└──────────────┘ └──────────────┘
══════════════════════════════════════════════════════════════
Each quadrant serves a different reader in a different mode.
Mixing them makes every type harder to use.
For a DeFi research pipeline, the separation matters because the audiences are completely different. A protocol engineer reading the concepts section wants mental models why does confidence degrade, why does the pipeline have five stages rather than three? A developer setting up the ingestion layer for the first time needs a guide with actual YAML config examples. An analyst debugging a scoring anomaly at 11pm needs the reference formula. A DevOps engineer at 2am needs the troubleshooting checklist, fast, without scrolling past prose.
Designing documentation architecture for a complex system is itself a form of research. It requires you to understand the system well enough to explain it clearly, to someone who doesn’t know what you know, in the context where they need it.
I did not implement this pipeline. I designed its documentation architecture.
The OSF-1 formula is derivable from first principles — it’s a real formula with worked examples and reference values that produce intuitive results. The JSON schemas are valid and complete. The API endpoints are real. The troubleshooting scenarios reflect actual failure modes in production DeFi systems.
What this project demonstrates is that understanding a sophisticated DeFi research system well enough to document it precisely is a distinct and valuable skill. The documentation is a contribution. Not a placeholder for code — a thing that helps developers build correctly.
The complete Diataxis-structured developer documentation — Concepts, Guides, Reference, and Troubleshooting, all in a single-page navigable interface — is live here:
cercuit-ola.github.io/defi-agent-pipeline2026
It covers:
Written by Samuel Olaide Oba · 2026 DeFi researcher and technical writer
The 2026 Guide to DeFi Research Agents: Data, Signals, and Scoring Systems was originally published in Coinmonks on Medium, where people are continuing the conversation by highlighting and responding to this story.


