Skip to content

Commit b40d1fa

Browse files
gnd: add mock IPFS client for file data source testing
Replaces the dummy IpfsRpcClient with a MockIpfsClient that serves pre-loaded CID → bytes from a "files" array in test JSON files. Missing CIDs are reported with a clear error instead of a 60-second timeout. Also adds IpfsResponse::for_test() helper and documents the unmocked eth_call timeout in the troubleshooting guide.
1 parent b2e2db9 commit b40d1fa

6 files changed

Lines changed: 212 additions & 11 deletions

File tree

gnd/docs/gnd-test.md

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -767,7 +767,7 @@ GraphQL queries → Assertions
767767
- **Real WASM runtime:** Uses `EthereumRuntimeAdapterBuilder` with real `ethereum.call` host function
768768
- **Pre-populated call cache:** `eth_call` responses are cached before indexing starts
769769
- **No IPFS for manifest:** Uses `FileLinkResolver` to load manifest/WASM from build directory
770-
- **Dummy RPC adapter:** Registered at `http://0.0.0.0:0` for capability lookup; never actually called
770+
- **Dummy RPC adapter:** Registered at `http://0.0.0.0:0` — exists so the runtime can resolve an adapter with the required capabilities. If a mapping makes an `ethereum.call` that has no matching mock in `ethCalls`, the call misses the cache and falls through to this dummy adapter. The connection is refused immediately (port 0 is invalid), which graph-node treats as a possible reorg and restarts the block stream. The indexer then loops until the 60-second test timeout. See [Unmocked eth_call](#unmocked-eth_call-causes-60-second-timeout) in Troubleshooting.
771771

772772
## Troubleshooting
773773

@@ -799,6 +799,28 @@ GraphQL queries → Assertions
799799
2. Check function signature format: `"functionName(inputTypes)(returnTypes)"`
800800
3. Ensure parameters are in correct order
801801

802+
### Unmocked eth_call Causes 60-Second Timeout
803+
804+
**Cause:** A mapping handler calls `ethereum.call` (directly or via a generated contract binding) for a call that has no matching entry in `ethCalls`. The call misses the pre-populated cache and is forwarded to the dummy RPC adapter at `http://0.0.0.0:0`. The connection is refused immediately, but graph-node interprets connection errors as a possible chain reorganisation and restarts the block stream instead of failing. The indexer loops indefinitely until the test runner's 60-second timeout expires.
805+
806+
**Symptom:** Test fails with `Sync timeout after 60s` with no indication of which call was missing.
807+
808+
**Fix:**
809+
1. Add the missing call to `ethCalls` in your test block:
810+
```json
811+
"ethCalls": [
812+
{
813+
"address": "0xContractAddress",
814+
"function": "myFunction(uint256):(address)",
815+
"params": ["42"],
816+
"returns": ["0xSomeAddress"]
817+
}
818+
]
819+
```
820+
2. If the call is not supposed to happen, check the mapping logic — a code path may be executing unexpectedly.
821+
822+
**Known limitation:** There is currently no fail-fast error for unmocked calls. The only signal is the timeout. A future improvement will make the dummy adapter panic immediately on a cache miss with a descriptive message.
823+
802824
### Block Handler Not Firing
803825

804826
**Cause:** Block handlers auto-fire, but might be outside data source's active range.

gnd/src/commands/test/mock_ipfs.rs

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
//! Mock IPFS client for `gnd test`.
2+
//!
3+
//! Replaces the real `IpfsRpcClient` with a map of pre-loaded CID → bytes.
4+
//! Any CID not found in the map is sent to the `unresolved_tx` channel and
5+
//! an error is returned so the `OffchainMonitor` retries with backoff.
6+
//! After sync, the runner drains the channel and reports missing CIDs.
7+
8+
use std::collections::HashMap;
9+
use std::sync::Arc;
10+
11+
use async_trait::async_trait;
12+
use graph::bytes::Bytes;
13+
use graph::ipfs::{
14+
ContentPath, IpfsClient, IpfsError, IpfsMetrics, IpfsRequest, IpfsResponse, IpfsResult,
15+
};
16+
use tokio::sync::mpsc::UnboundedSender;
17+
18+
pub struct MockIpfsClient {
19+
pub files: HashMap<ContentPath, Bytes>,
20+
pub metrics: IpfsMetrics,
21+
pub unresolved_tx: UnboundedSender<ContentPath>,
22+
}
23+
24+
#[async_trait]
25+
impl IpfsClient for MockIpfsClient {
26+
fn metrics(&self) -> &IpfsMetrics {
27+
&self.metrics
28+
}
29+
30+
async fn call(self: Arc<Self>, req: IpfsRequest) -> IpfsResult<IpfsResponse> {
31+
let path = match req {
32+
IpfsRequest::Cat(p) | IpfsRequest::GetBlock(p) => p,
33+
};
34+
35+
match self.files.get(&path) {
36+
Some(bytes) => Ok(IpfsResponse::for_test(path, bytes.clone())),
37+
None => {
38+
let _ = self.unresolved_tx.send(path.clone());
39+
Err(IpfsError::ContentNotAvailable {
40+
path,
41+
reason: anyhow::anyhow!("CID not found in mock 'files'"),
42+
})
43+
}
44+
}
45+
}
46+
}

gnd/src/commands/test/mod.rs

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ mod block_stream;
3232
mod eth_calls;
3333
mod matchstick;
3434
mod mock_chain;
35+
mod mock_ipfs;
3536
mod noop;
3637
mod output;
3738
mod runner;
@@ -170,7 +171,7 @@ pub async fn run_test(opt: TestOpt) -> Result<()> {
170171
}
171172
};
172173

173-
match runner::run_single_test(&opt, &manifest_info, &test_file).await {
174+
match runner::run_single_test(&opt, &manifest_info, &test_file, &path).await {
174175
Ok(result) => {
175176
output::print_test_result(&test_file.name, &result);
176177
if result.is_passed() {

gnd/src/commands/test/runner.rs

Lines changed: 68 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313
use super::assertion::run_assertions;
1414
use super::block_stream::StaticStreamBuilder;
1515
use super::mock_chain;
16+
use super::mock_ipfs::MockIpfsClient;
1617
use super::noop::{NoopAdapterSelector, StaticBlockRefetcher};
1718
use super::schema::{TestFile, TestResult};
1819
use super::trigger::build_blocks_with_triggers;
@@ -35,7 +36,7 @@ use graph::data::subgraph::schema::SubgraphError;
3536
use graph::endpoint::EndpointMetrics;
3637
use graph::env::EnvVars;
3738
use graph::firehose::{FirehoseEndpoint, FirehoseEndpoints, SubgraphLimit};
38-
use graph::ipfs::{IpfsMetrics, IpfsRpcClient, ServerAddress};
39+
use graph::ipfs::{ContentPath, IpfsMetrics};
3940
use graph::prelude::{
4041
DeploymentHash, LoggerFactory, NodeId, SubgraphCountMetric, SubgraphName, SubgraphRegistrar,
4142
SubgraphStore as SubgraphStoreTrait, SubgraphVersionSwitchingMode,
@@ -52,6 +53,7 @@ use graph_node::config::Config;
5253
use graph_node::manager::PanicSubscriptionManager;
5354
use graph_node::store_builder::StoreBuilder;
5455
use graph_store_postgres::{ChainHeadUpdateListener, ChainStore, Store, SubgraphStore};
56+
use std::collections::HashMap;
5557
use std::marker::PhantomData;
5658
use std::path::{Path, PathBuf};
5759
use std::sync::Arc;
@@ -256,6 +258,7 @@ pub async fn run_single_test(
256258
opt: &TestOpt,
257259
manifest_info: &ManifestInfo,
258260
test_file: &TestFile,
261+
test_file_path: &Path,
259262
) -> Result<TestResult> {
260263
// Warn (and short-circuit) when there are no assertions.
261264
if test_file.assertions.is_empty() {
@@ -286,6 +289,24 @@ pub async fn run_single_test(
286289
&manifest_info.receipt_required_selectors,
287290
)?;
288291

292+
// Build mock IPFS file map. Fails fast on invalid CIDs or unreadable file paths.
293+
let test_file_dir = test_file_path
294+
.parent()
295+
.map(|p| p.to_path_buf())
296+
.unwrap_or_else(|| PathBuf::from("."));
297+
298+
let mut mock_files: HashMap<ContentPath, graph::bytes::Bytes> = HashMap::new();
299+
for entry in &test_file.files {
300+
let path = ContentPath::new(&entry.cid)
301+
.map_err(|e| anyhow!("Invalid CID '{}' in test files: {}", entry.cid, e))?;
302+
let content = entry
303+
.resolve(&test_file_dir)
304+
.with_context(|| format!("Failed to resolve mock file for CID '{}'", entry.cid))?;
305+
mock_files.insert(path, content);
306+
}
307+
308+
let (unresolved_tx, mut unresolved_rx) = tokio::sync::mpsc::unbounded_channel::<ContentPath>();
309+
289310
// Create the database for this test. For pgtemp, the `db` value must
290311
// stay alive for the duration of the test — dropping it destroys the database.
291312
let db = create_test_database(opt, &manifest_info.build_dir)?;
@@ -296,7 +317,15 @@ pub async fn run_single_test(
296317

297318
let chain = setup_chain(&logger, blocks.clone(), &stores).await?;
298319

299-
let ctx = setup_context(&logger, &stores, &chain, manifest_info).await?;
320+
let ctx = setup_context(
321+
&logger,
322+
&stores,
323+
&chain,
324+
manifest_info,
325+
mock_files,
326+
unresolved_tx,
327+
)
328+
.await?;
300329

301330
// Populate eth_call cache with mock responses before starting indexer.
302331
// This ensures handlers can successfully retrieve mocked contract call results.
@@ -330,7 +359,34 @@ pub async fn run_single_test(
330359
)
331360
.await
332361
{
333-
Ok(()) => run_assertions(&ctx, &test_file.assertions).await,
362+
Ok(()) => {
363+
// Drain any CIDs that were requested but not found in the mock.
364+
// Deduplicate so each missing CID is listed once.
365+
let mut unresolved: Vec<ContentPath> = Vec::new();
366+
while let Ok(cid) = unresolved_rx.try_recv() {
367+
if !unresolved.contains(&cid) {
368+
unresolved.push(cid);
369+
}
370+
}
371+
372+
if !unresolved.is_empty() {
373+
let cid_list = unresolved
374+
.iter()
375+
.map(|p| format!(" - {}", p))
376+
.collect::<Vec<_>>()
377+
.join("\n");
378+
Ok(TestResult {
379+
handler_error: Some(format!(
380+
"File data source requested CID not found in mock 'files':\n{}\n\
381+
Add the missing CID(s) to the \"files\" array in your test JSON.",
382+
cid_list
383+
)),
384+
assertions: vec![],
385+
})
386+
} else {
387+
run_assertions(&ctx, &test_file.assertions).await
388+
}
389+
}
334390
Err(subgraph_error) => {
335391
// The subgraph handler threw a fatal error during indexing.
336392
// Report it as a test failure without running assertions.
@@ -614,6 +670,8 @@ async fn setup_context(
614670
stores: &TestStores,
615671
chain: &Arc<Chain>,
616672
manifest_info: &ManifestInfo,
673+
mock_files: HashMap<ContentPath, graph::bytes::Bytes>,
674+
unresolved_tx: tokio::sync::mpsc::UnboundedSender<ContentPath>,
617675
) -> Result<TestContext> {
618676
let build_dir = &manifest_info.build_dir;
619677
let manifest_path = &manifest_info.manifest_path;
@@ -643,13 +701,14 @@ async fn setup_context(
643701
FileLinkResolver::new(Some(build_dir.to_path_buf()), aliases),
644702
);
645703

646-
// IPFS client is required by the instance manager constructor but not used
647-
// for manifest loading (FileLinkResolver handles that).
704+
// Replace the real IPFS client with a mock that serves pre-loaded content.
705+
// FileLinkResolver handles manifest loading; the mock handles file data sources.
648706
let ipfs_metrics = IpfsMetrics::new(&mock_registry);
649-
let ipfs_client = Arc::new(
650-
IpfsRpcClient::new_unchecked(ServerAddress::test_rpc_api(), ipfs_metrics, logger)
651-
.context("Failed to create IPFS client")?,
652-
);
707+
let ipfs_client = Arc::new(MockIpfsClient {
708+
files: mock_files,
709+
metrics: ipfs_metrics,
710+
unresolved_tx,
711+
});
653712

654713
let ipfs_service = ipfs_service(
655714
ipfs_client,

gnd/src/commands/test/schema.rs

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@
3030
//! }
3131
//! ```
3232
33+
use graph::bytes::Bytes;
3334
use serde::Deserialize;
3435
use serde_json::Value;
3536
use std::path::{Path, PathBuf};
@@ -39,6 +40,10 @@ use std::path::{Path, PathBuf};
3940
pub struct TestFile {
4041
pub name: String,
4142

43+
/// Mock IPFS file contents keyed by CID. Used for file data sources.
44+
#[serde(default)]
45+
pub files: Vec<MockFile>,
46+
4247
/// Ordered sequence of mock blocks to index.
4348
#[serde(default)]
4449
pub blocks: Vec<TestBlock>,
@@ -48,6 +53,64 @@ pub struct TestFile {
4853
pub assertions: Vec<Assertion>,
4954
}
5055

56+
/// A mock IPFS file entry for file data source testing.
57+
///
58+
/// Exactly one of `content` or `file` must be set.
59+
#[derive(Debug, Clone, Deserialize)]
60+
pub struct MockFile {
61+
/// Syntactically valid IPFS CID (v0 `Qm...` or v1 `bafy...`).
62+
/// The CID does not need to be the actual hash of the content — the mock
63+
/// ignores the hash relationship.
64+
pub cid: String,
65+
66+
/// Inline UTF-8 content. Exactly one of `content` or `file` must be set.
67+
#[serde(default)]
68+
pub content: Option<String>,
69+
70+
/// Path to a file whose contents are loaded as UTF-8.
71+
/// Resolved relative to the test JSON file. Exactly one of `content` or
72+
/// `file` must be set.
73+
#[serde(default)]
74+
pub file: Option<String>,
75+
}
76+
77+
impl MockFile {
78+
/// Resolve this entry to bytes, given the directory of the test JSON file.
79+
///
80+
/// Fails if:
81+
/// - neither `content` nor `file` is set
82+
/// - both `content` and `file` are set
83+
/// - the referenced `file` path cannot be read
84+
pub fn resolve(&self, test_dir: &Path) -> anyhow::Result<Bytes> {
85+
match (&self.content, &self.file) {
86+
(Some(content), None) => Ok(Bytes::from(content.clone().into_bytes())),
87+
(None, Some(file)) => {
88+
let path = if Path::new(file).is_absolute() {
89+
PathBuf::from(file)
90+
} else {
91+
test_dir.join(file)
92+
};
93+
let data = std::fs::read(&path).map_err(|e| {
94+
anyhow::anyhow!("Failed to read file '{}': {}", path.display(), e)
95+
})?;
96+
Ok(Bytes::from(data))
97+
}
98+
(Some(_), Some(_)) => {
99+
anyhow::bail!(
100+
"MockFile entry for CID '{}' must have either 'content' or 'file', not both",
101+
self.cid
102+
)
103+
}
104+
(None, None) => {
105+
anyhow::bail!(
106+
"MockFile entry for CID '{}' must have either 'content' or 'file'",
107+
self.cid
108+
)
109+
}
110+
}
111+
}
112+
}
113+
51114
#[derive(Debug, Clone, Deserialize)]
52115
pub struct TestBlock {
53116
/// Block number. If omitted, auto-increments starting from `start_block`

graph/src/ipfs/client.rs

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -196,6 +196,16 @@ pub struct IpfsResponse {
196196
}
197197

198198
impl IpfsResponse {
199+
/// Construct an `IpfsResponse` from pre-buffered bytes.
200+
///
201+
/// Intended for mock `IpfsClient` implementations in tests.
202+
pub fn for_test(path: ContentPath, bytes: Bytes) -> Self {
203+
Self {
204+
path,
205+
response: reqwest::Response::from(http::Response::new(bytes)),
206+
}
207+
}
208+
199209
/// Reads and returns the response body.
200210
///
201211
/// If the max size is specified and the response body is larger than the max size,

0 commit comments

Comments
 (0)