Skip to content

Add offheap memory tracking to Bench YAML reports#671

Merged
tlwillke merged 2 commits into
mainfrom
bench_offheap_tracking
Jun 9, 2026
Merged

Add offheap memory tracking to Bench YAML reports#671
tlwillke merged 2 commits into
mainfrom
bench_offheap_tracking

Conversation

@MarkWolters

Copy link
Copy Markdown
Contributor

This PR updates the tracking of offheap memory usage by calculating max usage comparing the pre and post phase snapshots instead of just using the final snapshot, and adds offheap memory usage to the tabular results generated by BenchYAML.

Note that the reported offheap usage will appear static across configurations for a given dataset in this test. This is because off-heap memory is allocated during index loading (once per dataset). It includes memory-mapped index files and Direct ByteBuffers for vector storage. All query configurations reuse the same loaded index, so off-heap usage remains constant but different datasets have different index sizes, hence different off-heap usage.

Example output:

Disk Usage Summary Graph Index Build:
  [testDirectory]:
    Total Disk Used: 115.99 MB
    Total Files: 1
    Net Change: 115.99 MB, +1 files
  [indexCache]:
    Total Disk Used: 0 B
    Total Files: 0
    Net Change: 0 B, +0 files
  [Overall Total]:
    Total Disk Used: 115.99 MB
    Total Files: 1
    Net Change: 115.99 MB, +1 files
Index build time: 26.442781 seconds

cohere-english-v3-100k: ProductQuantization(M=128, clusters=256, centered=false) codebooks loaded from PQ_cohere-english-v3-100k_128_256_false_-1.0
cohere-english-v3-100k: ProductQuantization(M=128, clusters=256, centered=false) encoded 99685 vectors [13.17 MB] in 1.75s
cohere-english-v3-100k: Using OnDiskGraphIndex(layers=[LayerInfo{size=99685, degree=32}, LayerInfo{size=3224, degree=32}, LayerInfo{size=95, degree=32}, LayerInfo{size=2, degree=32}], entryPoint=NodeAtLevel(level=3, node=75059), features=NVQ_VECTORS):

Index configuration:
  featureSetForIndex   [NVQ_VECTORS]
  M                    32
  efConstruction       100
  neighborOverflow     1.2
  addHierarchy         true
  refineFinalGraph     true

Query configuration:
  usePruning           true

Overquery    Avg QPS         ± Std Dev    CV %        Mean Latency    Avg Visited    Recall@10    Max heap        Max offheap    
             (of 3)                                   (ms)                                        usage (MB)      usage (MB)     
---------------------------------------------------------------------------------------------------------------------------------
1.00         52315.1         378.7        0.7         0.163           355.3          0.78         755.2           116.4          
2.00         44155.5         619.7        1.4         0.201           515.2          0.92         834.3           116.4          
5.00         28832.6         264.1        0.9         0.306           951.1          0.98         834.3           116.4          
10.00        17835.6         1134.1       6.4         0.504           1536.7         0.99         786.6           116.4          


Overquery    Avg QPS         ± Std Dev    CV %        Mean Latency    Avg Visited    Recall@100    Max heap        Max offheap    
             (of 3)                                   (ms)                                         usage (MB)      usage (MB)     
----------------------------------------------------------------------------------------------------------------------------------
1.00         18204.6         119.4        0.7         0.522           1536.7         0.85          803.5           116.4          
2.00         11269.1         83.9         0.7         0.846           2495.2         0.97          803.5           116.4          

@github-actions

github-actions Bot commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

Before you submit for review:

  • Does your PR follow guidelines from CONTRIBUTIONS.md?
  • Did you summarize what this PR does clearly and concisely?
  • Did you include performance data for changes which may be performance impacting?
  • Did you include useful docs for any user-facing changes or features?
  • Did you include useful javadocs for developer oriented changes, explaining new concepts or key changes?
  • Did you trigger and review regression testing results against the base branch via Run Bench Main?
  • Did you adhere to the code formatting guidelines (TBD)
  • Did you group your changes for easy review, providing meaningful descriptions for each commit?
  • Did you ensure that all files contain the correct copyright header?

If you did not complete any of these, then please explain below.

@tlwillke tlwillke left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@jshook jshook left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only one comment, not a blocker.

@tlwillke tlwillke self-requested a review June 9, 2026 18:33
@tlwillke tlwillke merged commit 9e8d2e9 into main Jun 9, 2026
12 checks passed
@tlwillke tlwillke deleted the bench_offheap_tracking branch June 9, 2026 18:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants