Skip to content

ValkeySearch 1.2.0#354

Draft
adam-fowler wants to merge 12 commits intomainfrom
valkey-search-1.2
Draft

ValkeySearch 1.2.0#354
adam-fowler wants to merge 12 commits intomainfrom
valkey-search-1.2

Conversation

@adam-fowler
Copy link
Collaborator

@adam-fowler adam-fowler commented Feb 27, 2026

ValkeySearch are about to release 1.2. They have completely rebuilt their API model files and the generated code is completely different. This PR attempts to cleanup some of the breaking changes. Some features have been removed eg FT.SEARCH LOCALONLY, FT.AGGREGATE WITHCURSOR, FT.AGGREGATE SCORER

I am also going to suggest a couple of changes to their API model files to make the generated functions cleaner. I'll link to that PR when I post it.

EDIT: While looking at the json model file I found multiple issues with it. I have opened a PR with edits in the valley-search repo, but I thought I would add a phase in the command builder to cleanup up the valkey json model files. The cleanup includes

  • Removal of unnecessary blocks (each block generates a new structure)
    • Blocks that consist of a pure-token and one other parameter are collapsed down to just the second parameter with the token field set.
    • Blocks that consist of one parameter, are collapsed down to the parameter
    • Remove unnecessary counts from API where the count can be ascertained from the size of an array.
  • Added ability to patch json files inside code generator. Currently you can replace arguments wholesale or just edit one field of the argument. I use this twice in the valkey-search json to produce cleaner code.

Signed-off-by: Adam Fowler <adamfowler71@gmail.com>
Signed-off-by: Adam Fowler <adamfowler71@gmail.com>
Signed-off-by: Adam Fowler <adamfowler71@gmail.com>
Signed-off-by: Adam Fowler <adamfowler71@gmail.com>
@adam-fowler adam-fowler marked this pull request as draft February 27, 2026 14:49
@adam-fowler
Copy link
Collaborator Author

@eric-musliner you may find this of interest

@github-actions
Copy link

github-actions bot commented Feb 27, 2026

✅ Pull request no significant performance differences ✅

Summary

New baseline 'pull_request' is WITHIN the 'main' baseline thresholds.

Full Benchmark Comparison

Comparing results between 'main' and 'pull_request'

Host '072065ce4f82' with 4 'x86_64' processors with 15 GB memory, running:
#17~24.04.1-Ubuntu SMP Mon Dec  1 20:10:50 UTC 2025

ValkeyBenchmarks

Client: GET benchmark metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 51 53 55 57 58 58 58 7
pull_request 51 53 54 57 57 57 57 7
Δ 0 0 -1 0 -1 -1 -1 0
Improvement % 0 0 2 0 2 2 2 0

Client: GET benchmark | parallel 20 | 20 concurrent connections metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 51 54 57 59 61 65 65 24
pull_request 51 54 57 59 61 64 64 28
Δ 0 0 0 0 0 -1 -1 4
Improvement % 0 0 0 0 0 2 2 4

Connection: GET benchmark metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 4 4 4 4 4 4 4 10
pull_request 4 4 4 4 4 4 4 10
Δ 0 0 0 0 0 0 0 0
Improvement % 0 0 0 0 0 0 0 0

Connection: GET benchmark – NoOpTracer metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 4 7 8 9 9 9 9 10
pull_request 4 7 8 9 9 9 9 10
Δ 0 0 0 0 0 0 0 0
Improvement % 0 0 0 0 0 0 0 0

Connection: Pipeline array benchmark metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 14 21 24 28 28 29 29 10
pull_request 14 21 24 28 29 29 29 9
Δ 0 0 0 0 1 0 0 -1
Improvement % 0 0 0 0 -4 0 0 -1

Connection: Pipeline benchmark metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 7 24 24 25 25 25 25 9
pull_request 7 24 24 25 25 25 25 9
Δ 0 0 0 0 0 0 0 0
Improvement % 0 0 0 0 0 0 0 0

HashSlot – {user}.whatever metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 0 0 0 0 0 0 0 21
pull_request 0 0 0 0 0 0 0 22
Δ 0 0 0 0 0 0 0 1
Improvement % 0 0 0 0 0 0 0 1

ValkeyCommandEncoder – Command with 7 words metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 0 0 0 0 0 0 0 1032
pull_request 0 0 0 0 0 0 0 1040
Δ 0 0 0 0 0 0 0 8
Improvement % 0 0 0 0 0 0 0 8

ValkeyCommandEncoder – Simple GET metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 0 0 0 0 0 0 0 2518
pull_request 0 0 0 0 0 0 0 2537
Δ 0 0 0 0 0 0 0 19
Improvement % 0 0 0 0 0 0 0 19

ValkeyCommandEncoder – Simple MGET 15 keys metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 0 0 0 0 0 0 0 455
pull_request 0 0 0 0 0 0 0 476
Δ 0 0 0 0 0 0 0 21
Improvement % 0 0 0 0 0 0 0 21

@adam-fowler
Copy link
Collaborator Author

This PR to the json files valkey-io/valkey-search#832 reduces the generated code by 400 lines

@eric-musliner
Copy link
Contributor

@eric-musliner you may find this of interest

Thanks for the awareness! Glad I'm not making use of the ones they are cutting

Signed-off-by: Adam Fowler <adamfowler71@gmail.com>
Signed-off-by: Adam Fowler <adamfowler71@gmail.com>
Signed-off-by: Adam Fowler <adamfowler71@gmail.com>
Signed-off-by: Adam Fowler <adamfowler71@gmail.com>
Signed-off-by: Adam Fowler <adamfowler71@gmail.com>
Signed-off-by: Adam Fowler <adamfowler71@gmail.com>
@adam-fowler
Copy link
Collaborator Author

@eric-musliner I have been playing with some code to clean up badly setup json files. You can see the list of things it does in the PR description.

This completely changes the API for valkey-search but I think it is a lot cleaner. Could you have a look and give us some feedback.

Signed-off-by: Adam Fowler <adamfowler71@gmail.com>
Copy link
Contributor

@eric-musliner eric-musliner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like these changes to the API for valkey-search 1.2.0. I found the previous generated code for the commands when I did it a few months ago very clunky. I didn't like that the blocks with single pure tokens were turned into their own types. You've resolved the main issues I had with it originally. I think this looks great and it's a lot more ergonomic now.

request: .command([
"FT.AGGREGATE", "idx:testIndex", "@loc:[$lon $lat 10 km]",
"PARAMS", "4",
"PARAMS", "2",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For PARAMS I believe nargs or count it's now called should still be 4. According to the docs

https://redis.io/docs/latest/commands/ft.aggregate/
https://valkey.io/commands/ft.aggregate/

count is of the number of arguments, i.e., twice the number of name/value pairs

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For PARAMS I believe nargs or count it's now called should still be 4. According to the docs

https://redis.io/docs/latest/commands/ft.aggregate/
https://valkey.io/commands/ft.aggregate/

count is of the number of arguments, i.e., twice the number of name/value pairs

Oh that's weird and messes with some of my assumptions. I'll need to do some work there.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree it is not very intuitive

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants