feat: Add native Go 1.23 Iterator support #3916
Conversation
- Create tools/gen-iterators to generate type-safe iterators for all List methods. - Generate github/iterators.go containing generic iter.Seq2 wrappers. - Update tools/metadata to ignore generated iterators (avoiding metadata duplication). - Add tests verifying pagination logic.
- Create tools/gen-iterators to generate type-safe iterators for all List methods. - Generate github/iterators.go containing generic iter.Seq2 wrappers. - Update tools/metadata to ignore generated iterators (avoiding metadata duplication). - Add table-driven tests verifying pagination and safety (copying opts). - Add benchmark comparing iterators to manual pagination. - Add Godoc example.
|
Review when you can guys. Hoping this feature closes the gap between GO and AWS/Azure. Happy to help :) - Let me know of any changes, hiccups or additions you want. Let's keep the momentum up. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #3916 +/- ##
===========================================
- Coverage 92.45% 78.91% -13.55%
===========================================
Files 203 204 +1
Lines 14954 17565 +2611
===========================================
+ Hits 13826 13861 +35
- Misses 926 3501 +2575
- Partials 202 203 +1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
I'm guessing the binary file should not be committed. Let's add it to .gitignore
There was a problem hiding this comment.
We already have the gen-accessors.go in the github package. Maybe we should move this file to the github/gen-iterators.go for consistency.
There was a problem hiding this comment.
I think that all functions in the file should be covered by tests (can be generated, of course).
There was a problem hiding this comment.
Iterators generation should be added to the //go:generate instruction, see
Line 6 in e32b6b2
2 participants
This PR introduces native Go 1.23 iterator support to the entire library via a new code generator.
The Problem: Pagination in go-github currently requires verbose boilerplate:
opt := &github.RepositoryListByUserOptions{ListOptions: github.ListOptions{PerPage: 10}}
for {
repos, resp, err := client.Repositories.ListByUser(ctx, "user", opt)
// handle err
for _, repo := range repos { ... }
if resp.NextPage == 0 { break }
opt.Page = resp.NextPage
}
The Solution: With this PR, users can simply write:
for repo, err := range client.Repositories.ListByUserIter(ctx, "user", opt) {
if err != nil { log.Fatal(err) }
fmt.Println(repo.GetName())
}
Implementation Details:
tools/gen-iterators: A new AST-based tool (similar to gen-accessors) that scans all List* methods, identifies their pagination patterns (Page/Cursor), and generates a corresponding *Iter method returning iter.Seq2.
It handles methods using standard ListOptions embedding and explicit Page fields.
It copies the opts struct to avoid mutating the caller's options during iteration.
github/iterators.go: The generated file containing hundreds of type-safe iterators.
Metadata Handling: Updated tools/metadata to exclude the generated iterators from API operation mapping validation, as they wrap existing operations.
Testing & Benchmarks:
Benchmarks: github/iterators_benchmark_test.go confirms that the iterator abstraction introduces negligible overhead compared to manual looping.
Correctness: github/iterators_test.go (table-driven) verifies single-page, multi-page, and error handling scenarios.
Safety: Tests explicitly verify that the user's opts struct is NOT mutated by the iterator.
Documentation:
Added github/example_iterators_test.go to provide a clear example in the Godoc.
This modernizes the library to leverage the latest Go features, significantly improving Developer Experience (DX) by eliminating error-prone pagination boilerplate.