Skip to content

fix: Stable StarRocks Driver and Add test code #8619

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
EdwardArchive wants to merge 4 commits into
base: main
Choose a base branch
from
Open

fix: Stable StarRocks Driver and Add test code #8619

EdwardArchive wants to merge 4 commits into from

Conversation

@EdwardArchive
Copy link
Contributor

@EdwardArchive EdwardArchive commented Jan 12, 2026 •
edited
Loading

Related PR : #8454

Problem:

  • VARCHAR columns were returned as []byte instead of string due to MySQL
    driver behavior when scanning into any type
  • DECIMAL columns returned as float64, causing precision loss for large values

Solution:

  • Add starrocksRows wrapper with custom MapScan() using sql.NullXXX types
  • DECIMAL now returns as string to preserve full precision (same as MySQL driver)
  • Add prepareScanDest() for type-safe scan destinations

Testing:

  • Add Docker-based integration tests using StarRocks 4.0.3 all-in-one container
  • 29 test cases covering all StarRocks data types:
    • Basic types (BOOLEAN, INT, BIGINT, FLOAT, DOUBLE, VARCHAR, etc.)
    • Complex types (ARRAY, MAP, STRUCT)
    • Aggregate types (HLL, BITMAP)
    • High-precision DECIMAL (DECIMAL32, DECIMAL64, DECIMAL128)
    • Unicode strings (Korean, Chinese, Japanese, Emoji)
    • Boundary values and NULL handling
  • Refactor teststarrocks to read from init.sql file for better maintainability

Type mapping (MySQL protocol):

  • BOOLEAN → int16 (reported as TINYINT)
  • DECIMAL → string (precision preserved)
  • LARGEINT → string (128-bit exceeds int64)
  • ARRAY/MAP/STRUCT → string (JSON format)
  • JSON → string (reported as VARCHAR)

Checklist:

  • Covered by tests
  • Ran it and it works as intended
  • Reviewed the diff before requesting a review
  • Checked for unhandled edge cases
  • Linked the issues it closes
  • Checked if the docs need to be updated. If so, create a separate Linear DOCS issue
  • Intend to cherry-pick into the release branch
  • I'm proud of this work!

@EdwardArchive
Copy link
Contributor Author

EdwardArchive commented Jan 12, 2026

These test failures are due to missing environment variables (S3, GCS, Druid, MotherDuck credentials)

@EdwardArchive
fix(starrocks): align binary type mapping with MySQL driver
6401aa6 
  - Change BINARY/VARBINARY/BLOB from CODE_BYTES to CODE_STRING
    for consistency with MySQL driver behavior
  - Expand type mapping unit tests to cover all StarRocks types:
    - Boolean, Integer, Float, String, Binary, DateTime
    - Semi-structured types (JSON, ARRAY, MAP, STRUCT)
    - Aggregate-only types (HLL, BITMAP, PERCENTILE) as unsupported
  - Update integration test expectations for VARBINARY type

Signed-off-by: Edward Kim <[email protected]>
@EdwardArchive EdwardArchive changed the title [Enhancement] Stable StarRocks Driver and Add test code fix : Stable StarRocks Driver and Add test code Jan 13, 2026
@EdwardArchive EdwardArchive changed the title fix : Stable StarRocks Driver and Add test code fix: Stable StarRocks Driver and Add test code Jan 13, 2026
@EdwardArchive
Copy link
Contributor Author

EdwardArchive commented Jan 13, 2026

Hi, @k-anshul
Could you take a look when you have time?

@k-anshul
Copy link
Member

k-anshul commented Jan 14, 2026 •
edited
Loading

Hey @EdwardArchive

Thank you for your contribution.
Another super important tests for stability of the driver are the tests in runtime/queries pkg especially metricsview_*_test.go. Would you like to contribute tests those tests as well ? You can do it in a follow up if preferred.

@EdwardArchive
Copy link
Contributor Author

EdwardArchive commented Jan 14, 2026

Hi @k-anshul,
Thanks for the advice — that’s an important point! I’ll add more tests for metricsview_*_test.go.

@EdwardArchive
feat(starrocks): add metricsview tests and fix driver issues
bc51e3d 
  - Add metricsview aggregation, toplist, and comparison tests for StarRocks
  - Implement Stream Load API for test data loading with FE→BE redirect handling
  - Add NULLS LAST for consistent NULL ordering in StarRocks
  - Fix TIMESTAMP handling to use sql.NullTime with parseTime=true
  - Add CONVERT_TZ for timezone support in DateTruncExpr
  - Fix schema resolution using InformationSchema lookup
  - Cast percent_of_total result to DOUBLE for consistent numeric handling
  - Update StarRocks documentation to match driver implementation

Signed-off-by: EdwardKim <[email protected]>
@EdwardArchive
Copy link
Contributor Author

EdwardArchive commented Jan 15, 2026

Hey @k-anshul,
I just added tests for metricsview_*_test.go.
and some changes on backend.
Could you take a look when you have time?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@EdwardArchive @k-anshul