test: remove pandas version restrictions and fix precision issue#17005
test: remove pandas version restrictions and fix precision issue#17005chalmerlowe wants to merge 4 commits intomainfrom
Conversation
There was a problem hiding this comment.
Code Review
This pull request removes pandas version restrictions from system and unit tests and updates the pandas Series assertion logic in test_bq_to_arrow_array_w_pandas_timestamp. The reviewer suggests adding check_names=False to the new assert_series_equal call to prevent potential test failures caused by name mismatches, maintaining the original behavior of checking only value equality.
| series = series.astype(roundtrip.dtype) | ||
| pandas.testing.assert_series_equal(series, roundtrip) |
There was a problem hiding this comment.
While pandas.testing.assert_series_equal provides better diagnostic information than Series.equals(), it is stricter and validates the name attribute by default. Since series is initialized without a name (line 557) and roundtrip might have a name assigned during the Arrow-to-Pandas conversion (e.g., from bq_field.name), this assertion may fail. Adding check_names=False ensures the test continues to focus on value equality, maintaining consistency with the previous equals() check.
| series = series.astype(roundtrip.dtype) | |
| pandas.testing.assert_series_equal(series, roundtrip) | |
| series = series.astype(roundtrip.dtype) | |
| pandas.testing.assert_series_equal(series, roundtrip, check_names=False) |
This PR removes the pandas version restrictions from tests that were skipped for Pandas >= 2.0 without a stated reason and enables testing by addressing a precision issue in the tests.
It also adds a catch for ImportError when tqdm tries to import matplotlib when matplotlib is not installed.
It also adds two small ignore pragmas to resolve mypy's struggles with handling optional imports.
This PR partially fixes #16042 (for bigquery).
For completeness, we did an assessment of any remaining test skips and they all appear to be legitimate skips that are applied when a dependency is not installed.