Merge pull request #2244 from elementary-data/release/v0.24.0#2246
Closed
github-actions[bot] wants to merge 803 commits into
Closed
Merge pull request #2244 from elementary-data/release/v0.24.0#2246github-actions[bot] wants to merge 803 commits into
github-actions[bot] wants to merge 803 commits into
Conversation
update dbt package revision
…clickhouse Handle empty result in clickhouse
…ifact-upload Handle invalid characters when uploading CI artifact
…ult_rows Update DBT package version
* change for model runs * change for source freshness results * change for test results
- Introduced a new function to append an attribution block to alert messages. - Updated the alert message construction to include the attribution block, enhancing message content with a link to Elementary. - Refactored imports for better organization and clarity.
…tion-in-synced-schema Updated DBT pacakge revision
…entary-in-oss-alerts Add attribution block to alert messages in data monitoring
* add dbt fusion runner * remove call_macro * fixes * bugfix * bugfix - parent dataclass can't have defaults * replace dbt.create_table_as with our implementation * don't run deps from the runner for dbt fusion
* add the former deprecated tests project from the package to the CLI as the e2e project * e2e project models: remove stage ifs * remove irrelevant macros * test-warehouse: change CI to use a static dbt project rather than the deprecated package tests * fixes * bugfix * add deps * test-warehouse: fix packages for e2e project * forgot to update packages * generate data before seeding * full refresh seeds
* update the CLI to use the new package version * update package lock
Replaces the tabulate-based ASCII table in code blocks with Slack's native Table Block (type: "table"), giving column-aligned, scannable tables directly in alert messages. - Bold rich_text header row, raw_text data cells - Falls back to JSON code block for >20 columns (Slack limit) or if a second table would appear in the same message - Removes unused tabulate import and cell-truncation helpers from BlockKitBuilder - Regenerates all block_kit test fixtures to match the new format Closes #2225
Slack rejects raw_text cells with empty text. Convert None to "NULL" (represents database NULL values) and guard against any other value that stringifies to an empty string with a single-space fallback. Adds a dedicated test to assert no raw_text cell ever has empty text.
Use native Slack Table Block for test results sample rendering
Adds an explicit `urllib3>=2.7.0,<3.0.0` constraint to pyproject.toml to address two open Dependabot high-severity alerts. urllib3 is a transitive dep (via `requests` and `boto3`), and without a lock file this constraint is the canonical way to ensure consumers install the fixed version. CVEs addressed: - GHSA-mf9v-mfxr-j63j: Decompression-bomb safeguards bypassed in parts of the streaming API (vulnerable >=2.6.0,<2.7.0) - urllib3: Sensitive headers forwarded across origins in proxied low-level redirects (vulnerable >=1.23,<2.7.0) [run-e2e] Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Thread a new disable_samples parameter from ReportAPI.get_report_data through TestsAPI -> TestsFetcher -> the get_test_results dbt macro. When disable_samples=true, the macro skips the get_result_rows_agate query against test_result_rows entirely instead of relying on the caller to clear sample_data after the SQL has already run. This is consumed by elementary-internal's cloud report generation, where results_sample has zero consumers (522 GB / 249.6M rows scanned per cycle for nothing). The default remains false so OSS users and other callers are unaffected. Co-Authored-By: mika@elementary-data.com <mika.kerman@gmail.com>
…ults When disable_samples=true, ensure no sample data leaks via the legacy elementary_test_results.result_rows JSON column. Previously only the test_result_rows table query was skipped; get_test_rows_sample would still return data from the legacy column. Thread disable_samples into _process_raw_test_results and gate the entire get_test_rows_sample call so disable_samples=true means no sample data at all. Co-Authored-By: mika@elementary-data.com <mika.kerman@gmail.com>
Thread disable_samples into current_tests_run_results_query and the clickhouse inline query so the legacy elementary_test_results.result_rows column is replaced with null/empty when samples are disabled. Avoids materializing the column into the temp table and serializing it through agate. Co-Authored-By: mika@elementary-data.com <mika.kerman@gmail.com>
…ization Separates the macro-level DB optimization from the existing OSS Python-level disable_samples mask: - skip_test_result_rows (new, macro-level): pure DB optimization. Skips the test_result_rows table query, gates get_test_rows_sample, selects null/'' as result_rows. Affects ALL test types. Used by cloud only. - disable_samples (unchanged, Python-level): masks sample_data for dbt_test rows only. Used by OSS --disable-samples CLI flag for PII protection. Anomaly metrics are preserved. Co-Authored-By: mika@elementary-data.com <mika.kerman@gmail.com>
…results Pre-existing bug surfaced by CodeRabbit: the ClickHouse variant of get_test_results passes elementary_tests_allowlist_status into _process_raw_test_results without ever initializing it. The default__ and fabric__ variants both set it from disable_passed_test_metrics at the top of the macro. Without this, sample selection for non-dbt_test rows (anomaly/schema_change) would silently skip on ClickHouse because the 'status in elementary_tests_allowlist_status' check would fail on an undefined value. Co-Authored-By: mika@elementary-data.com <mika.kerman@gmail.com>
Co-Authored-By: mika@elementary-data.com <mika.kerman@gmail.com>
…samples-macro-skip APP-1123: add skip_test_result_rows to skip test_result_rows query in get_test_results macro
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
…ormed Unicode) idna is a transitive dependency via requests. Pinning to >=3.15,<4 ensures the fix for CVE-2025-46816 is installed for all downstream users. Co-Authored-By: Noy Arie <noyarie1992@gmail.com>
Co-Authored-By: Noy Arie <noyarie1992@gmail.com>
dbt-sqlserver no longer includes fabric in its adapter dispatch chain, so fabric__ macros in elementary_cli are not picked up for sqlserver targets. Add sqlserver__ delegates for get_test_results and get_adapter_unique_id. Co-authored-by: Cursor <cursoragent@cursor.com>
Address review feedback — the explicit sqlserver__ macro makes the delegation path self-evident. Co-authored-by: Cursor <cursoragent@cursor.com>
…dabot-alert fix: pin idna>=3.15 to address medium Dependabot alert (SSRF via malformed Unicode)
…er-cli-dispatch fix: add explicit sqlserver__ dispatches in elementary_cli (CORE-877)
CORE-877 Co-authored-by: Cursor <cursoragent@cursor.com>
…ate-packages-yml chore: update dbt-data-reliability to v0.24.0 (CORE-877)
CORE-877 Co-authored-by: Cursor <cursoragent@cursor.com>
…-report-bundle chore: add report bundle 1.0.33 (CORE-877)
Co-authored-by: Cursor <cursoragent@cursor.com>
release/v0.24.0
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| @staticmethod | ||
| def run(cmd: list[str], check: bool = True, **kw: object) -> subprocess.CompletedProcess: # type: ignore[type-arg] | ||
| """Run a command, printing it first.""" | ||
| print(f" -> {shlex.join(cmd)}") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
release/v0.24.0