feat(sentinel-graph): Add Microsoft Sentinel Graph API integration (public preview)#1088
Open
SinsBre wants to merge 16 commits into
Open
feat(sentinel-graph): Add Microsoft Sentinel Graph API integration (public preview)#1088SinsBre wants to merge 16 commits into
SinsBre wants to merge 16 commits into
Conversation
Implements a new plugin for querying Microsoft Sentinel Graph API (Microsoft
Security Platform) and visualizing graph data with Graphistry.
Key Features:
- Simple API following Kusto plugin pattern: configure_sentinel_graph() + sentinel_graph(query)
- Auto-converts API responses to Graphistry nodes/edges via defensive JSON parsing
- Supports multiple authentication methods:
- Service principal (tenant_id, client_id, client_secret)
- Interactive browser credential (default)
- Device code authentication
- Custom TokenCredential
- Production-ready security hardening:
- HTTPS enforcement with HTTP endpoint rejection
- SSL certificate verification (enabled by default)
- Sanitized error messages to prevent information disclosure
- Credentials and tokens never logged
- Query content not logged (could contain sensitive filters)
- Token storage with repr=False to prevent accidental exposure
- Robust error handling:
- HTTP retry with exponential backoff
- Configurable timeout and max retries
- Token caching with 5-minute expiry buffer
- Comprehensive test coverage (30+ unit tests)
Files Added:
- graphistry/plugins/sentinel_graph.py - Main plugin implementation
- graphistry/plugins_types/sentinel_graph_types.py - Type definitions and config
- graphistry/tests/plugins/test_sentinel_graph.py - Complete test suite
- demos/demos_databases_apis/microsoft/sentinel/sentinel_graph_examples.ipynb - Demo notebook
Files Modified:
- graphistry/client_session.py - Add sentinel_graph config property
- graphistry/plotter.py - Integrate SentinelGraphMixin
- setup.py - Add 'sentinel-graph' extras dependency
Example Usage:
import graphistry
from azure.identity import InteractiveBrowserCredential
g = graphistry.configure_sentinel_graph(
graph_instance='YourGraphInstance',
credential=InteractiveBrowserCredential()
)
viz = g.sentinel_graph('MATCH (n)-[e]->(m) RETURN * LIMIT 50')
viz.plot()
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Create comprehensive test fixture module to enable testing Sentinel Graph functionality without requiring live Azure credentials or actual threat intelligence data. This improves developer experience and enables faster test iteration. **What changed:** - Created `graphistry/tests/fixtures/` package with synthetic response data - Added `sentinel_graph_responses.py` with 9 fixture functions covering: - Minimal/simple graphs for basic testing - Duplicate node scenarios for deduplication logic - Malformed JSON for error handling validation - Empty responses for edge case coverage - Complex multi-type graphs for real-world simulation - Orphan edges, special characters, and null properties - Updated `test_sentinel_graph.py` to use fixtures instead of hardcoded constants - Reformatted notebook cells (Jupyter format standardization) **Benefits:** - Tests can run without Azure credentials or Sentinel Graph instance - Fixtures mimic actual API response structure (Graph.Nodes + RawData.Rows) - Easier to add new test cases by creating additional fixtures - Validates parsing logic across diverse response scenarios - All fixtures are JSON-serializable and structure-validated **Testing:** All 9 fixtures validated successfully with proper response structure. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
The Microsoft Sentinel Graph API returns fields with sys_* prefix (sys_sourceId, sys_targetId, sys_label) instead of the underscore prefix (_sourceId, _targetId, _label) that was originally expected. - Update node/edge extraction to detect both _* and sys_* field formats - Dynamically capture all properties from nodes and edges instead of hardcoding specific fields - Normalize key fields (id, label, source, target, edge) while preserving all original properties - Add test fixture mimicking actual Sentinel Graph API response format - Add tests for sys_* field format parsing
Enable cleaner API usage without requiring bind():
graphistry.configure_sentinel_graph('instance')
graphistry.sentinel_graph(query)
Changes:
- Add GraphistryClient wrapper methods for sentinel_graph functions
- Export sentinel_graph methods at module level in pygraphistry.py
- Re-export in __init__.py for public API access
- Update docstring examples to use module-level pattern
Security: No additional risk - module-level access uses same session
model as bind() pattern. Tokens and credentials remain protected.
Microsoft moved Sentinel custom graph to public preview with a new
response schema. Updates the plugin to match:
- Rewrite response parsing for new envelope: result.graph.{nodes,edges}
and result.rawData.tables (replacing the old Graph/RawData format)
- Add responseFormats request parameter (default: ["Graph"])
- Add sentinel_graph_list() to discover available graph instances via
GET /graphs/graph-instances?graphTypes=Custom
- Remove sys_* / JSON-encoded-string field handling (pre-preview only)
- Rewrite test fixtures and tests for new schema; add TestSentinelGraphList,
TestResponseFormats, and TestTableFormatParsing test classes
- Update demo notebook with list-then-configure pattern and responseFormats example
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
- Replace substring containment with startswith host-prefix check to satisfy CodeQL 'incomplete URL substring sanitization' (alert #9). An attacker-controlled URL like https://evil.com/api.securityplatform.microsoft.com/ would still pass the old 'in url' check; the startswith form anchors the host to position 0. - Remove unused 'result =' assignment in test_execute_query_retry_on_timeout (F841 — surfaced by python-lint-types CI; assertions only check retry call counts, not the return value). Both fixes are test-only; runtime sentinel_graph.py is unchanged.
# Conflicts: # setup.py
- Add empty 'outputs' and 'execution_count' to two code cells in sentinel_graph_examples.ipynb (cells 7 and 9). nbformat 4 requires both keys on every code cell; nbsphinx errored with AttributeError: outputs during RTD's sphinx-build, killing the docs build after the pytz import was resolved upstream. - Add azure.core.* and azure.identity to mypy.ini ignore_missing_imports. Mirrors the existing azure.kusto.* entry. Without these, mypy 1.20 flags the 'azure.core.credentials' and 'azure.identity' imports in sentinel_graph.py / sentinel_graph_types.py as missing stubs, failing python-lint-types CI on every Python version. Verified locally: ruff/mypy/pytest all green; nbformat.validate passes; 'import graphistry' loads cleanly.
Return the locally-bound token (typed 'str') instead of cfg._token (typed 'Optional[str]'). Functionally equivalent — token is assigned to cfg._token on the previous line — but mypy 1.14 (the pinned mypy on the Python 3.8 lockfile) does not narrow the field-access form and flagged: 'Incompatible return value type (got str | None, expected str)'. mypy 1.20+ (3.10+ lockfiles) accepted the original code, which is why the failure was 3.8-specific. Verified clean with both mypy 1.14 and 1.20.2; sentinel-graph tests still pass.
Contributor
|
@SinsBre ci is red |
Contributor
|
Autoreview, generally should work until |
Contributor
|
expanded autoreview PR #1088 Internal Review ReportTarget: #1088 SummaryPR #1088 adds a Microsoft Sentinel Graph connector, tests, fixtures, optional Wave Table
BlockersF1 — CI fails because Sentinel auth tests require optional Azure dependencySeverity: BLOCKER Evidence:
Expected fix direction: make auth tests independent of installed Azure packages via module injection/mocking, skip those tests when DRY/architecture note: Kusto already uses an optional-dependency guard pattern in ImportantF2 — Top-level
|
TestAuthenticationToken patches azure.identity.* but azure-identity is only installed via the optional sentinel-graph extra. The minimal CI job (test-minimal-python) runs the package with --no-deps, so the four auth tests failed with ModuleNotFoundError on Python 3.8 and 3.14. Mirror the optional-dep guard pattern used by test_kusto.py (graphistry/tests/test_kusto.py:7-12, :119, :236): try-import azure.identity at module load, expose HAS_AZURE_IDENTITY, and skipif the TestAuthenticationToken class when it's missing. Verified locally: - With azure available: 44 passed, 1 skipped (no behavior change for full-extras CI). - With azure blocked (sys.meta_path masker simulating minimal env): 40 passed, 5 skipped, 0 failed — the 4 previously-failing auth tests now skip cleanly. Addresses F1 (BLOCKER) in PR #1088 autoreview.
F2: graphistry.sentinel_graph_list() was defined as a GraphistryClient method and module-level alias in pygraphistry.py (2257-2259, 2805) but missing from graphistry/__init__.py's re-export block. As a result 'import graphistry; graphistry.sentinel_graph_list()' raised AttributeError despite being advertised as a top-level discovery API in the demo notebook. Add it next to the other sentinel_graph_* re-exports. F5: CONTRIBUTING.md:55-59 requires manual CHANGELOG.md updates for PRs. Add an 'Added' entry under [Development] covering the public API surface (configure / query / list / from_credential / close), auth methods supported, response-envelope parsing for the public-preview API, security hardening, the optional sentinel-graph install extra, test scaffolding, and the demo notebook. Verified: - python -c 'import graphistry; assert hasattr(graphistry, "sentinel_graph_list")' - ruff: all checks passed Addresses F2 and F5 in PR #1088 autoreview.
…n fallback
F3 — Provider properties named id/label/labels/source/target/edge no
longer silently overwrite Plottable bindings. _parse_graph_response()
now binds reserved graphistry columns (mirroring the kusto.py:343-352
convention):
Nodes: g_NodeId (was 'id')
g_label (was 'label')
g_labels (was 'labels')
Edges: g_src (was 'source')
g_dst (was 'target')
g_EdgeId (was 'id')
g_edge (was 'edge')
g_labels (was 'labels')
Extraction now spreads response.properties FIRST and overlays the
reserved binding fields LAST, so binding columns are always correct
and provider property keys are preserved verbatim under their own
names (no silent data loss either direction). Test assertions
updated for the rename; two new regression tests
(test_node_properties_do_not_overwrite_binding_columns,
TestEdgeBindingIsolation::test_edge_properties_do_not_overwrite_binding_columns)
pin the contract with an adversarial fixture whose properties dict
intentionally collides on all binding names.
F4 — DefaultAzureCredential fallback now also runs when the
interactive-browser path's get_token() raises, not only when its
constructor does. Headless/server environments where
InteractiveBrowserCredential constructs fine but cannot complete a
browser flow now reach the documented fallback instead of failing
with SentinelGraphConnectionError. Tracked via explicit
interactive_path flag (works with mocked credentials, doesn't
depend on isinstance against the real class). New regression test
test_default_azure_fallback_on_get_token_failure pins the contract.
CHANGELOG entry expanded to document the reserved-column public
contract and the DefaultAzureCredential token-acquisition fallback.
Verified:
- Full env (azure installed): 47 passed, 1 skipped
- Minimal env simulated (azure blocked): 42 passed, 6 skipped,
0 failed — F1's optional-dep guard still holds.
- ruff: all checks passed.
Addresses F3 and F4 in PR #1088 autoreview.
Newer mypy picked up via UV_EXCLUDE_NEWER cooldown flagged
'graphistry/_version.py:57: error: Need type annotation for
"HANDLERS"'. mypy.ini's exclude=_version filter is bypassed because
graphistry/__init__.py imports from _version, so mypy follows the
import and analyzes the file.
Add explicit 'HANDLERS: Any = {}' annotation matching the existing
'LONG_VERSION_PY : Any = {}' annotation one line above. Same precedent
as the prior fix(version): mypy commit (48cd352).
Local repro with 'uvx mypy@latest --config-file mypy.ini graphistry':
Success: no issues found in 259 source files.
Master-drift surfaced by PR #1088 CI; not caused by sentinel-graph
changes (the same _version.py contents passed 3.13 lint on 7576b0e
two weeks ago).
test-core-python runs pytest with -n auto (pytest-xdist). On Python 3.8 workers, the @patch('graphistry.plugins.sentinel_graph.requests.post') decorators failed with 'module graphistry has no attribute plugins' during dot-string resolution — even though 'from graphistry.plugins.sentinel_graph import SentinelGraphMixin' had run. Some xdist worker import modes (importlib-based) do not reliably set the parent-package attribute that 'from X.Y.Z import N' is supposed to set. Add an explicit 'import graphistry.plugins.sentinel_graph' statement alongside the existing from-import. The bare-import form uses __import__ semantics, which always sets the parent attribute, so the mock dotted-string lookup resolves cleanly under any import mode. Why it didn't surface before: - test-core-python's path filter classified prior commits as plugins-only, so the job stayed in 'skipping 0' status and never exercised these 12 tests under -n auto. - Once _version.py and graphistry/__init__.py landed in this stack, the path filter promoted test-core-python to run, and the latent xdist resolution issue surfaced. Verified locally with 'python -m pytest -n 2': 47 passed, 1 skipped. Addresses test-core-python failures on 3.8-3.14; not part of Leo's explicit autoreview list but a latent xdist-only issue this stack's broader file touch surface unmasked.
…lookup graphistry/__init__.py explicitly imports compute, pygraphistry, render, and arrow_uploader as subpackage attributes via 'from . import X as X' (see lines 137-141). The plugins subpackage was missing from that block. Symptom: under pytest-xdist parallel workers on Python 3.8 (test-core-python's 'pytest -n auto'), the mock dot-string resolver for @patch('graphistry.plugins.sentinel_graph.requests.post') failed with 'module graphistry has no attribute plugins' — even with explicit 'import graphistry.plugins.sentinel_graph' at the top of the test file. xdist's importlib-based import-mode doesn't always set the parent-package attribute that 'from X.Y import Z' or 'import X.Y' would normally set. Mock's _importer fallback to __import__ also didn't set it, hence the second getattr failed. Adding 'from . import plugins as plugins' to __init__.py makes graphistry.plugins a deterministic module attribute under every import mode, mirroring how the four sibling subpackages are already handled. 12 affected tests in TestQueryExecution / TestResponseFormats / TestSentinelGraphList now resolve cleanly. Also revert the speculative explicit submodule import in the test file from 6c3f84c — the real fix is in __init__.py and the test-file import added noise without addressing the root cause. Verified locally with 'python -m pytest -n 2': 47 passed, 1 skipped.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
SentinelGraphMixinplugin that integrates with the Microsoft Sentinel Graph REST API (now in public preview). Users can query their Sentinel custom graph instances using GQL and visualize the results directly in Graphistry.Features
sentinel_graph_list()— discover available instances viaGET /graphs/graph-instances?graphTypes=CustomresponseFormatsparameter — defaults to["Graph"]; pass["Table", "Graph"]to request both formats in a single callTokenCredential,DefaultAzureCredentialfallbackWhat this PR adds (new files)
graphistry/plugins/sentinel_graph.pySentinelGraphMixin— config, auth, query, list, parsegraphistry/plugins_types/sentinel_graph_types.pySentinelGraphConfigdataclass + error typesgraphistry/tests/plugins/test_sentinel_graph.pydemos/demos_databases_apis/microsoft/sentinel/sentinel_graph_examples.ipynbsetup.py[sentinel-graph]extra (azure-identity)Plus minimal wiring in
graphistry/__init__.py,graphistry/pygraphistry.py, andgraphistry/client_session.pyto exposeconfigure_sentinel_graph/sentinel_graph/sentinel_graph_list/sentinel_graph_close/sentinel_graph_from_credentialat the top-level API and per-Plotter.Notes for reviewers
result.graph.{nodes,edges}envelope). The pre-previewsys_*field format has been removed.Graphresponse format is preferred overTablefor Graphistry's use case — it gives the full connected subgraph rather than just the per-rowRETURNclause matches.@pytest.mark.integration) — they require live credentials.azure-identity. Install viapip install graphistry[sentinel-graph].Test plan
python -m pytest graphistry/tests/plugins/test_sentinel_graph.py -v— 44 unit tests pass, 1 integration test skippedsentinel_graph_list()returns correct instance metadataCI status (as of latest push)
test-gfql-core,test-minimal-python,test-docs): running for the first time after the master merge