Fix double-nested regexp key in Elasticsearch Advanced Filter queries

[Copilot]

Describe your changes:

RegEx conditions in Advanced Filter generated malformed Elasticsearch queries with double-nested "regexp" keys, causing x_content_parse_exception errors.

Root cause: The elasticSearchFormatValue function for the regexp operator returned { regexp: { [field]: {...} } }, which buildEsRule then wrapped again with { regexp: {...} }, creating invalid double-nesting.

Fix: Remove the extra wrapper to match the pattern used by other operators (like, equal). Now returns { [field]: {...} } directly.

Before:

case 'regexp':
  return {
    regexp: {
      [fieldName]: { value: newValue, case_insensitive: true },
    },
  };

After:

case 'regexp':
  return {
    [fieldName]: { value: newValue, case_insensitive: true },
  };

This produces valid Elasticsearch queries:

{
  "regexp": {
    "databaseSchema.displayName.keyword": {
      "value": "gold.*",
      "case_insensitive": true
    }
  }
}

Added comprehensive test coverage for all text widget operators including explicit verification that regexp does not double-nest.

Type of change:

• Bug fix

Checklist:

• I have read the CONTRIBUTING document.

• My PR title is Fixes <issue-number>: <short explanation>

• I have commented on my code, particularly in hard-to-understand areas.

• For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

• I have added a test that covers the exact scenario we are fixing. For complex issues, comment the issue number in the test for future reference.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• iojs.org
  • Triggering command: /usr/bin/curl curl -q --fail --compressed -L -s REDACTED -o - (dns block)
• www.antlr.org
  • Triggering command: /usr/bin/curl curl REDACTED (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

Problem

When using a RegEx condition in the Advanced Filter, OpenMetadata generates an incorrect Elasticsearch query structure with a double-nested "regexp" key, resulting in the following error:

Search failed due to Request failed: [x_content_parse_exception]
[1:93] [bool] failed to parse field [must]

Current Behavior

The system currently generates this incorrect query payload:

{
  "query": {
    "bool": {
      "must": [
        {
          "bool": {
            "must": [
              {
                "regexp": {
                  "regexp": {
                    "databaseSchema.displayName.keyword": {
                      "value": "gold.*",
                      "case_insensitive": true
                    }
                  }
                }
              }
            ]
          }
        }
      ]
    }
  }
}

Expected Behavior

The query should be structured as:

{
  "query": {
    "bool": {
      "must": [
        {
          "bool": {
            "must": [
              {
                "regexp": {
                  "databaseSchema.displayName.keyword": {
                    "value": "gold.*",
                    "case_insensitive": true
                  }
                }
              }
            ]
          }
        }
      ]
    }
  }
}

Root Cause

The bug is in openmetadata-ui/src/main/resources/ui/src/utils/AdvancedSearchClassBase.ts at lines 130-135.

The elasticSearchFormatValue function for the text widget's regexp operator case returns:

case 'regexp':
  return {
    regexp: {  // ← Extra nested "regexp" key causing the issue
      [fieldName]: { value: newValue, case_insensitive: true },
    },
  };

This creates a double-nested structure because the buildEsRule function in QueryBuilderElasticsearchFormatUtils.js (line 318) already wraps the parameters with [queryType] (which is "regexp"):

mainQuery = {
  [queryType]: { ...parameters },  // ← queryType is "regexp", wrapping the already nested object
};

Solution

Remove the extra regexp wrapper from the return value in AdvancedSearchClassBase.ts. The function should return just the field and its parameters, letting buildEsRule handle the wrapping.

Change:

case 'regexp':
  return {
    regexp: {
      [fieldName]: { value: newValue, case_insensitive: true },
    },
  };

To:

case 'regexp':
  return {
    [fieldName]: { value: newValue, case_insensitive: true },
  };

Files to Fix

• openmetadata-ui/src/main/resources/ui/src/utils/AdvancedSearchClassBase.ts (line 130-135)

Testing

After the fix, verify that:

1. RegEx conditions in Advanced Filter generate correct Elasticsearch queries
2. Elasticsearch accepts the query without x_content_parse_exception errors
3. RegEx searches return expected results
4. Other operators (like, not_like, equal, etc.) continue to work correctly

This pull request was created from Copilot chat.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[gitar-bot]

Code Review ✅ Approved

Clean, surgical fix for double-nested regexp query structure bug with comprehensive test coverage.

What Works Well

The fix is minimal and consistent with how other operators handle the return value. The test suite comprehensively covers all text widget operators and includes a specific assertion to prevent regression of the double-nesting issue.

Options

Auto-apply is off Gitar will not commit updates to this branch.
Display: compact Hiding non-applicable rules.

Comment with these commands to change:

Auto-apply	Compact
gitar auto-apply:on	gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | This comment will update automatically (Docs)}