diff --git a/.agents/skills/new-client-library-generator/SKILL.md b/.agents/skills/new-client-library-generator/SKILL.md new file mode 100644 index 000000000000..d333fdda1576 --- /dev/null +++ b/.agents/skills/new-client-library-generator/SKILL.md @@ -0,0 +1,124 @@ +--- +name: new-client-library-generator +description: Generates new Google Cloud Java client libraries by processing service information and updating the hermetic build configuration. Use this skill when tasked with onboarding a new service or version to the google-cloud-java repository. +--- + +# New Client Library Generator + +This skill automates the process of adding a new client library to the `google-cloud-java` repository using the hermetic build system. It retrieves service information from a Buganizer ticket (which links to a Service YAML file) and runs the configuration script to update `generation_config.yaml`. + +## Workflow + +### 1. Retrieve Service Information from Buganizer + +Use the available Buganizer MCP server to fetch the ticket content. The ticket will contain a link to a Service YAML file. Parse this YAML file to extract the fields below. + +> [!IMPORTANT] +> Not all fields in the Service YAML file may exist. You must ensure that all **Required** fields are identified and populated before executing the generation script. + +**Required:** +- `api_shortname`: Unique service identifier (e.g., `alloydb`). +- `name_pretty`: Human-friendly name (e.g., `AlloyDB API`). +- `proto_path`: Versioned path to protos (e.g., `google/cloud/alloydb/v1`). Must include the version component — root-level paths like `google/cloud/alloydb` are not supported. +- `product_docs`: Product documentation URL (must start with `https://`). +- `api_description`: First sentence of the service summary. + +**Optional:** +- `rest_docs`: REST reference documentation URL. +- `rpc_docs`: RPC/proto reference documentation URL. +- `library_name`: Override the default `java-` directory name. +- `distribution_name`: Override Maven coordinates (default: `com.google.cloud:google-cloud-`). + +For the field-to-flag mapping, see [references/service_yaml_mapping.md](references/service_yaml_mapping.md) (Service YAML fields → script flags). + +### 2. Check for Conflicts + +Before running the script, verify `api_shortname` is not already in use: + +```bash +grep "api_shortname: " generation_config.yaml +``` + +If a conflict exists, determine a unique name or use `--library-name` to set a distinct directory. See [references/generation_guide.md](references/generation_guide.md) for examples. + +### 3. Special Cases + +Some APIs require non-default Maven coordinates and `api_shortname` values: + +| Proto path prefix | `--api-shortname` | `--distribution-name` | +|---------------------|------------------------------|----------------------------------------------------------| +| `google/maps/*` | `maps-` | `com.google.maps:google-maps-` | +| `google/shopping/*` | `shopping-` | `com.google.shopping:google-shopping-` | + +where `` is the value from the Service YAML. + +### 4. Execution + +#### Adding a new library + +If the library does not exist yet, run the script with the gathered information: + +```bash +python3 generation/new_client_hermetic_build/add-new-client-config.py add-new-library \ + --api-shortname="[API_SHORTNAME]" \ + --name-pretty="[NAME_PRETTY]" \ + --proto-path="[PROTO_PATH]" \ + --product-docs="[PRODUCT_DOCS]" \ + --api-description="[API_DESCRIPTION]" \ + [OPTIONAL_FLAGS] +``` + +The script modifies `generation_config.yaml` and sorts the `libraries` list alphabetically. + +To see all available flags: +```bash +python3 generation/new_client_hermetic_build/add-new-client-config.py add-new-library --help +``` + +#### Adding a new version to an existing library + +If the client library module already exists and the request is to add a new version, do NOT run the script. Instead, manually add the corresponding `proto_path` for the new version to the `GAPICs` list of the existing library entry in `generation_config.yaml`. + +Example entry update: +```yaml +- api_shortname: myapi + ... + GAPICs: + - proto_path: google/cloud/myapi/v1 + - proto_path: google/cloud/myapi/v2 # Manually added +``` + +### 5. Verification + +After execution or manual update: +1. Confirm `generation_config.yaml` has a new or updated entry with the correct fields. See [references/generation_config_schema.md](references/generation_config_schema.md). +2. Confirm `proto_path` under `GAPICs` includes a version component (e.g., `v1`, `v1beta`, `v1alpha`). +3. Confirm `product_documentation` starts with `https://`. + +### 6. Create a Branch and PR + +After verifying the changes: + +```bash +git checkout -b "new-library/[API_SHORTNAME]" +git add generation_config.yaml +git commit -m "feat: new module for [API_SHORTNAME]" +``` + +Then open a pull request. The Pull Request body must only contain the `add-new-library` command used. + +**PR Body Template:** + +```text +Command used: + +python3 generation/new_client_hermetic_build/add-new-client-config.py add-new-library \ + --api-shortname="[API_SHORTNAME]" \ + --name-pretty="[NAME_PRETTY]" \ + --proto-path="[PROTO_PATH]" \ + --product-docs="[PRODUCT_DOCS]" \ + --api-description="[API_DESCRIPTION]" \ + [OPTIONAL_FLAGS] +``` + +The hermetic library generation workflow will be triggered automatically upon changes to `generation_config.yaml`. diff --git a/.agents/skills/new-client-library-generator/references/generation_config_schema.md b/.agents/skills/new-client-library-generator/references/generation_config_schema.md new file mode 100644 index 000000000000..075db9054032 --- /dev/null +++ b/.agents/skills/new-client-library-generator/references/generation_config_schema.md @@ -0,0 +1,57 @@ +# generation_config.yaml Schema + +The `generation_config.yaml` file at the root of the repository controls the generation of all client libraries. + +## Key Fields + +- `gapic_generator_version`: The version of the GAPIC generator used globally (unless overridden). +- `googleapis_commitish`: The commit of the `googleapis` repository used as the source for protos. +- `libraries`: A list of library configurations. + +### Library Configuration Fields + +**Required (always present):** +- `api_shortname`: (String) Identifier (e.g., `alloydb`). +- `name_pretty`: (String) Display name (e.g., `AlloyDB API`). +- `product_documentation`: (URL) Product documentation URL. +- `api_description`: (String) Service description. +- `client_documentation`: (URL) Auto-generated link to the generated client docs. +- `release_level`: (String) Usually `preview` for new libraries. +- `distribution_name`: (String) Maven coordinates (e.g., `com.google.cloud:google-cloud-alloydb`). +- `api_id`: (String) API identifier (e.g., `alloydb.googleapis.com`). +- `library_type`: (String) Usually `GAPIC_AUTO`. +- `group_id`: (String) Maven group ID (e.g., `com.google.cloud`). +- `cloud_api`: (Boolean) `true` if distribution name starts with `google-cloud-`. +- `GAPICs`: (List) List of proto paths to generate from. + - `proto_path`: (String) Path to versioned protos (must include version, e.g., `google/cloud/alloydb/v1`). + +**Optional:** +- `library_name`: (String) Override the output directory name (without `java-` prefix). +- `rest_documentation`: (URL) REST reference documentation URL. +- `rpc_documentation`: (URL) RPC/proto reference documentation URL. +- `requires_billing`: (Boolean) Whether billing is required (default: `true`). +- `api_reference`: (URL) API reference documentation link. +- `codeowner_team`: (String) GitHub team responsible for this library. +- `googleapis_commitish`: (String) Pin to a specific `googleapis/googleapis` commit (overrides repo-level setting). +- `issue_tracker`: (URL) Issue tracker for this library. +- `extra_versioned_modules`: (String) Extra modules managed via `versions.txt`. +- `excluded_dependencies`: (String) Comma-separated dependencies excluded from postprocessing. +- `excluded_poms`: (String) Comma-separated pom files excluded from postprocessing. + +## Example Library Entry + +```yaml +- api_shortname: alloydb + name_pretty: AlloyDB API + product_documentation: https://cloud.google.com/alloydb/docs + api_description: AlloyDB for PostgreSQL is an open source-compatible database service. + client_documentation: https://cloud.google.com/java/docs/reference/google-cloud-alloydb/latest/overview + release_level: preview + distribution_name: com.google.cloud:google-cloud-alloydb + api_id: alloydb.googleapis.com + library_type: GAPIC_AUTO + group_id: com.google.cloud + cloud_api: true + GAPICs: + - proto_path: google/cloud/alloydb/v1 +``` diff --git a/.agents/skills/new-client-library-generator/references/generation_guide.md b/.agents/skills/new-client-library-generator/references/generation_guide.md new file mode 100644 index 000000000000..486c4c20253f --- /dev/null +++ b/.agents/skills/new-client-library-generator/references/generation_guide.md @@ -0,0 +1,75 @@ +# New Client Library Generation Guide + +This guide describes how to generate a new client library by: +1. Appending a new library configuration to `generation_config.yaml` using the provided scripts. +2. Relying on the automated GitHub Actions to generate the code once the configuration is pushed in a Pull Request. + +## Components + +### `generation/new_client_hermetic_build/add-new-client-config.py` +This script updates `generation_config.yaml` with the necessary metadata for a new library. It takes several arguments that map to fields in the configuration file. + +## Local Execution + +To add a new library configuration, you must run the `add-new-client-config.py` script. The parameters for this script are typically found in the **Service YAML** file linked in the Buganizer request. + +### Basic Usage + +```bash +python3 generation/new_client_hermetic_build/add-new-client-config.py add-new-library \ + --api-shortname="[API_SHORTNAME]" \ + --name-pretty="[NAME_PRETTY]" \ + --proto-path="[PROTO_PATH]" \ + --product-docs="[PRODUCT_DOCS]" \ + --api-description="[API_DESCRIPTION]" +``` + +### Parameter Details + +#### API Short Name (`--api-shortname`) +The unique identifier for the library (e.g., `alloydb`). This is used to determine the directory name and default artifact name. +- **Source:** `api_short_name` in the Service YAML. + +#### Proto Path (`--proto-path`) +The path from the root of the `googleapis` repository to the directory containing the versioned proto definitions. +- **Example:** `google/cloud/alloydb/v1beta` +- **Note:** Root-level paths like `google/cloud/alloydb` are not supported. + +#### Name Pretty (`--name-pretty`) +The human-friendly name of the API. +- **Source:** `title` in the Service YAML. +- **Example:** `AlloyDB API` + +#### Product Docs (`--product-docs`) +The URL for the product documentation. Must start with `https://`. +- **Source:** `documentation_uri` in the Service YAML. + +#### API Description (`--api-description`) +A concise summary of the API for the README. +- **Source:** `documentation.summary` or `documentation.overview` in the Service YAML. Use the first sentence. + +## Prerequisites + +Ensure you have the following set up in your local environment: + +1. **Python 3.9+**: The scripts require Python 3.9 or higher. +2. **Dependencies**: Install the required Python packages from the root of the repository: + ```bash + pip install -r generation/new_client_hermetic_build/requirements.txt + ``` + +## Advanced Options + +For full control over the generation (e.g., overriding Maven coordinates or library names), run the script with the `--help` flag: + +```bash +python3 generation/new_client_hermetic_build/add-new-client-config.py add-new-library --help +``` + +### Special Case: Non-Cloud APIs +Some libraries (like Google Maps or Shopping) require special handling for Maven coordinates. + +| API Path Prefix | `--api-shortname` | `--distribution-name` | +|---|---|---| +| `google/maps/*` | `maps-` | `com.google.maps:google-maps-` | +| `google/shopping/*` | `shopping-` | `com.google.shopping:google-shopping-` | diff --git a/.agents/skills/new-client-library-generator/references/service_yaml_mapping.md b/.agents/skills/new-client-library-generator/references/service_yaml_mapping.md new file mode 100644 index 000000000000..f865062aee11 --- /dev/null +++ b/.agents/skills/new-client-library-generator/references/service_yaml_mapping.md @@ -0,0 +1,41 @@ +# Service YAML Mapping to Generation Script Parameters + +This document explains how to extract information from a Service YAML file (linked in a Buganizer ticket) to populate the parameters for `generation/new_client_hermetic_build/add-new-client-config.py`. + +| Service YAML Field | Script Parameter (`--flag`) | Description | +|---|---|---| +| `api_short_name` | `api-shortname` | Unique identifier for the service (e.g., `alloydb`). | +| (extracted from proto) | `proto-path` | Path from the root of the googleapis repository to the versioned proto directory (e.g., `google/cloud/alloydb/v1alpha`). Root-level paths like `google/cloud/alloydb` are not supported. | +| `title` | `name-pretty` | Human-friendly name (e.g., `AlloyDB API`). | +| `documentation_uri` | `product-docs` | Product documentation URL. Must start with `https://`. | +| `rest_reference_documentation_uri` | `rest-docs` | REST documentation URL. Optional. | +| `proto_reference_documentation_uri` | `rpc-docs` | RPC/Proto documentation URL. Optional. | +| `documentation.summary` or `documentation.overview` | `api-description` | Concise summary of the service. Use the first sentence. | + +## Example Mapping + +**Service YAML File excerpt:** +```yaml +title: "Discovery Engine API" +api_short_name: "discoveryengine" +documentation_uri: "https://cloud.google.com/generative-ai-app-builder/docs/introduction" +rest_reference_documentation_uri: "https://cloud.google.com/generative-ai-app-builder/docs/reference/rest" +proto_reference_documentation_uri: "https://cloud.google.com/generative-ai-app-builder/docs/reference/rpc" +documentation: + summary: "Discovery Engine for Search and Recommendations" +``` + +**Protos in `googleapis` repository:** +`google/cloud/discoveryengine/v1/` + +**Generated Command:** +```bash +python3 generation/new_client_hermetic_build/add-new-client-config.py add-new-library \ + --api-shortname="discoveryengine" \ + --name-pretty="Discovery Engine API" \ + --proto-path="google/cloud/discoveryengine/v1" \ + --product-docs="https://cloud.google.com/generative-ai-app-builder/docs/introduction" \ + --rest-docs="https://cloud.google.com/generative-ai-app-builder/docs/reference/rest" \ + --rpc-docs="https://cloud.google.com/generative-ai-app-builder/docs/reference/rpc" \ + --api-description="Discovery Engine for Search and Recommendations" +```