integration test suite

.github/workflows/integration.yml

@@ -0,0 +1,60 @@
+name: "Integration Tests"
+
+on:
+  push:
+    branches:
+      - 'main'
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  integration:
+    runs-on: ubuntu-latest
+    env:
+      GOPROXY: https://proxy.golang.org/,direct
+      GOPRIVATE: ""
+      GONOPROXY: ""
+      GONOSUMDB: github.com/github/*
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-go@v5
+        with:
+          go-version: ">=1.22"
+          check-latest: true
+
+      - name: Build gh-models binary
+        run: script/build
+
+      - name: Run integration tests (without auth)
+        working-directory: integration
+        run: |
+          go mod tidy
+          go test -v -timeout=5m
+        env:
+          # Explicitly unset any GitHub tokens to test unauthenticated behavior
+          GITHUB_TOKEN: ""
+          GH_TOKEN: ""
+
+      - name: Install gh CLI
+        if: github.event_name == 'workflow_dispatch'
+        run: |
+          curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | sudo dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg
+          echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | sudo tee /etc/apt/sources.list.d/github-cli.list > /dev/null
+          sudo apt update
+          sudo apt install gh
+
+      - name: Run integration tests (with auth)
+        if: github.event_name == 'workflow_dispatch' 
+        working-directory: integration
+        run: |
+          gh auth login --with-token <<< "${{ secrets.GITHUB_TOKEN }}"
+          go test -v -timeout=10m
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.gitignore

@@ -4,3 +4,6 @@
 /gh-models-linux-*
 /gh-models-windows-*
 /gh-models-android-*
+
+# Integration test dependencies
+integration/go.sum

DEV.md

@@ -34,6 +34,21 @@ make vet  # to find suspicious constructs
 make tidy # to keep dependencies up-to-date
 ```
 
+### Integration Tests
+
+In addition to unit tests, we have integration tests that use the compiled binary to test against live endpoints:
+
+```shell
+# Build the binary first
+script/build
+
+# Run integration tests
+cd integration
+go test -v
+```
+
+Integration tests are located in the `integration/` directory and automatically skip tests requiring authentication when no GitHub token is available. See `integration/README.md` for more details.
+
 ## Releasing
 
 When upgrading or installing the extension using `gh extension upgrade github/gh-models` or

integration/README.md

@@ -0,0 +1,76 @@
+# Integration Tests
+
+This directory contains integration tests for the `gh-models` CLI extension. These tests are separate from the unit tests and use the compiled binary to test actual functionality.
+
+## Overview
+
+The integration tests:
+- Use the compiled `gh-models` binary (not mocked clients)
+- Test basic functionality of each command (`list`, `run`, `view`, `eval`)
+- Are designed to work with or without GitHub authentication
+- Skip tests requiring live endpoints when authentication is unavailable
+- Keep assertions minimal to avoid brittleness
+
+## Running the Tests
+
+### Prerequisites
+
+1. Build the `gh-models` binary:
+   ```bash
+   cd ..
+   script/build
+   ```
+
+2. (Optional) Authenticate with GitHub CLI for full testing:
+   ```bash
+   gh auth login
+   ```
+
+### Running Locally
+
+From the integration directory:
+```bash
+go test -v
+```
+
+Without authentication, some tests will be skipped:
+```
+=== RUN   TestIntegrationHelp
+--- PASS: TestIntegrationHelp (0.05s)
+=== RUN   TestIntegrationList
+    integration_test.go:90: Skipping integration test - no GitHub authentication available
+--- SKIP: TestIntegrationList (0.04s)
+```
+
+With authentication, all tests should run and test live endpoints.
+
+## CI/CD
+
+The integration tests run automatically on pushes to `main` via the GitHub Actions workflow `.github/workflows/integration.yml`.
+
+The workflow:
+1. Builds the binary
+2. Runs tests without authentication (tests basic functionality)
+3. On manual dispatch, can also run with authentication for full testing
+
+## Test Structure
+
+Each test follows this pattern:
+- Check for binary existence (skip if not built)
+- Check for authentication (skip live endpoint tests if unavailable)
+- Execute the binary with specific arguments
+- Verify basic output format and success/failure
+
+Tests are intentionally simple and focus on:
+- Commands execute without errors
+- Help text is present and correctly formatted
+- Basic output format is as expected
+- Authentication requirements are respected
+
+## Adding New Tests
+
+When adding new commands or features:
+1. Add a corresponding integration test
+2. Follow the existing pattern of checking authentication
+3. Keep assertions minimal but meaningful
+4. Ensure tests work both with and without authentication

integration/go.mod

@@ -0,0 +1,11 @@
+module github.com/github/gh-models/integration
+
+go 1.22
+
+require github.com/stretchr/testify v1.10.0
+
+require (
+	github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
+	github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
+	gopkg.in/yaml.v3 v3.0.1 // indirect
+)