Connect Webview Funnel & Error Analysis Dashboard

Overview

This dashboard provides comprehensive analysis of connect webview performance, drop-off points, errors, and conversion metrics across all providers.

Quick Stats (Last 30 Days - Verified)

• Total Webviews: ~43,000
• Overall Success Rate: ~15%
• Most Common Drop-off: brand_select_pane (11,700 webviews)
• Fastest Authorizations: 45% complete in < 30 seconds

Critical Issues Identified

1. Brivo Integration: 97% failure rate - requires immediate investigation
2. Brand Selection Abandonment: 27% of users never select a provider
3. Kwikset Login Issues: 87% abandonment at login pane
4. Generic Error Messages: 99.7% of errors show as "unknown_error"

Important: Funnel Analysis

⚠️ Different providers have different authentication flows!

• OAuth providers (schlage, august): use redirect_pane
• Credential providers (kwikset, yale): use login_pane
• Fields providers (wyze, lockly): use fields_pane

Three funnel approaches:

1. Mixed Funnel (02): Quick overview, combines all flows
2. Per-Provider Funnel (02b): ⭐ Most accurate, filter by provider
3. Abstract Funnel (02c): Universal stages, works across all providers

📖 Read FUNNEL_GUIDE.md for detailed explanation and best practices

Implementation Guide

Prerequisites

• Access to Metabase instance
• Database connection to seam database already configured in Metabase
• Permissions to create questions and dashboards

Step 1: Create Questions (Queries)

Navigate to the queries/ directory and create each query as a "Question" in Metabase:

1. 01_kpi_metrics.sql - Overall funnel metrics (Number visualization)
2. 02_funnel_stages.sql - Mixed funnel (all providers) ⚠️ See FUNNEL_GUIDE.md
   • 02b_funnel_by_provider.sql - Per-provider funnel ⭐ RECOMMENDED
   • 02c_funnel_by_flow_type.sql - Abstract universal funnel
3. 03_top_errors.sql - Top errors by count (Table)
4. 04_provider_performance.sql - Provider comparison (Table + Bar chart)
5. 05_daily_trends.sql - Daily trend analysis (Line chart)
6. 06_event_analysis.sql - Event analysis (Table)
7. 07_failed_panes.sql - Failed webviews by pane (Table)
8. 08_time_to_authorize.sql - Time-to-authorize by provider (Table)
9. 09_abandoned_webviews.sql - Abandoned webviews by stage (Table)
10. 10_authorization_time_dist.sql - Authorization time distribution (Bar chart)
11. 11_errors_by_provider.sql - Errors by provider (Table)

Step 2: Set Up Parameters

Create these parameters in Metabase:

1. date_from: Date parameter, default: relative_datetime(-30, 'day')
2. date_to: Date parameter, default: relative_datetime(0, 'day')
3. min_attempts: Number parameter, default: 50

Step 3: Create Dashboard

1. Create a new dashboard named "Connect Webview Funnel & Error Analysis"
2. Add questions in this layout:

Row 1: KPI Cards (Full Width)

• Add question #1 (KPI Metrics)
• Configure as 5 separate number cards:
  • Total Webviews
  • Successful
  • Failed
  • Pending/Abandoned
  • Success Rate %

Row 2: Funnel Visualization (Full Width)

• Add question #2 (Funnel Stages)
• Visualize as Funnel Chart

Row 3: Two Columns

• Left: Question #3 (Top Errors) - Table
• Right: Question #11 (Errors by Provider) - Table

Row 4: Provider Performance (Full Width)

• Add question #4 (Provider Performance)
• Create two visualizations:
  1. Table showing all metrics
  2. Bar chart showing success_rate by provider

Row 5: Trends (Full Width)

• Add question #5 (Daily Trends)
• Line chart with dual Y-axes (volume + success rate)

Row 6: Three Columns

• Left: Question #7 (Failed Panes)
• Middle: Question #9 (Abandoned Webviews)
• Right: Question #10 (Authorization Time Distribution)

Row 7: Two Columns

• Left: Question #8 (Time to Authorize by Provider)
• Right: Question #6 (Event Analysis)

Step 4: Add Dashboard Filters

1. Click "Add Filter" → "Time" → Connect to all questions
2. Click "Add Filter" → "Text or Category" → Label: "Provider" → Connect to selected_provider field
3. Default date range: Last 30 days

Step 5: Add Documentation Cards

Add markdown text cards between sections:

1. Above KPIs: Explain what the success/failure/pending statuses mean
2. Above Funnel: Explain the different pane stages
3. Above Errors: Note about generic "unknown_error" issue
4. Above Provider Performance: How to interpret success vs failure vs abandonment rates

Step 6: Configure Auto-Refresh

1. Dashboard settings → Auto-refresh: 30 minutes
2. Save dashboard

Using the Dashboard

Key Metrics to Monitor

1. Success Rate Trends: Watch for sudden drops indicating integration issues
2. Provider Performance: Identify which providers need attention
3. Error Messages: Look for spikes in specific error types
4. Abandonment Points: Find where users drop off most frequently

Interpreting Results

High Pending Count: Users started but didn't complete - may indicate UX issues or confusion

High Failure Rate: Integration or authentication problems - requires engineering investigation

Brand Select Pane Abandonment: Users aren't seeing their desired provider or are confused by options

Long Authorization Times: May indicate API slowness or complex authentication flows

Recommended Actions

1. Daily: Check KPI metrics for anomalies
2. Weekly: Review provider performance table for new issues
3. Monthly: Analyze trends to identify long-term patterns
4. Ad-hoc: Investigate specific providers when alerted to issues

Query Optimization Notes

• All queries are optimized for recent data (typically 30-90 days)
• Queries use indexes on created_at, workspace_id, and selected_provider
• Event analysis query may be slow for large date ranges - limit to 7-30 days

Maintenance

• Review and update error categorization quarterly
• Add new providers to provider list as they're onboarded
• Monitor query performance and add indexes if needed

Support

For issues with:

• Dashboard display: Contact Metabase admin
• Data accuracy: Contact Data Engineering team
• Connect webview issues: Contact Platform team