Skip to content

N+1 detector gem Prosopite (replacing gem Bullet)#6744

Merged
compwron merged 8 commits intomainfrom
prosopite-2
Mar 26, 2026
Merged

N+1 detector gem Prosopite (replacing gem Bullet)#6744
compwron merged 8 commits intomainfrom
prosopite-2

Conversation

@compwron
Copy link
Collaborator

@compwron compwron commented Mar 3, 2026

@github-actions github-actions bot added dependencies Pull requests that update a dependency file ruby Pull requests that update Ruby code labels Mar 3, 2026
@compwron compwron marked this pull request as ready for review March 20, 2026 18:29
@compwron compwron requested a review from Copilot March 20, 2026 18:29
@compwron compwron enabled auto-merge (rebase) March 20, 2026 18:29
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Replaces Bullet with Prosopite for N+1 query detection, adding configuration for development/test (and optional production toggles) plus RSpec integration and an ignore mechanism to support gradual rollout.

Changes:

  • Swap bullet for prosopite in dependencies and environment config.
  • Add a Rails initializer to configure Prosopite (middleware + core settings).
  • Add RSpec support to scan examples, pause during FactoryBot creation, and support an ignore list / TODO tracking.

Reviewed changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
spec/support/prosopite.rb Adds RSpec + FactoryBot integration and ignore-list logic for Prosopite scanning
spec/.prosopite_ignore Introduces ignore list file for incremental rollout
config/initializers/prosopite.rb Configures Prosopite middleware/settings after Rails initialization
config/environments/test.rb Enables Prosopite settings for test environment (replacing Bullet config)
config/environments/development.rb Enables Prosopite settings for development environment (replacing Bullet config)
config/environments/production.rb Adds env-driven Prosopite toggles for production
PROSOPITE_TODO.md Captures detected N+1 locations for follow-up work
Gemfile.lock Removes Bullet and adds Prosopite gem
Gemfile Replaces Bullet with Prosopite dependency (dev/test group)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +3 to +21
# Only enable Rack middleware if Prosopite is configured on
if Rails.configuration.respond_to?(:prosopite_enabled) && Rails.configuration.prosopite_enabled
require "prosopite/middleware/rack"
Rails.configuration.middleware.use(Prosopite::Middleware::Rack)
end

Rails.application.config.after_initialize do
# Core settings
Prosopite.enabled = Rails.configuration.respond_to?(:prosopite_enabled) &&
Rails.configuration.prosopite_enabled

# Minimum repeated queries to trigger detection (default: 2)
Prosopite.min_n_queries = Rails.configuration.respond_to?(:prosopite_min_n_queries) ?
Rails.configuration.prosopite_min_n_queries : 2

# Logging options
Prosopite.rails_logger = true # Log to Rails.logger
Prosopite.prosopite_logger = Rails.env.development? # Log to log/prosopite.log
end
Copy link

Copilot AI Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prosopite is referenced unconditionally inside after_initialize, but the gem is only in the :development, :test group. In production (and any env without the gem), this initializer will raise NameError during boot. Wrap the entire initializer (including the after_initialize block) in return unless defined?(Prosopite) / if defined?(Prosopite) or move the gem to a group that is bundled in the environments where this initializer runs (and handle the case where prosopite_enabled is true but the gem isn't available).

Copilot uses AI. Check for mistakes.
Comment on lines +60 to +68
def use_prosopite?(example)
# Explicit metadata takes precedence
return false if example.metadata[:disable_prosopite]
return true if example.metadata[:enable_prosopite]

# Check against ignore list
PROSOPITE_IGNORE.none? do |path|
File.fnmatch?("./#{path}/*", example.metadata[:rerun_file_path].to_s)
end
Copy link

Copilot AI Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use_prosopite? is defined at the top level, which adds a method to Object and can leak into other specs/helpers. Define it in a module (e.g., Support::Prosopite) and call it from the RSpec hook, or define it as a local lambda inside the RSpec.configure block to avoid global namespace pollution.

Copilot uses AI. Check for mistakes.

# Optional: Load ignore list from file for gradual rollout
PROSOPITE_IGNORE = if File.exist?("spec/.prosopite_ignore")
File.read("spec/.prosopite_ignore").lines.map(&:chomp).reject(&:empty?)
Copy link

Copilot AI Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ignore list loader keeps comment lines (starting with #) and doesn’t strip surrounding whitespace. That makes it easy to accidentally add a path like spec/features that never matches, and the file header comments will always be processed. Consider filtering out comment lines and trimming whitespace before matching.

Suggested change
File.read("spec/.prosopite_ignore").lines.map(&:chomp).reject(&:empty?)
File.read("spec/.prosopite_ignore").lines
.map { |line| line.strip }
.reject { |line| line.empty? || line.start_with?("#") }

Copilot uses AI. Check for mistakes.
@compwron compwron requested a review from Copilot March 23, 2026 15:08
compwron and others added 7 commits March 23, 2026 08:08
- Set Prosopite.raise = true so N+1s fail specs instead of silently logging
- Populate .prosopite_ignore with known-issue directories for gradual rollout
- Scope initializer to development only; spec/support/prosopite.rb owns test config
- Remove redundant prosopite settings from test.rb and production.rb
- Replace fragile FactoryBot::Strategy::Create monkey-patch with SyntaxRunner hook
- Filter comment lines in .prosopite_ignore parser

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 8 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +64 to +66
group :development, :test, :production do
gem "brakeman" # Security inspection
gem "bullet" # Detect and fix N+1 queries
gem "prosopite" # N+1 query detection via SQL pattern analysis
Copy link

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing this group to include :production will also bundle/require a large set of dev/test-only gems (e.g., rspec-rails, factory_bot_rails, linters) in production, increasing boot time/memory and potentially introducing production-only dependency conflicts. Keep the group as :development, :test and, if Prosopite truly needs to be available in production, place only prosopite in its own appropriate group (or mark it require: false) instead of pulling the entire dev/test toolchain into production.

Copilot uses AI. Check for mistakes.
Comment on lines +70 to +71
PROSOPITE_IGNORE.none? do |path|
File.fnmatch?("./#{path}/*", example.metadata[:rerun_file_path].to_s)
Copy link

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ignore matching here appears to use a nonstandard RSpec metadata key (:rerun_file_path), which is likely to be nil for most examples. That would prevent the ignore list from ever matching and cause Prosopite to run/raise in specs that are supposed to be ignored. Use the actual spec file path metadata key (e.g., :file_path/:absolute_file_path) when comparing against .prosopite_ignore.

Suggested change
PROSOPITE_IGNORE.none? do |path|
File.fnmatch?("./#{path}/*", example.metadata[:rerun_file_path].to_s)
spec_path = example.metadata[:absolute_file_path] ||
example.metadata[:file_path] ||
example.metadata[:rerun_file_path]
PROSOPITE_IGNORE.none? do |path|
File.fnmatch?("./#{path}/*", spec_path.to_s)

Copilot uses AI. Check for mistakes.
@compwron compwron requested a review from Copilot March 26, 2026 03:22
@compwron compwron changed the title WIP N+1 detector gem Prosopite (replacing gem Bullet) N+1 detector gem Prosopite (replacing gem Bullet) Mar 26, 2026
@compwron compwron disabled auto-merge March 26, 2026 03:23
@compwron compwron merged commit 8399953 into main Mar 26, 2026
13 checks passed
@compwron compwron deleted the prosopite-2 branch March 26, 2026 03:24
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 8 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

Gemfile:76

  • group :development, :test, :production pulls a large set of dev/test-only gems (brakeman, byebug, rspec-rails, rubocop, etc.) into production. This can bloat the production image and may introduce operational/security risk. Consider reverting this group back to :development, :test and, if Prosopite must be available in production, add only prosopite to a separate :production (or :development-only) group as appropriate.
group :development, :test, :production do
  gem "brakeman" # Security inspection
  gem "prosopite" # N+1 query detection via SQL pattern analysis
  gem "byebug", platforms: %i[mri mingw x64_mingw] # Debugger console
  gem "dotenv-rails" # Environment variable management
  gem "erb_lint", require: false # ERB linter
  gem "factory_bot_rails" # Test data factories
  gem "faker" # Creates realistic seed data, valuable for staging and demos
  gem "parallel_tests" # Run tests in parallel
  gem "pry" # Enhanced Ruby console
  gem "pry-byebug" # Pry debugger integration
  gem "rspec_junit_formatter" # JUnit XML formatter for RSpec
  gem "rspec-rails" # RSpec testing framework

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +69 to +72
# Check against ignore list
PROSOPITE_IGNORE.none? do |path|
File.fnmatch?("./#{path}/*", example.metadata[:rerun_file_path].to_s)
end
Copy link

Copilot AI Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignore matching is currently very brittle: it relies on example.metadata[:rerun_file_path], which may be unset for many runs, and it matches against a pattern prefixed with ./ (while RSpec file paths are typically spec/... or absolute). If :rerun_file_path is nil, the ignore list will never match and Prosopite will scan everything. Consider using example.metadata[:file_path] (or example.location) and normalizing paths before matching (e.g., via File.expand_path).

Copilot uses AI. Check for mistakes.
users.each { |u| u.orders.count }

# After (eager loading)
users.includes(:orders).each { |u| u.orders.count }
Copy link

Copilot AI Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "After (eager loading)" example still calls u.orders.count, which issues a separate COUNT(*) query per user even when the association is preloaded. To demonstrate fixing an N+1, this should use u.orders.size/length (or a counter cache / precomputed aggregate) instead of count.

Suggested change
users.includes(:orders).each { |u| u.orders.count }
users.includes(:orders).each { |u| u.orders.size }

Copilot uses AI. Check for mistakes.
Comment on lines +36 to +42
alias_method :original_create, :create

def create(*args, **kwargs, &block)
if defined?(Prosopite) && Prosopite.enabled?
Prosopite.pause { original_create(*args, **kwargs, &block) }
else
original_create(*args, **kwargs, &block)
Copy link

Copilot AI Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This FactoryBot monkey-patch can break under Spring: each spec run re-enters before(:suite) and re-aliases original_create to the already-wrapped create, which can lead to recursion/stack overflow. Guard the aliasing so it only happens once per process (e.g., check method_defined?) or prefer a Module#prepend wrapper that is idempotent.

Suggested change
alias_method :original_create, :create
def create(*args, **kwargs, &block)
if defined?(Prosopite) && Prosopite.enabled?
Prosopite.pause { original_create(*args, **kwargs, &block) }
else
original_create(*args, **kwargs, &block)
unless method_defined?(:original_create)
alias_method :original_create, :create
def create(*args, **kwargs, &block)
if defined?(Prosopite) && Prosopite.enabled?
Prosopite.pause { original_create(*args, **kwargs, &block) }
else
original_create(*args, **kwargs, &block)
end

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file ruby Pull requests that update Ruby code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants