Test Prof Versions Save

Changes

Make let_it_be and before_all work with include_context in RSpec.configure.

Fixes #117

Features

• Add JSON report support for StackProf.

Use TEST_STACK_PROF_FORMAT=json to generate a JSON dump (e.g. to use with https://www.speedscope.app)

• Add ability to specify report/artifact name suffixes.

Use TEST_PROF_REPORT env variable to add custom suffix to generated reports (to distinguish runs from each other).

Features

• Added header with the general factories usage stats to FactoryProf report (PR):

[TEST PROF INFO] Factories usage

# new is added
Total: 12321
Total top-level: 3234
Total uniq factories: 42

# then goes the existing table
total       top-level            name
# ...

• Added event time percentage to EventProf report (PR):

[TEST PROF INFO] EventProf results for factory.create

Total time: 03:07.353 of 07:25:232 (42.02%)
Total events: 7459

Something (./my_spec.rb:21) – 09:16.100 (7 / 3) of 12:29:233 (74.13%)

Improvements

• Test sampling now allows to sample examples and preserver RSpec filters (PR):

# runs 10 random test examples
SAMPLE=10 rake test

# runs 10 random example groups
SAMPLE_GROUPS=10 rake test

# run 10 random examples with filtering
SAMPLE=10 rspec --tag slow

From Cult of Martians.

Ruby 2.3+ is required

Paris.rb 2018 special.

Features

• Add EventProf.monitor to instrument arbitrary methods 📝 Docs

Add custom instrumetation easily:

class Work
  def do
    # ...
  end
end

# Instrument Work#do calls with "my.work" event
TestProf::EventProf.monitor(Work, "my.work", :do)

• Adapterize before_all 📝 Docs

Now it's possible to write your own adapter for before_all to manage transactions.

• Add before_all for Minitest 📝 Docs

Fixes & Improvements

• Show top let declarations per example group in RSpecDissect profiler.

The output now includes the following information:

Top 5 slowest suites (by `let` time):

FunnelsController (./spec/controllers/funnels_controller_spec.rb:3) – 00:38.532 of 00:43.649 (133)
 ↳ user – 3
 ↳ funnel – 2
ApplicantsController (./spec/controllers/applicants_controller_spec.rb:3) – 00:33.252 of 00:41.407 (222)
 ↳ user – 10
 ↳ funnel – 5

Enabled by default. Disable it with:

TestProf::RSpecDissect.configure do |config|
  config.let_stats_enabled = false
end

• Added ability to preserve traits in FactoryDefault. 📝 Docs

Disabled by default for compatibility.

Enable globally by FactoryDefault.preserve_traits = true or for a single create_default: create_default(:user, preserve_traits: true)

When enabled the default object is only used when there's no traits in the association.

• Add ability to run only let or before profiler with RSpecDissect.

• Collect raw data with StackProf by default.

• Refactor :with_clean_fixture to clean data once per group.

• Fix RSpec/Aggregate failures with non-regular examples.

Do not take into account xit, pending, its, etc. examples, only consider regular it, specify, scenario, example.

BalkanRuby 2018 special.

TestProf documentation now lives here: https://test-prof.evilmartians.io.

Features

• Add events support to TagProf. 📝 Docs

Example usage:

TAG_PROF=type TAG_PROF_EVENT=sql.active_record rspec

• Add logging helpers for Rails. 📝 Docs

Enalbe verbose logging globally:

LOG=all rspec

Or per example (group):

it 'does smth weird', :log do
  # ...
end

• Add HTML report for TagProf.

Generate HTML report by setting TAG_PROF_FORMAT to html.

• Add ability to track multiple events at the same time with EventProf.

• Add AnyFixture DSL. 📝 Docs

Example:

# Enable DSL
using TestProf::AnyFixture::DSL

# and then you can use `fixture` method (which is just an alias for `TestProf::AnyFixture.register`)
before(:all) { fixture(:account) }

# You can also use it to fetch the record (instead of storing it in instance variable)
let(:account) { fixture(:account) }

• Add AnyFixture usage report. 📝 Docs

Enable AnyFixture usage reporting with ANYFIXTURE_REPORTING=1 or with:

TestProf::AnyFixture.reporting_enabled = true

• Add ActiveRecordSharedConnection recipe. 📝 Docs

Force ActiveRecord to use the same connection between threads (to avoid database cleaning in browser tests).

NOTE: For Rails <5.1.

• Add FactoryAllStub recipe. 📝 Docs

• Add ActiveRecordRefind refinement. 📝 Docs

Fixes & Improvements

• Disable referential integrity when cleaning AnyFixture.

Features

• EventProf now works with Minitest. (#29)

• Fabrication support for FactoryProf added. (#30)

This release bring some new features as long as bug fixes and improvements (see Changelog).

RSpecDissect profiler

RSpecDissect tracks how much time do you spend in before hooks and memoization helpers (i.e. let) in your tests:

RD_PROF=1 rspec ...

[TEST PROF INFO] RSpecDissect enabled

Total time: 25:14.870
Total `before(:each)` time: 14:36.482
Total `let` time: 19:20.259

Top 5 slowest suites (by `before(:each)` time):

Webhooks::DispatchTransition (./spec/services/webhooks/dispatch_transition_spec.rb:3) – 00:29.895 of 00:33.706 (327)
FunnelsController (./spec/controllers/funnels_controller_spec.rb:3) – 00:22.117 of 00:43.649 (133)
...

See PR#14 and guides.

let_it_be helper for RSpec

Just like let, but persist the result for the whole group (i.e. let + before_all).

See PR#13 and guides.

RSpecStamp improvements

RSpecStamp is now integrated with FactoryDoctor, EventProf and RSpecDissect to automatically mark slow (or bad) examples and groups with custom tags:

EVENT_PROF="sql.active_record" EVENT_PROF_STAMP="slow:sql" rspec ...

After running the command above the top 5 slowest example groups would be marked with slow: :sql tag.

Ruby 2.2.x support has been added.