Ruby Tests Profiling Toolbox
Factory Default profiles shows which factories (and variations) were created via associations and how many times. This information can help you estimate the effect of adding a default record.
Here is an example report:
$ FACTORY_DEFAULT_PROF=1 bin/rspec
[TEST PROF INFO] Factory associations usage:
factory count total time
track 281 00:12.671
user{organization:<Organization#<id>>} 62 00:05.830
user 46 00:04.401
assessment 6 00:02.599
specialist_vertical 24 00:02.209
user[without_plan] 16 00:01.201
organization 352 00:01.138
admin 341 00:00.999
After adding default factory records, you can now also get the information about the actual usage:
$ FACTORY_DEFAULT_STATS=1 bin/rspec spec/models/user_spec.rb
[TEST PROF INFO] FactoryDefault usage stats:
factory hit miss
track 224 51
admin 83 0
organization 77 89
user 51 82
FactoryDefault summary: hit=435 miss=222
skip_factory_default(&block) to temporary disable default factories.You can also use TestProf::FactoryDefault.disable!(&block).
Now create_default(:user) and create_default(:user, :admin) would result into two defaults corresponding to the specified traits.
preserve_attributes = false | true configuration option.Allow skipping defaults if association is defined with overrides, e.g.:
factory :post do
association :user, name: "Post Author"
end
create_default into create) by setting the FACTORY_DEFAULT_DISABLED=1 environmental variable.before_all hooks.TestProf::BeforeAll.configure do |config|
config.before(:begin, reset_sequences: true, foo: :bar) do
warn <<~MESSAGE
Do NOT create objects outside of transaction
because all db sequences will be reset to 1
in every single example, so that IDs of new objects
can get into conflict with the long-living ones.
MESSAGE
end
end
before_all / let_it_be.Default factories created within before_all or let_it_be are not reset 'till the end of the corresponding context. Thus, now it's possible to use create_default within let_it_be without any additional hacks. Currently, RSpec only.
let_it_be(:x, freeze: true) are now frozen during initialization, not at the access (let) time.Added experimental support for using before_all with Rails' test parallelization using processes.
Make sure to include TestProf::BeforeAll::Minitest before you call parallelize.
Fixed access to let_it_be variables in after(:all) hook.
TestProf.configure do |config|
config.logger = Logger.new($stdout, level: Logger::WARN)
end
nate_heckler mode for FactoryProf.Drop this into your rails_helper.rb or test_helper.rb:
require "test_prof/factory_prof/nate_heckler"
And for every test run see the overall factories usage:
[TEST PROF INFO] Time spent in factories: 04:31.222 (54% of total time)
This is the first major version of TestProf: the APIs have been stabilized, all the features we planned initially have been implemented and battle-tested. The work on v1.x will continue (mostly, maintenance, minor features), and the work on new version, v2, is starting.
P.S. You can support us on GitHub Sponsors.
AnyFixture#register_dump to cache fixtures using SQL dumps.before_all.You can load and access fixtures when explicitly enabling them via before_all(setup_fixtures: true, &block).
Add after_all to Minitest in addition to before_all.
Add support for RSpec aliases detection when linting specs using let_it_be/before_all with rubocop-rspec 2.0.
Minitest's before_all is not longer experimental.
Remove deprecated AggregateFailures cop.
Remove ActiveRecordSharedConnection.
Replaced TestProf::AnyFixture.reporting_enabled = true with TestProf::AnyFixture.config.reporting_enabled = true.
let_it_be.See documentation.
let_it_be modifiers.TestProf::LetItBe.configure do |config|
# Make refind activated by default
config.default_modifiers[:refind] = true
end
let_it_be modifiers via metadata.context "with let_it_be reload", let_it_be_modifiers: {reload: true} do
# examples
end
TEST_STACK_PROF_INTERVAL env variable.Now you can use $ TEST_STACK_PROF=1 TEST_STACK_PROF_INTERVAL=10000 rspec to define a custom interval (in microseconds).
SAMPLE and SAMPLE_GROUP work consistently with seed in RSpec and Minitest.
Make sure EventProf is not affected by time freezing.
EventProf results now is not affected by Timecop.freeze or similar.
See more in #181.
let_it_be modifiers.In addition to two built-in modifiers (reload and refind), you can add your own:
TestProf::LetItBe.config.register_modifier :reload do |record, val|
# ignore when `reload: false`
next record unless val
# ignore non-ActiveRecord objects
next record unless record.is_a?(::ActiveRecord::Base)
record.reload
end
For more complex scenarios feel free to use your own report name generator:
# for RubyProf
TestProf::RubyProf::Listener.report_name_generator = ->(example) { "..." }
# for Stackprof
TestProf::StackProf::Listener.report_name_generator = ->(example) { "..." }
let_it_be with modifiers.# Now you can use modifiers with arrays
let_it_be(:posts, reload: true) { create_pair(:post) }
ActiveRecordSharedConnection is used in the version of Rails
supporting lock_threads (5.1+).This release is inspired by ideas discussed and implemented in this PR to Discourse. Specials thanks to @danielwaterworth and @SamSaffron.
tl;dr before_all hooks, let_it_be aliases, better Fabrication support.
before_allbefore_all.Now you can execute additional code before and after every before_all transaction begins and rollbacks:
TestProf::BeforeAll.configure do |config|
config.before(:begin) do
# do something before transaction opens
end
config.after(:rollback) do
# do something after transaction closes
end
end
let_it_belet_it_be aliases with predefined options.TestProf::LetItBe.configure do |config|
config.alias_to :let_it_be_with_refind, refind: true
end
Then use it in your tests:
describe "smth" do
let_it_be_with_refind(:foo) { Foo.create }
# refind can still be overridden
let_it_be_with_refind(:bar, refind: false) { Bar.create }
end
[TEST PROF INFO] Factories usage
Total: 15285
Total top-level: 10286
Total time: 299.5937s
Total uniq factories: 119
total top-level total time top-level time name
6091 2715 115.7671s 50.2517s user
2142 2098 93.3152s 92.1915s post
Added Fabrication support.
Added threshold and instrumentation event customization.
NOTE: default threshold is 0.01s (i.e. if the DB time is less for the example, we consider it "good").
$ FDOC=1 FDOC_EVENT="sql.rom" FDOC_THRESHOLD=0.1 rspec
guard and top_level options to EventProf::Monitor.For example:
TestProf::EventProf.monitor(
Sidekiq::Client,
"sidekiq.inline",
:raw_push,
# top_level: true means that we do not trigger events when the method is
# called recursively within itself
top_level: true,
# only trigger the event when guard returns `true`
guard: ->(*) { Sidekiq::Testing.inline? }
)
factory.create event.Cosmonautics Day special.
This release makes Ruby 2.4+ required and also RSpec 3.5+ required (for RSpec-related functionality).
before_all compatible with isolator.📖 Documentation
with_logging and with_ar_logging helpers to logging recipe.📖Documentation