Some more background
The motivation here is the implementation of tests mocking/intercepting real browser requests, to improve Stripe test coverage.
1) What are feature tests?
“Integration test”, “end-to-end test” and “acceptance test”: In RSpec terminology, these types of tests have historically been called feature specs.
Source: https://www.codewithjason.com/difference-system-specs-feature-specs/
2.1) What are system tests? These are Rails native tests, to “test user interactions with your application, running tests in either a real or a headless browser. System tests use Capybara under the hood.” Source: https://guides.rubyonrails.org/testing.html#system-testing
2.2) What are system specs?
System specs are a wrapper around Rails’ system tests. The benefits of using system specs instead of feature specs is that you don’t have to explicitly include the Capybara gem – this is being used, under the hood.
You don’t need Database Cleaner either, because system tests are already run inside a transaction.
Source: https://www.codewithjason.com/difference-system-specs-feature-specs/
2.2.1) What’s a transaction!? This refers to database transactions. More on this on the reference below.
Source: https://anti-pattern.com/transactional-fixtures-in-rails
2) Whats the difference between feature and system specs?
Both can cover end-to-end tests, but on system specs capybara already runs under the hood and "transactional_fixtures" are activated by default.
Source: https://www.codewithjason.com/difference-system-specs-feature-specs/
3) If a spec is driven by selenium,
is it (necessarily) a feature spec?
No, you can have a system specs driven by selenium, too. But we can setup system specs to use to other drivers as well, like cuprite
.
4) Why would you do that?
There are some advantages to have specs using ferrum/cuprite instead of selenium.
Selenium tests are usually pretty slow, because...
Selenium needs browser-specific drivers to emulate user-browser interactions
Cuprite makes use of ChromeDevTools (CDP) protocol:
https://github.com/ChromeDevTools/awesome-chrome-devtools
So, it's lighter and easier to maintain, as it is a a pure Ruby Capybara driver (using CDP).
Source: https://evilmartians.com/chronicles/system-of-a-test-setting-up-end-to-end-rails-testing
5) Why wouldn't you do that?
Selenium is battle tested. The cuprite/ferrum
setup is fairly new. Some other bits of the testing stack might not have been designed to integrate with cuprite/ferrum.
That leads us to...
6) ...the million dollar question: Can puffing-billy gem
be used on any of these test setups? Must we stick to feature or go with system? Do we need selenium webdriver or rather the nifty cuprite/ferrum
combo?
On the section below we can see that, as of today, puffing-billy only supports this list of drivers:
Source: https://github.com/oesmith/puffing-billy#setup-for-capybara
So one approach would be to customize the javascript driver:
https://github.com/oesmith/puffing-billy#customising-the-javascript-driver
But this might drive us away from the current goal... It seems we're not the only one attempting the combo system spec + puffing-billy: https://github.com/oesmith/puffing-billy/issues/269
Conclusion 😅
After going through the points above it seems to me that the best way forward, for now, is to:
make
puffing billy
work with a feature spec, which usesselenium webdriver
.
Useful references
1) https://dev.to/nejremeslnici/migrating-selenium-system-tests-to-cuprite-42ah
2) https://evilmartians.com/chronicles/system-of-a-test-setting-up-end-to-end-rails-testing
3) https://relishapp.com/rspec/rspec-rails/docs/system-specs/system-spec
4) https://www.codewithjason.com/capybara-system-specs-resources/
5) https://guides.rubyonrails.org/testing.html#system-testing
Last updated