Some more background

The motivation here is the implementation of tests mocking/intercepting real browser requests, to improve Stripe test coverage.

1) What are feature tests?

“Integration test”, “end-to-end test” and “acceptance test”: In RSpec terminology, these types of tests have historically been called feature specs.

Source: https://www.codewithjason.com/difference-system-specs-feature-specs/

2.1) What are system tests? These are Rails native tests, to “test user interactions with your application, running tests in either a real or a headless browser. System tests use Capybara under the hood.” Source: https://guides.rubyonrails.org/testing.html#system-testing

2.2) What are system specs?

System specs are a wrapper around Rails’ system tests. The benefits of using system specs instead of feature specs is that you don’t have to explicitly include the Capybara gem – this is being used, under the hood.

You don’t need Database Cleaner either, because system tests are already run inside a transaction.

Source: https://www.codewithjason.com/difference-system-specs-feature-specs/

2.2.1) What’s a transaction!? This refers to database transactions. More on this on the reference below.

Source: https://anti-pattern.com/transactional-fixtures-in-rails

2) Whats the difference between feature and system specs?

Both can cover end-to-end tests, but on system specs capybara already runs under the hood and "transactional_fixtures" are activated by default.

Source: https://www.codewithjason.com/difference-system-specs-feature-specs/

3) If a spec is driven by selenium, is it (necessarily) a feature spec?

No, you can have a system specs driven by selenium, too. But we can setup system specs to use to other drivers as well, like cuprite.

4) Why would you do that?

There are some advantages to have specs using ferrum/cuprite instead of selenium.

  • Selenium tests are usually pretty slow, because...

  • Selenium needs browser-specific drivers to emulate user-browser interactions

Cuprite makes use of ChromeDevTools (CDP) protocol:

https://github.com/ChromeDevTools/awesome-chrome-devtools

So, it's lighter and easier to maintain, as it is a a pure Ruby Capybara driver (using CDP).

Source: https://evilmartians.com/chronicles/system-of-a-test-setting-up-end-to-end-rails-testing

5) Why wouldn't you do that?

Selenium is battle tested. The cuprite/ferrum setup is fairly new. Some other bits of the testing stack might not have been designed to integrate with cuprite/ferrum.That leads us to...

6) ...the million dollar question: Can puffing-billy gem be used on any of these test setups? Must we stick to feature or go with system? Do we need selenium webdriver or rather the nifty cuprite/ferrum combo?

On the section below we can see that, as of today, puffing-billy only supports this list of drivers:

Source: https://github.com/oesmith/puffing-billy#setup-for-capybara

So one approach would be to customize the javascript driver:

https://github.com/oesmith/puffing-billy#customising-the-javascript-driver

But this might drive us away from the current goal... It seems we're not the only one attempting the combo system spec + puffing-billy: https://github.com/oesmith/puffing-billy/issues/269

Conclusion 😅

After going through the points above it seems to me that the best way forward, for now, is to:

  • make puffing billy work with a feature spec, which uses selenium webdriver.

Useful references

1) https://dev.to/nejremeslnici/migrating-selenium-system-tests-to-cuprite-42ah

2) https://evilmartians.com/chronicles/system-of-a-test-setting-up-end-to-end-rails-testing

3) https://relishapp.com/rspec/rspec-rails/docs/system-specs/system-spec

4) https://www.codewithjason.com/capybara-system-specs-resources/

5) https://guides.rubyonrails.org/testing.html#system-testing

Last updated