Automate release testing
This page describes the process for replacing the weekly performed manual release testing, by automated tests
The process below is centered around the column Automated tests, found see the OFN-delivery pipe. Here, an the epic #4333 is currently open, to be be populated with the issues necessary to replace each manual test, found on our weekly manual release testing procedure.
Below we detail a process to achieve this goal. To best contribute to this task, we recommend setting up your local system - see our wiki for details. This consists in an iterative steps, based on the following:
1) Identify relevant manual test-cases from #4127 and from the currently used “release-testing” procedure
2) Create the corresponding tech-debt issues. Add these to the epic #4333
3) Check for existing specs on the relevant /spec/feature files, on the current master-branch:
i) if the necessary specs (assertions/examples) already exist and cover the respective manual test-cases, then the issue opened in 2) can be closed - this should be clearly documented/justified as a comment, in that issue. To aid documentation, please indicate the verbose part of each spec which replacing the manual tests; The command “rspec -fd spec.rb” can be useful to output this in the documentation format.
ii) if the necessary specs are missing or only partially address the manual test-cases then it's best to proceed as PRs are usually handled:
- write the necessary tests to close the tech-debt issue (which would take us back to point 3.i)
- close the tech-debt issue at hand but open "smaller" issues to address whatever is still missing to address the necessary test cases.
4) After the tech-debt issue was closed, then that should mean some manual test cases can now be removed from the manual release-testing procedure. Remove them, and inform other testers on the team, so the manual testing burden gets smaller for everyone.
Last modified 2yr ago