Using Test Kits in CI/CD

Since inferno execute enables running Inferno tests in CLI, it may be used within a Continuous Integration or Continuous Deployment (CI/CD) pipeline. The generic steps around automatically running a Test Kit are:

1. Setup a container or environment with Ruby and Docker
2. Obtain your target Test Kit version and make it your working directory
3. Launch background Inferno services: bundle exec inferno services start. This command works on top of docker compose and daemonizes the processes for you.
4. Confirm that background services are ready, possibly with the wait-for script.
5. Launch the system under test and confirm that it is ready to receive requests.
6. Select the Tests, Test Groups, and Test Suites you want in CI and run them with inferno execute. See Running a Test Kit in Command Line for more information. The command will exit with status 0 if the Inferno Summarizer deems it a pass.
   • You can run multiple inferno execute commands, but remember that each call represents its own test session.
   • Since TLS cannot always be emulated in CI, you can add the environment variable INFERNO_DISABLE_TLS_TEST=true to mark all TLS tests as omitted and may allow the Test Suite to pass.
7. Clean up. You can stop Inferno services with bundle exec inferno services stop.

If you want to use GitHub Actions for your CI/CD pipeline, or run the ONC Certification (g)(10) Test Kit in CI/CD, you can view our workflow file on the Inferno Reference Server.