Testing

Hero image for Testing

Testing is an essential component of how we develop and deliver robust software applications. Each time a new feature is added or a line code is simply modified, developers write new tests or modify existing ones to validate the implementation. These tests are then executed each time new code is pushed to the code repository. On top of that, product managers only accept a user story after going through a checklist of manual or automated end-to-end testing.

General Best Practices

  • Not over-engineering tests. In order to have maintainable test suites, tests need to be written as a series of simple procedures that can be read by a human. If a test fails, it should be fairly obvious why and fixed in a short amount of time. Many test frameworks come with their own complex domain syntax language (DSL) and convenience methods. But we prefer focusing on the true essence of testing (arrange - act - assert) over complex and non-standard setup.

  • Each test must run in isolation. Flaky tests are often the offspring of tests over-sharing fixtures and setup. Instead, each test must set up its own fixtures and not inherit or rely on any previous states. While it might result in code repetition, it’s an acceptable trade-off for stable tests.

  • Continuously optimize tests suites:

    • Remove old or deprecated tests. As the software application evolves, some tests, while still passing, might not add much value to the overall suite as newer tests already perform the same validations. Don’t be afraid to remove tests.

    • Maintain or reduce the total time to run the whole test suite. Having long running test suites decreases productivity of developers and the effectiveness of tests. Having a short feedback loop is paramount.

Continuous Integration

Automate All The Things! – Anonymous meme character

  • Automate the execution of tests suites each time new code is pushed to the code repository. Developers get fast feedback on their current work thus can prepare the necessary changes if their branch breaks the application. In addition, code reviewers can only start reviewing and approve pull requests that pass the test suites.

  • Execute tests on a dedicated server i.e. developers should not have to run the tests suites on their machine. Running tests usually use lots of computing resources and require isolated environments. In addition, when working as a team, tests suites need to be broken down into several jobs and executed in parallel to have fast feedback.

For web applications, we currently use Semaphore CI and for mobile projects self-managed Jenkins CI on Digital Ocean.