Per wikipedia, smoke testing was probably first used in the context of plumbing systems, in referring to tests for the detection of cracks, leaks or breaks in closed systems of pipes. Perhaps a QA in the software field may feel like a plumber sometimes, using various wrenches to correctly connect the series of tubes that make the application work for the end user.
This is where a smoke test comes in so handy – especially when the application is in its infancy, undergoing a lot of code churn and with fewer baked in features. Developers are committing code regularly and the build server is running non-stop, generating multiple instance of application to be tested by the overworked plumber … er, QA.
Decisions need to be made
When it comes to testing, not all plumbing problems are the same. Software is complex and every line of change can affect any of the existing code, leading to interminable permutations of paths for the QA to test. There is never enough time. As viable test candidate builds are spun out by the build server, the QA has to decide:
- “whether to test or not to test”: Decisions have to be made whether to test the build at all. Obviously if there is a compilation error, or if static analysis indicates linting errors, or there is any other build issue, no point in testing the build.
- “what to test”: Having decided the build is correct, the next decision is to figure out what is the scope of the code change and how that affects the quality of the application. Diffidence and distrust are a QA’s best friends in this regard … especially as a starting estimate. Let the code commit prove it does not reduce the quality level of the application.
- “who should test”: If the code change needs to be tested, the question must be asked – who is best suited to test. Sometimes, its best that the developer run unit test automation, and validate the build manually by doing a smoke test, to confirm the application meets basic entry level criteria before the busy QA gets a crack at verifying the code change.
Smoking out errors
Developers are by definition the ones who know their code best – after all they are the ones who wrote it. If you consider the QA role as helping to highlight the work of developers, then smoking out errors early and often allows said developers to fix anything that they missed before those errors make it out to end users.
Good developers already write unit tests which they commit along with their code. Ideally these unit tests verify the various paths inside their code change. The interfaces to the module under test are typically mocked up so that unit tests run blazing fast and don’t depend on the correctness of other components to work.
However, to ensure that code changes under test play well with existing components of the application, we need end to end tests. A smoke test is one such end to end test that tests basic functionality, but with the following characteristics.
I’m a little smoke test, short & sweet
Having understood the need for a smoke test, the QA needs to define it so that it is:
- short meaning the test scenarios can be completed in under 15 minutes.
A smoke test that is too long becomes a bottle neck to code flowing through the development pipeline. QA needs to make the smoke test easy for the developers to execute themselves. Or better still, be converted into an automation script that serves as the entrance criteria for any code change making it into the system.
- sweet meaning that areas covered by the test are the classic “happy path”
A smoke test needs to cover the easy and efficient workflow, touching the basic functionality and successful scenarios. Smoke testing helps ensure that the code changes don’t mess up something so simple and elemental that its a waste of time running more expensive functional or regression testing.
So, go forth and smoke out those pesky failures, QA … as soon as code is committed, hopefully by developers who made the commit, but definitely by tests that are short and sweet.
how2qa.wordpress.com 