Top Tips for the BDD Tester

Behaviour Driven Development (BDD) is 'outside-in' software development methodology, which has changed the tester's role dramatically in recent years. One of the key benefit of BDD, is communication enhancement. BDD has encouraged a collaborative way of working between product owners, business analysts, project managers scrum masters, developers and testers, resulting in a higher level of customer satisfaction. BDD doesn't replace QA testing but it has changed the approach to testing dramatically. In this short article let's find out how a QA individual can bring quality and business value into BDD projects.

A few words about BDD

Just in case you are new to BDD, it's a term coined by Dan North to overcome the pitfalls in Test Driven Development and bridge the communication gap between business and technology. In a summary, it works like this:
• Write some user stories.
• Capture the intended behaviour in the form of 'features' or examples BEFORE writing any code [Specification By Example]
• Think of all possible scenarios that cover the intended functionality, reducing the risk of creating bugs.
• Implement the behaviour in the form of 'step definitions'.
• Passing tests verify that the implementation of the behaviour has been successful.

Positive results from adopting BDD

• Bridge the communication gap between business and technology
• Faster feedback
• Business and technology speaking the same language
• Business people start engaging in technology

Where does the Tester fit in to all of this?

Developers play a key role in BDD by writing step definitions in order to implement a system's behaviour. Once these steps are implemented, the tests are effectively automated, the benefit is a reduction in manual test effort.

If developers undertake most of the work in BDD projects, then the burning question arises, "Why do we need testers in a BDD project?"

Just because you start automating tests it doesn't mean you don't need QA. It simply provides more time for the tester to do they really want to do, proper, considered exploratory testing.

Here are some tips for testers to make BDD efficient and effective:

• Know BDD Process

As a QA, It's very important to understand the basic concepts of Behaviour Driven Development. The QA process is different from SCRUM or any other methodologies. BDD is an 'out-side in' development methodology, so all the evil thinking that we use to find defects post development, has to be used before writing code. This will help to prevent bugs rather than finding bugs. Identification of all possible scenarios will help to massively reduce the number of bugs entering the system.

In BDD, a "bug is a missing scenario". Therefore, if you find something not working after implementation it could mean you haven't identified that scenario before. As QA, you should act like 'quality police' throughout the project. Your role should be to prevent defects in the early stage of implementation. Be proactive in obtaining all possible information from any source, be it the internet, books, forums or meeting other BDD practitioners.

• Get Involved Early

It's very important to get involved in early stage of the project. It will help you to know the ins and outs of the requirements, which will help you in designing scenarios. It is also important know scope of the project and design sensible scenarios. Mostly, project managers make the mistake of not involving QA in the early meetings of a project, therefore, the QA guys always lag behind. This is a major cause of why many projects didn't work out well. All the teams should realise that QA plays a major role in driving a BDD project. As QA, you should think through all possible scenarios regardless if they are making sense. Every scenario you think about may prevent a bug in the system. Remember, if you miss any scenario this MAY turn into a defect in the implemented system.

• Be Knowledgeable

Being knowledgeable is nothing less than gathering all the business requirements and thinking of all the possibilities surrounding them, confirming this with the product owner. Once you have a complete understanding of the behaviour, you can provide much better insight for what the developers have to implement. In a team, you should be as knowledgeable as the next person. With so many eyes on the project, it is much more likely that bad smells will be uncovered. In most teams, the communication between product owner, developer and tester is not as effective as it could be. It is very important that you should know the ins and outs of the system as much as the developer to make a BDD project a success.

• Take a Leading Role in Specification Workshops

Specification workshops are an opportunity for testers to be proactive in asking questions, in order to get clear requirements from your product owner and discuss all possible scenarios. You should play key role in this meeting. If you fail to understand the systems behaviour in this meeting, then the chances are you will lag behind. Ideally, you should ask business owners a sensible list of questions, which in turn become acceptance tests. Here you can use your evil thinking to break the software before implementation. We are used to breaking software after implementation by applying every possible technique to break the software. It is challenging, but you need to apply those techniques before implementation, so bugs can be prevented from entering the system. In the specification workshop, you shouldn't use 'GivenWhenThen' language. Proper communication is the key in this meeting, using your experience in writing acceptance tests, you just need to make sure you have identified and discussed all possible scenarios of the desired system. Group thinking is important in the scenario design process because we are distilling the user stories with different mindsets. Mostly, business analyst used to think about happy paths. Developers used to think about edge cases in terms of an implementation. Testers used to think in a way that how to break the system. Different perceptions in the specification workshop makes requirements more clearer and unambiguous.

• Write Better Feature Files

Once you have all necessary communication with business, next step is to write feature files. Ideally, QA should take lead to write down all scenarios, preconditions and expected results in the form of feature files. Business analysts / product owners or even scrum masters can contribute to feature files. Once the team agrees all possible scenarios have been covered, then developers can kick start implementation.

A few tips to write better feature files:

1. Use Background

Gherkin is a Domain Specific Language used to describe features in the 'GivenWhenThen' format. Gherkin has some cool features in order to avoid duplication or repetitive steps. Gherkin helps readability for business users. Proper use of the 'background' feature of Gherkin can help to avoid repetitive tasks in each scenario. It is not mandatory to use 'background' in each and every feature, but it makes the feature file more readable. There are few things you need to keep in mind while using 'background'. You have to keep your background section very short, just enough to keep the scenario in context. You shouldn't include technical stuff like starting some services, clearing cache, launching browsers in the background, as the reader will assume these things. You'll implement this in the step definitions by using proper hooks.

2.Use of Data Tables, Examples, Doc String, Multiple tables of example & nested steps

Gherkin provides ability to enhance your feature files by using tables. You can use 'Data Tables'. Data tables allow you to set-up certain set of data as a pre-condition for the tests (Given) or as an example in assertions (Then). You need to use 'Scenario Outline' to extend feature files by examples. Doc String is another cool feature of Gherkin which allows you to hard-code 'text' into your feature files e.g. you can write email message in the quotes as an assertion. Gherkin also allows you to specify multiple tables of examples. Nesting Steps means hiding the implementation details.

3.Use tags and Sub-folders to organise features

Tagging the features/scenarios with certain tags helps categorise them into different groups. You can apply a filter to run the features/scenarios with specific tags. You can also create sub-folders according to the functionality to organise features effectively.
Here is an example feature file covering almost all the above mentioned points:

Feature: Login
In order to secure my website
,
as a business owner
,
I want to implement login functionality

Background: Given I am on login page

@wip @smoke @browser @ci

Scenario Outline: Check user's login credentials
Given I filled <username> <email> and <password>

When I clicked on Login

Then I should see <success or failure> message
 a
And I should get an email saying:
"Dear User,
You have just tried to login to our system. If your login is successful then you will get another email to verify you details. 
Thanks
"

Examples: Valid login
|username |email |password |success or failure|

|Andy |andy@gmail.com |abcd1234 |success |

|Mike |mike@gmail.com |abcd1234 |success |

Examples: InValid login
|username |email |password |success or failure |

|Andy |andy@gmail.com| |failure |

• Watch Developers & Understand the Code-Base

Remember, you're not finished once you've written solid feature files. The next step is to keep watch on how these features are being implemented by the development team without disturbing them. It would be an additional benefit to see how the programmers organise and re-factor their code. You should try to implement some features/scenarios by your own by referring the step definitions written by developers. This approach will speed up the implantation process. You should be fully aware of the test code, as you will be the owner of the test code. You should know how to fix those tests in case they got broken in future.

• Balance Tester / Developer Collaboration Time

Collaborative working is always better in agile team but we need to balance the time we are spending with developers with the business value we are getting from collaboration. We shouldn't slow down developers by them spending hours and hours explaining the code base. This will definitely slow down the developer's workflow. Always come up with list of questions/concerns and ask for a suitable time to discuss those questions with the developers. This time should be covered in the estimation to keep things on track.

• Ensure You Are Writing the Correct Tests

There should be proper balance between API level tests and interface tests. Service level tests are fast and stable while interface tests are brittle, slow and easy to break. You need to analyse the business value of features that need to be run in real browser. You also need to select proper libraries and tools in order to run interface tests.

• Debug the tests

In case you get enough time during the feature implementation, don't forget to debug the implemented steps with developers, by providing different test data, or verifying negative test scenarios. This will help identify tests that are give false positives.

• Drive the Continuous Integration

Continuous Integration is key for successful agile projects. As a QA, you should drive the continuous integration. Keep a keen watch on the builds. Ensure the builds are always green before deployment. Visible continuous integration monitors highlight misdemeanors and help to ensure quality in the BDD projects. Choose proper continuous integration tool which has powerful reporting features including HTML reporting, charts, graphs etc. This will help non-technical people to analyse test reports.

• Consider living documentation

Living documentation is the specifications of the system to in the form of automated feature file which can be edited by an authorised person. Your nicely written feature files should be the living documentation. You should have some collaborative tools to keep all your feature files accessible by everyone. It wouldn't be a good idea to tell non-technical people to go to the source control system to read feature files. Living documentation should be easily accessible without any technical constraints.

• Be active

Remember, BDD is a collaborative process, you should be an active throughout entire project. Don't just be a click tester who run specific suite of tests after every iteration or who just busy in writing automation framework with heads down. You should be active and talk to the team about the progress of implementation. It's all about understanding the product with proper communication.

In BDD, there are few things you shouldn't do or shouldn't allow the team to do. Success of BDD projects depends on proper collection and execution of the business requirements. Below are the some of the points you should keep in mind and avoid as much as possible:

• Don't Allow Developers to Change Feature Files

It may happen that developers wish to change the feature files as per their need so that implementation becomes easier for them. A feature the team has agreed could be complicated to implement, or the developer feels that by changing the feature file they can make code more robust and re-usable. Whatever the case, you shouldn't change the feature files unless agreed by team and business people. Changing feature files means indirectly changing in the requirements. As a technology team we shouldn't change requirements unless agreed by business.

• Don't Slow Down Developers/Delivery

BDD projects are very volatile in nature and everybody in the team has tight schedule in terms of the task assigned to them. Developers always complain that testers are passive and non-technical so they have spend more time with them. It is quite natural that testers are not that technical, so they always need some help from developers, for example, in setting up the environments, or setting up the projects on their local machine. As a QA, you should be technical enough to set it by yourself, if you are having some issues then you can call them for help but don't get your all jobs done by developers. This will definitely affect the productivity of the development team.

• Don't Allow Developers to Inadvertently Code Bugs

When someone finds a bug in production, they always complain about how QA missed that bug. What should also be taken into consideration is how did our programmer code the bug? The key idea of the BDD is to avoid creating bugs in the first place. The question here, is how do you help prevent developers from inadvertently coding bugs. Here are few things you could do:

1. Test the features as implemented. Testing features one by one can help to verify that implementation is well
2. Make sure the developer understands the functionality properly by looking into the code.
3. Keep keen watch on CI if the developer's check-ins are breaking any tests.

• Don't Miss Any Scenario. That Would Be a Bug!

A good understanding of the concept, coupled with deep thinking about all possibilities, will result in a comprehensive set of scenarios for some particular functionality. Remember, if you miss a scenario, it means you have added defect into the system. To avoid this, it is good practice to involve everybody from team in the scenario design process.

Beware of the BDD pitfalls

• Flaky, Brittle & Slow Tests

BDD is all about keeping business and technology engaged. Flaky tests means tests failing randomly without proper reason. Test failure is expensive, so business and non-technical people become frustrated and start loosing interest in the tests. Brittle tests mean too much detail and result in features that are boring to read. Business people hate reading scenarios with steps giving lot of information about the system. Brittle features have lots of "AND" in them. In addition, tests that are slow to run are annoying for everyone.

Tip: Use robust test automation strategies for implementation, which makes tests reliable, fast and readable.

• Lack of Collaboration

Communication enhancement can be achieved through proper channels, which keeps business people interested in the technology.

Tip: Use proper collaboration tools to bridge communication gap.

• The Business Not Reading Feature Files
Think of it from the business' view. If you wrote feature file which has lot of implementation details like 'click here, click there', a business stakeholder is definitely going to loose a lot of interest when reading the feature files.

Tip: Write better quality feature files to keep business people interested.

• Frequent modification and deletion of Scenarios
Frequent modification in the feature files indicates that team hasn't understood requirements clearly and therefore, probably a result of a lack of proper communication.

Tip: Effective collaboration & good scenario design may help.

• Information Loss

Information loss means leaking of an information while communicating with others who were not involved. Information loss is quite often when there are few people involved in the early stage of the project. When those people unable to communicate 100% of the requirements to the team then there is information loss. Information loss causes misunderstandings and misunderstandings are major cause of bugs.

Tip: Get involved everyone from the scratch to avoid information loss

Final Thought

Working with 'out-side in' software development methodologies like BDD are always challenging for testers. Use of communication skills and proper understanding of the process and role will help in delivering quality products.