Sunday, 15 May 2016


QA plays a crucial part in product quality and product delivery. In agile development methodologies like scrum, QA needs to work in highly compressed test execution cycles. One of the challenges the QA engineer's face is that the code that worked in previous sprints often gets churned by the new features and bug fixes in each subsequent sprint, increasing the risk of regression. Without test automation it might be hard to achieve sufficient test coverage and provide rapid feedback.  
So is test automation a bullet?
Before we jump to the conclusion...we need to understand the following:
Check if your application is automation ready? Below are the questions you need to ask yourself before rushing to automate the tests.
 1. Is your application ready for automation?

We need to be careful when automating GUI. Never start automating at the start of the project or you might end up re-writing the automation scripts that you have written.
To start automating, you need to have a set of stable and functional features ready. It is wise to start automating when you are sure that these features and the GUI will not change. If you proceed and automate the features that are undergoing continuous changes in GUI then the cost of maintaining the script will be too high. Predicting what will change and what will remain stable is tough to determine when working in an agile environment as we need to embrace and welcome the changes.
Another important thing is not to overlook is the maintenance that is required to keep all your scripts up to date and working. Even after spending hours on programming the automation scripts, your automation scripts might fail when you get a new version of the product. You need to design the scripts in such a way that it will require the smallest amount of maintenance in the future. We should have a plan of how we are going to accomplish that in our 'automation strategy'.
2. Do you have skilled testers in your team?

Your QA team need tester's with the skill-set to utilise the capabilities of the tool. No tool can help you unless you have skilled resources to implement automation. Your test team should capable of handling challenges that might arise in automating the test suite. Most of the tools demand testers to code. Most of the automation tool vendors provide training and support to the resources else you should consider bringing in an expert from outside. Test Automation is a long term investment. It will take time and expertise in developing and maintaining test automation frameworks and automated scripts. 
 3. Have you selected the right tool?

Choosing the right tool is crucial to succeed in test automation. There are several factors that need to be considered when selecting a test automation tool. Some of the tools are free, some are expensive.
Several times after few days of initial excitement and after automating few tests the tool often becomes "shelf-ware" and testing returns to Manual.
  Here are some factors that you might need to consider before selecting the tool:
  •          Does the tool fit into current process and infrastructure?
  •          Does the tool vendor support training and provide good services,
  •          Do you have the budget for the tool?
  •          Product features: How does the tool identify the objects? Image based recognition? Object-based recognition?
  •          Which programming languages does it support? Does the QA team have the knowledge on these languages?
  •          If using an open source tool: Does the tool have good community support and enough documentation?
  •          The list goes on…

There are risks of the vendor going out of stock or the tool being acquired or retired. It's a good practice to explore the tool on a small scale with a pilot project and determine if the tool can accomplish what is needed.
We should keep in mind that no tool is perfect, be it an open-source or commercial. We need to consider the above factors and select the tool that serves our purpose and provides good ROI.
Did you say "Everything"?
 Then it is an unrealistic and impractical expectation!! 
Testing software involves a lot of repetitive work which is tedious to do manually. People become bored and make mistakes when doing the same task over and over. People tend to do the same task differently when they are repeating it again and again. A tool will exactly reproduce what it did before, so each time it is run the result is consistent. The regression suite is the ideal candidate for automation.

Repetitive work is excellent job for a computer to do.


Assume you are testing an app which has a feature called ‘People’ that sync's all your contacts from various social media and email accounts. You are testing if the ‘contact count’ displayed is accurate. The test account has 5000+ contacts. It is not feasible count the contacts manually and check if the count displayed is accurate. 
Assume your testing an email app and the app crashes frequently when you 'archive' around 30-35 emails in succession (one after another). When developers ask you to replicate this issue, think how tiresome it would be to do it manually. Let's be positive and think that you replicated the issue after few attempts..and the developer fixes it. But that's not the end of the story.... you need to confirmation test if the issue is really fixed…Good luck replicating that bug again!
Automation tool eases the life of a QA engineer in such situations and can save a lot of time and effort by automating these annoying tests.

BVT(Build Verification tests) are cursory tests that are helpful to determine the stability and testability of the build. These tests need to be run every time a build is deployed. The BVT’s are ideally quick tests that concentrate on core functionality.
Automating BVT's and integrating them to the continuous integration server will ensure that the changes made have not led to regressions. One thing to keep in mind while designing these tests is that the test should be short and should execute in few minutes. There may be multiple commits a day it does not make sense if the automated BVT's run for hours.
Also consider the following while automating:
  •  Automation is a great option when we have to validate the same feature with large set of different inputs and data (Data-driven testing).
  •  Load and performance testing should be ideally automated where we test the system’s ability to handle load by creating virtual users. 

Do not try to automate the tests that would require human intelligence and intuition.
  •          Tests which are not executed frequently or would be too cumbersome to automate.
  •          Tests like exploratory testing which is based on intuition and expertise of the tester.
  •          Tests that need human intervention. An automated test should be able to run and verify the outcome entirely unattended.
  •          Do not automate the volatile features which will have frequent changes.
  •       Prioritize your automation using the 20-80-20 rule. Automate 20 percent of the scripts that take 80 percent of the execution time and 20 percent of the resource skills.

        One of the developers who watched me automate was fascinated and said “This is cool stuff! Now you need not have to do anything, the computer will do everything for you. I feel that computers will replace the testers soon”.
        I replied “The day when testers will be replaced by automated tester's is the day when developer's will be replaced by an auto-developer or a CEO will be replaced by an auto-CEO ...”
        QA experts do not use the word 'test automation'. They rather call it ‘automation checks’ as they feel testing cannot be automated, but checking can be completely automated. Humans have the ability to do things, notice things, and analyze things that computers cannot. Our humanity as testers plays a big role in identifying defects while testing. A computer cannot experience or cannot express emotions.
        A computer cannot 'determine if the game is fun or not?' ..... 'if an app has good user experience or a pleasant interface'..... Or to 'take decisions using heuristics'  etc. Automated test can only find defects that you already thought of or predicted might happen.
        You may ask if the test automation can run the same test more quickly, more accurately and with same consistency every time, is it not a better than a human tester?
        No! Because running a test quickly with more precision and consistency does not produce better software but it would definitely reduce the time required to complete testing. Test Automation is probably one of the most useful weapons in tester's inventory. It has its own place in the inventory. The weapon is useless without its wielder. Automated tests require scripts to be written and maintained; this in turn requires a HUMAN.

Automation tools are here to assist the testers not to replace them.

                Test Automation is certainly not a "Silver bullet".
         About the Author

Prashant Hegde is a passionate tester. He leads the QA team in Razorthink software and ensures that the products he works on meet the highest standards in tight schedules and deadlines. Prashant loves to write and he has authored articles for leading software testing magazines. He writes a blog on best practices in software testing in his spare time. Prashant is a gamer, a biker, and an author.
      Follow Prashant on his website -
      Subscribe to his blog –

Mind maps: A killer way to increase your test coverage

What is a mindmap?

A mind map is a diagram used to visually organize information. It can be called a visual thinking tool. A mind map allows complex information to be presented in a simplified visual form. A mind map is created around a single concept. The concept is represented as an image in the centre to which the associated ideas are added. Major ideas are connected directly to the central concept, and other ideas branch out from those.

Mind maps are great for note taking, planning, studying, brainstorming, etc. The term 'mind map' was first used by Tony Buzan in 1974. I drew my first mind map when I was in school. I preferred mind mapping over text notes and it proved to be a great aid to revise and recall the concepts quickly. This is because the information in mindmap is structured in a way that mirrors exactly how the brain functions - in a radiant rather than linear manner. A Mind Map literally ‘maps’ out your thoughts, using associations, connections and triggers to stimulate further ideas.

How to draw a mind map

Tony Buzan the "Father of Mind Mapping" suggests the steps below:

1. Start in the centre of a page.
Why? Because starting in the centre, gives your brain freedom to spread out in all directions and to express itself more freely and naturally.

2. Use an IMAGE or PICTURE for your central idea.
Why? Because an image is worth a thousand words and helps you use your Imagination. A central image is more interesting, keeps you focused, helps you concentrate, and gives your Brain more of a buzz!

3. Use COLOURS throughout.
Why? Because colours are as exciting to your Brain as are images. Colour adds extra vibrancy and life to your mind map, adds tremendous energy to your Creative Thinking, and is fun!

4. CONNECT your MAIN BRANCHES to the central image and connect your second and third level branches to the first and second levels, etc.
Why? Because your Brain works by association. It likes to link two (or three, or four) things together. If you connect the branches, you will understand and remember a lot more easily.

5. Make your branches CURVED rather than straight-lined.
Why? Because having nothing but straight lines is boring to your Brain.

Why? Because single keywords give your mind map more power and flexibility

7. Use IMAGES throughout.
Why? Because each image, like the central image, is also worth a thousand words. So if you have only 10 images in your mind map, it’s already the equal of 10,000 words of notes!

  • If a mind map is getting too big or complicated try splitting it.
  • Do not use long detailed sentences in mind maps. Using one word per line improves clarity and understanding. Using single keywords will make your mind maps more powerful and flexible.
  • Develop your own personal style of mind mapping.
How to use mind mapping techniques in software testing

Mind maps can be used in all the test stages from test planning to test case execution. They can be used for:
  • Test Planning
  • Requirement analysis
  • Impact analysis
  • Task allocation
  • Test case design
  • Traceability
  • Test reporting -Quick test reports
Test Planning

While test planning, you can draw an initial mindmap keeping in mind the list of tasks, schedules, tools, roles, responsibilities, milestones,etc. Present the mindmap and discuss it with your stakeholders. Modify the mindmap if any changes are required. One thing you will love about mind maps is its flexibility to adapt to changes. All you might have to do is to add or remove a node/branch. This flexibility might not happen when you draw on a paper, but mind mapping software assists any changes easily.

The final mind map shows you the scope of testing in one glance. This mindmap can be used as a blueprint and later converted into a plan. This ensures that no test activity is missed.

Test case design

Mind maps are an efficient way of creating lean test cases. It reduces the time required for creating test cases yielding better results. Mindmaps are very easy to maintain and are flexible to changing requirements.
Draw branches from every user story/epic and associate all its functionalities as sub-nodes.Start adding test ideas/test case for each functionality.

I created a mindmap covering test ideas for the major functionality. My team started to expand the mindmap by branching out more and more test ideas. We kept adding new nodes when we found unique scenarios that uncovered the bugs during our test sessions. This drastically increased our test coverage.The final mindmap can be used as the basis for test case document or it's cool if it's used as it is.
The good result of mind mapping is that you generate more ideas when drawing them. Collaborative mind mapping with the team gives you the best results.

Traceability mindmap

A traceability matrix is an essential tool for every tester to analyse and improvise the test coverage. You can use a mindmap instead of a tabular traceability matrix.

To create a traceability mindmap - add nodes of all the Epics. Draw branches from every module and associate all its user stories as subsequent nodes. Now link the test cases for all functionality. You can link the requirement number of the test management tool.

This ensures that you have not missed out writing test cases for any user story. This mindmap gives you the birds-eye view of your test coverage. You can identify the areas where you need to strengthen your coverage.

You can use mind maps anywhere and everywhere!

Below is a mindmap for "App store compliance-iOS":

There are tons of commercial and open-source tools that let you visualize your ideas as a mindmap. I prefer the following tools:
  • Xmind(Windows/Mac/Linux)- Probably the most popular and free mind mapping tool.
  • Coogle(Web app) - Coogle is a web app that lets your team collaborate and work on a single mindmap.
The use of mindmaps is getting popular with agile testers and lean test practitioners. When will you start using mindmaps?

About the Author

Prashant Hegde is a passionate tester. He leads the QA team in Razorthink software and ensures that the products he works on meet the highest standards in tight schedules and deadlines. Prashant loves to write and he has authored articles for leading software testing magazines. He writes a blog on best practices in software testing in his spare time. Prashant is a gamer, a biker, and an author.

Follow Prashant on his website -
Subscribe to his blog –

Saturday, 14 May 2016

How to Configure & Use JMeter Plugins

The Problem
One challenge while working on JMeter is managing plug-in. It is a tedious task to download appropriate *.jar file, store jar file at relevant folder. If new version of plug-in comes, as a JMeter user, I need to repeat all the steps. In addition, I have to visit JMeter plugin website & check if any new version is released or not.

A Solution
The idea of JMeter Plugins Manager is simple, Install Plugins Manager once and it will do it all for you – like installing, upgrading, uninstalling of JMeter plugins.
  • In order to install JMeter Plugins Manager, we need to Download the Plugins Manager JAR file and store it into JMeter's lib/ext directory.  JMeter Plugins Manager can be downloaded from link jmeter-plugins-manager.
  • Restart JMeter and go to “Options” menu to access Plugins Manager

  • In case any of your installed plugins have upgrades, The label on the menu item will say "has upgrades".

  •      Clicking on menu item will bring up the plugins manager dialog:

  •       The dialog has two tabs, "Installed Plugins" and "Available Plugins" button.
  •       Removing ticks from plugins on "Installed Plugins"  & click on button “Apply Changes and Restart JMeter” tab will uninstall them. Same way, setting ticks on "Available Plugins" tab and click on “Apply Changed and Restart JMeter” will install the plugin.

Keep in mind that JMeter Plugin Manager is still in beta phase, so in coming time many more new feature might get added.

Happy Testing

Monday, 2 May 2016

Does QTP testing require in depth knowledge of VB script

You need to understand what you are trying to do with QTP. Generally, QTP testers work on three different levels -

1. User - Framework and automation test scripts are created by other people in this scenario. The user only executes the prewritten scripts and analyze the results. You will need little knowledge of vbscript in this case, just to be able to troubleshoot if something goes wrong. You should know following in VB Script - Focus on flow controls, creating methods, basic string operations, date operations, variables type, reading and writing to Excel, and commonly used VB script methods.

2. Script developer - In this case, you will need to prepare test scripts, adhering to the guidelines of the framework in your organization. You will need deeper understanding of VBScript and programming in general to play this role.

3. Framework designer/developer - This is a highly specialized role. You will need in depth knowledge of VBScript and at least another high level language, understanding of design patterns and good grasp of data structures and algorithms to play this role.

Think where you want to be in your career. In case you want to opt for #3, You should start learning VBScript.

You can visit W3Schools Online Web Tutorials and complete online tutorial on VB.