Over the years the demand is growing for the team be operating in Agile fashion and with Agile Testing methods almost at every project location. Project manager wants model or prototype built quickly and testing to be performed in an iterative manner. Obviously, there are advantages of this iterative short cycle testing & the benefits are enormous.
Agile testing method focuses on an incremental, short term deliveries that allows frequent review responses & continuous feedback mechanism. Last week we have seen how Agile mindset helps in project execution, various roles and terms associated with it. In this blog we are going to see how a testing team performs testing using agile methodology. The quintessential difference is that the testing is not done during only one phase, but testing is done iteratively with every release of the software or the product. This way of testing early and repetitively ensures minimal damage and risk towards the end of project Everyone would have seen the classic curve of time vs cost when defect is identified & cost of fixing it. In Agile approach this is exactly the area that testers can best tackle for you. For them to achieve this project manager and project teams must have an agile mindset.
Let’s see how exactly an agile tester can contribute and participate effectively in an agile project environment.
- Attend Sprint Planning Session.
As a member of Agile Team, quality leads and tester should always attend Sprint Planning sessions. This helps to ensure that they are in sync with the development team from the start and allow QL to identify and highlight possible problem arrears and risk as early as possible. During the Sprint Planning, developers estimate the effort it will take them to write code, similar way testers should estimate effort required for testing code during planning session. So, resulting in a win-win situation for developers and testers both.
- Attend Daily Stand-up meeting.
QA Tester should attend daily stand up meeting with the developers, Scrum Master and the Product Owner. This promotes a strong collaboration amongst the team members and a sense of one-team prevails. Based on the updates received from the Developers, Business Analysis and Testers, product owner stays up to date with how the sprint is going on and teams are able to plan their workload efficiently. In case, if the tester has a blocker, they can bring it up in the daily stand-up meeting and update every team member on known issue which in turn allow developers to speed up testing progress and plan their workload.
- Don’t wait to report bugs, report it immediately & frequently throughout the Sprints:
This is very important, in order to deliver high quality of software in a short period of time your team needs to work efficiently. Any issue that is found during sprint must be reported immediately in the same Sprint. Clubbing all the incidents and reporting only at the end does not help in agile setup. If this happens then closing the sprint becomes difficult and makes it ineffective, given that it is a time boxed activity. By continuous integration of testing and development, it allows both the teams to work together and resolve their issue faster and this help in leading to higher quality of results.
- Be an active participant of Sprint Demo
At the end of each Sprint, tester and developer teams demonstrate the product features that are developed to the product owner and other stakeholders. Any feedback has been received related to look and feels & other features that is incorporated on immediate effects. This help to deliver the product with high quality customer satisfaction and meet the business requirement.
- Document the test cases
Although in agile project where documentation is minimum, due to frequent requirement changes, testing team cannot afford to skip necessary documentation. Documentation is very important for QA. Execution for test cases & pass logs is the proof of successful testing and is enquired to be documented at various levels. This becomes useful for Bank internal or external audit requirements. This also covers the scenarios where a person is replaced by another one.
- Attend Sprint retrospectives
At the end of each sprint, team needs to discuss and provide review comments about what went well & what did not work during the sprint. This helps to improve in future sprint planning and execution. Sprint retrospective is the opportunity to define weaknesses and determine solutions for them. QA need to be involve in these discussions for any concerns that needs to be addressed before the next Sprint can start.
There are certain cautionary warnings that must be taken care during the agile testing:
- Avoid delays in delivering a story point. This will have cascading effects on QA, testing and eventually in achieving the sprint goal.
- The scope and vision must be clear for a sprint. Any changes identified during the sprint should be parked & then incorporated in the next sprint. There is always a temptation to include everything in the on-going sprint. This approach can prove counterproductive and losing the sprint altogether. Hence must be avoided.
- Try to avoid falling in a trap of using Agile as an excuse for missing documents. Agile does not mean no documentation; bust it means minimal documentation that is essential. Ever increasing scope and improvement feature, just because it is agile project one should not keep adding features, wish lists and ideas to the existing project. There must a definite start and an end. Otherwise the project can become an eternal agile project.
The contribution of agile testers in various ways listed above, results in multiple benefits such as:
- Improvement in successive Sprints: By producing frequent builds and conducting testing and reviews in an iterative model, there is an opportunity to fix at every instance. Thus, early detection and fixes result in improvement of product over successive Sprints.
- Early Prototype: Everyone gets to see how the end product will look like, any changes can be quickly incorporated.
- One Team Approach: Due to daily scrums and frequent interactions the team develops a truly one-team bond that delivers better and faster.
- Working software: Getting a working software is the Key. Planning and building for months resulting in an inadequate software is the nightmare for any project team. This risk is very efficiently handle due to agile testing approach
Knowing all these warnings signs & benefits, project teams reap the benefit of successful product as end result. The It is flexible, quick to see results, quick to incorporate and implement changes. Minimal essential documentation and high-quality software / product output. If we compare these things with a traditional approach one can easily see the immense benefits. So, let’s get going and Be Agile!
DevOps!! The new trend coming in the Industry….
Let’s see the whole world of IT Industry with the lens of DevOps. DevOps does not go with only one thing, it has got the combination of Continuous Integration, Delivery, Continuous Deployment and more. DevOps can give the beautiful experience to the Industries handling large customer volume, bringing more enhancements on Automation services and Product Life cycle. As every business needs to stay agile, wants to improve their speed and innovate faster, DevOps methodology supports quality assurance and risk management factors.
DevOps equals Continuous Delivery. DevOps is a (combination of development and operations) software development method in which the operations and the development engineers participate together in the entire service lifecycle, from design through the development process to production support helping an organization rapidly produce software products and services. DevOps practices arena started nearly around 2007.DevOps innovate faster.
Dev and Ops are the two different terminologies which shows that the developers work with Ops to understand the impact of code changes also developer works more closely with production equivalent systems.
DevOps an evolution in the Software Delivery and what does it address?
Several times we have come across conflict conversations like “It’s not the server, it’s your code”, “It works on my machine” etc. What can be the major reason behind this conflict?? There might be the lack of synchronization between the development and the operations or infrastructure team?? Well, the answer to this question is “Yes”. The above conflict can be reduced by adopting DevOps. The development team is responsible for the creating and modifying the code on the other hand the operations team is responsible for creating stability and enhancing services on the developed code. With today’s constant pressures to provide new and innovative services—IT organizations must encourage and institute a culture of continuous learning and improvement. DevOps, a relatively new approach that draws on agile IT development but goes far beyond, can help IT achieve these goals.
DevOps brings out the collaborative mindset between Devs and Ops covering all the aspects of functional and non-functional requirements of an Industry. The complex banking applications takes months or up to a year to develop and launch. However, with the adoption of DevOps methodology and the tools the entire process can be completed in the matter of weeks after rigorous testing.
Before adopting the DevOps methodology, the development and the operations teams used to work individually for the launch of the application. After the introduction of the DevOps the teams collaborated with the single shared objective enabling Agile development and continuous delivery. The API’s introduction made the development process and helped in enhancing the existing services. Performing the non-functional testing approaches from the cloud became a part of it. Thus, DevOps has got great advancements within itself.
Agile and DevOps – How is it connected?
How Agile and DevOps works together is now a question in our mind. Basically, the answer could be “DevOps is an extension of Agile Methodology” Or “Agile Methodology works for DevOps software development processes and product/application delivery enhancements”.
The end to end Automation and the Collaboration between the Dev and Ops teams are the two key ingredients for making Agile and DevOps successful. The question also arises that what can be the use of Testing in DevOps? There are various testing tools (Jenkins a CI tool) and the activities which we need to sink along with the development process in order to help or to speed up the application delivery in a transparent manner. When the developer deploys the build to the QA and the QA has to run the test automatically providing the test results to give an indication where the build can be deployed to the production. As always said finding a bug in an earlier stage is less expensive. Hence, the testing is triggered automatically when the build is available. The diagram below shows how the Agile and DevOps works together hand in hand.
Agile deals with the processes like scrums, sprints etc whereas DevOps deals with the technical practices like CI etc.
How DevOps solves the problems that the Banking Industry is facing????
We all know that the many people are doing the online banking like never before. According to 2016 MX customer survey, clients find it more important to have an easy digital banking experience (67%) rather than a friendly teller or staff (33%) when choosing where to open an account and contact their banks mainly through mobile devices. The 3 main priorities that have been listed by the banking IT executives to enhance the customer service experience and banking software’s: –
- Redesign or enhance the digital experience
- Find ways to reduce operating costs
- Enhance data analytics capabilities to enhance customer needs
In the highly competitive banking industry, we always look to leverage the power of Software. As IT is becoming the heart of the Banks, with DevOps we can have it all (The cost, The Time, The Quality and The Delight). The large automated banking applications and the team have noticed an integration issues with the older codes. The Continuous Integration (Each new application build is automatically deployed and regression tests are triggered against the build) and the automation implementation ideas can bring out the best in the DevOps delivery chain.
Well with all the advancements DevOps is becoming the predominant industry solution. In summary, DevOps is evolving rapidly and each of these pieces provide a slightly different outlook on 2017
“In God we trust, the rest we test” is a very popular line in testing world. Testing team or testers are a very important cog in the wheel as far as product development life cycle is concerned.
A tester essentially conducts prescribed tests on programs and applications for quality, design integrity, and functionality, before they are implemented. A good software tester applies punctilious testing methods and considerable end-user simulations to expose program bugs so that they can be eliminated by the software programmers.
What makes for an excellent tester? Along with possessing a very refined skill set that allows them to do their job well and add value to a development project, a software tester needs to have a few qualities that will make him excel in his field.
Although this is not a complete list, I believe following are traits that an excellent tester should have:
- The ability to analyse data and be curious:
Creating scripts, and executing them numerous times is not a big deal. A good tester does not limit himself to just testing, he analyses and understands the data collected from testing to study the specific behaviour of the application or the product. He goes in depth and fishes out the root cause of bugs and thoroughly analyses the test environment, test data, interruptions etc. He should find out everything about the product under test.
- The knack of good reporting:
A good tester meticulously works for hours together to execute a number of test cases, mark them in management tool, and project a status. An excellent tester along with that, knows exactly how to report it to be understood best. He reports the status to the clients in few crisp sentences as to what bug numbers he found and what will be the next action and not just how many tests were done during the day.
- The will to be a constant and quick learner:
A primary difference between a good software tester and an excellent one is the former is satisfied after gaining expertise while the latter leaves no stone unturned in keeping up with the latest in technology and automation tools. An excellent tester learns to create new ideas, and then form an experience that comes from constant thriving. The ability to grasp the essence and do it quickly is absolutely key.
- The ability to give attention to detail:
Often a software tester only has his eyes set for the glaring issues but fails to look past the obvious and find out small things that can have a trickle-down effect on the whole project. An excellent tester therefore is the one who pays great attention to details and picks out the little issues that affect the big picture of the project.
- The flair to communicate well:
A software tester is often in very close contact with the developers, business analysts, and the major stakeholders in a project. Effective (verbal and written) communication skills therefore are necessary. An excellent tester has great communication skills and therefore he has the ability to understand requirements, describe test criteria and is able to explain how to recreate issues. Apart from good communication skills, a tester also need to have the courage to vocalize concerns and ask doubts.
- Ability to think outside the box:
Any tester can get involved in functional tests and use cases but it takes an excellent tester to push the envelope and strive hard to think outside the box. Users always don’t end up handling the software just as intended. Thinking about situations that aren’t ideal, analysing environmental conditions, and doing case based test on applications are signs of a great tester.
- Highly adaptable to varying situations:
When a project is moving rapidly, requirements sometimes can change in a jiffy, features can shift focus and deadlines can move forward or backward. An excellent tester therefore has to be highly adaptable to different situations.
- Ability to code (programming skills)
An excellent tester will have the capability to at least read and code as to understand the intricacies of product under test. Also the reasonable amount of programming skills will help during automation testing as tester can create tools and snippets of codes needed to automate the manual testing.
The movement of the software industry has been rapid in a short span of time. People’s attention span has become shorter, there is a steep rise in the need for faster deployments and it has become tough to keep tabs on the rapidly changing technology. The industry’s need for quality testers is fast increasing and in this tough game, only the best will survive to do work that is truly worthwhile.
Any important or additional traits that need to be part of this list?
I have been leading Verinite Labs for 6 months now where the mandate has been to develop our own IP by building technology solutions for some of the problems faced by the financial institutions. Being from services background, these 6 months have given me an interesting perspective about lot of things and one of them is about an off-running argument of “Whether today’s testers need programming skills?”.
In a product team, a tester plays a unique and an important role. He is an advocate for the end user while collaborating with his team in maintaining the right agility in its operations and the product development effort.
To be able to understand the product under development more technically, to automate tests, develop test frameworks, and customize tools to meet the team’s testing needs, programming skills come handy.
Testers come under 7 categories:
- User expert
Every tester type has its own value, plus a tester does not have to confine himself to a particular category.
Usually, most testers come with an engineering background, and the least coding skills. They lose touch though if they don’t get sufficient opportunities to code.
Testers can, if they wish, ensure they don’t let their coding skills rust. For that they need to do periodic static code reviews, work on test automation, review existing test automation code etc.
Some argue that since all testers don’t do test automation, all testers need not have programming skills. Because testers who do exploratory testing, bring along a different yet valuable set of skills to the team. They have good analytical skills, investigative skills, and a deep understanding of risk, and other hidden issues along with some technical knowhow like databases, networks, system administration etc. Some of the very best testers don’t know even a bit of programming.
Some others argue that coding and testing skills should not be mutually exclusive.
They say that if a tester has programming skills it will benefit him and the organisation immensely.
Here are a few benefits:
- Ability to code, makes the tester comfortable in owning and preparing test environments. He can pick out the problems, ask for help, and resolve matters more quickly. He is also more confident of dealing with web servers, DBs, Message queues etc., and can write scripts to maintain, monitor, and prepare test environments.
- If the tester is comfortable with coding, it is easier to drive testability. Even a fairly good understanding, of say what might make underlying code more testable, can help a tester discuss test only end points, hooks in the system, and test only configurations with the developers. He can also appreciate the limitations to implementing them if any.
- Programming knowledge helps a tester see the boundaries and layers of underlying systems more clearly. This helps him avoid duplication of test efforts as well as gives him an opportunity to focus on things which are important to test from the system’s point of view.
- The Ability to code gives the tester a better understanding of the release process. He can appreciate the different branching strategies and can see the importance of continuous integration. It can also aid discussions of merits and risks of continuous deployments. It becomes easier for him to own continuous integration and automated testing.
- A tester with programming know how can better understand the complexities of development. They find it easier to spot risks, find problems, and speak the language of developers. They are more in the position of contributing meaningfully in technical discussions.
So there are many reasons why skilled testers should build their programming skills so that they are able to read, write and understand code.
In the end, with areas such as artificial intelligence, big data and interactive computing, manifesting themselves in several domains, technology is growing at a very rapid pace. So in today’s world, the value and the versatility a tester brings with his programming skills will go a long way in differentiating him and his testing team. So it makes more sense to start understanding the value of a tester with programming skills rather than making it mandatory for testers to now programming.
Challenges in Testing the Banking Application
In a continuous endeavor to leverage technology, improve Cost-to-Income ratio, and enhance the services offered to customers, banks keep upgrading their systems and introduce new customer touch points. Innovative products and solutions are launched to make banking even more safe, secure and convenient for customers. With increase in complexity of these systems, testing of the banking applications becomes more and more challenging.
Typically a banking application has to undergo following testing to ensure that it meets the goals:
- Functional Testing
- Security/ Penetration testing
- Performance Testing
- Compatibility / Usability testing (for Applications with Customer facing Interface)
Let’s discuss some common challenges faced while conducting each of these testing.
- To meet the ever changing business need of banking industry, the applications are made highly configurable. Introduction of new parameters increases the number of functional paths that should be verified during testing by manifolds. All these possible scenarios along with the boundary conditions should be rigorously tested. This increases complexity in Test design phase. Covering all the relevant and critical scenarios within the stipulated time of a project can be a daunting task
- With the increase in customer touch points like internet banking, mobile apps, self service kiosks to list a few, testing should make certain that specific scenarios for all access channels are thoroughly covered. Simulation and capturing of test results for all these channels is an eternal challenge
- Compliance and regulatory requirements vary from region to region. Any change in these requires regression testing of the system. Understanding the changes and translating these changes into system impact and test scenarios can be complicated at times
- Lastly, importance of proper data can’t be undermined for a complete and effective functional testing. However many of the times it is difficult to prepare the required input data to simulate all scenarios
Security / Penetration Testing:
- Owing to the sensitivity of data, banking applications are most vulnerable to hacking, spam and other fraudulent activities. As more number of end points are added to an application, more avenues are presented to the hackers through which an application can be attacked. To ensure data security, all these channels and the types of accesses needs to be tested thoroughly
- Different countries have different bank secrecy laws for customer data protection. Moreover, international data security norms are updated regularly to keep the standards abreast with new security threats. This makes the testing even more essential and tough
- Most of the performance testing scenarios can’t be simulated in test environment, therefore a near simulation or simulation through some external tool is used for testing. This limits the effectiveness of testing as the actual issues can’t be traced which might arise in production later
- Normally the test environment does not have the same processing capacity as the production environment. So most of the time some issues are attributed to the limited capacity of the test environment. For a tester to prove that the errors are due to some flaws in application and not due to the limitation in test environment is very difficult
- Moreover, creating and maintaining the extensive range and volume of data similar to production is a tedious task
Compatibility and Usability Testing:
- The ease of usage of any customer facing application decides its fate. The same applies to banking applications as well. The application should be user friendly and should be accessible to bank’s targeted socio-economic segments. In some cases the success of an application can help the bank to acquire, retain and grow their customer base. This aspect should be envisaged and covered in testing
- A customer can access a banking application using different platforms like different browsers, devices or networks. In order to provide an optimal user experience, bank needs to ensure that the look and feel of applications on any of the access medium remains the same. Hence testing should cover compatibility across all platforms while adhering to their security standards. Also, scalability of the design to accommodate new delivery channels should be considered. This makes the scope determination and execution of compatibility testing for any customer facing banking application a massive task
After considering all these challenges, there is no doubt that software testing for banking application is an arduous task, where so many things can go wrong.
But, as per the famous quote from Mosher’s law of software engineering:
“Don’t worry if it doesn’t work right. If everything did, you would be out of a job.”
Test data helps us examine a software application and determine its functionality. The data used to test the features of a newly developed software program is simply, thus, called test data. It helps identify bugs provided it is properly designed. If the test data itself is faulty or of low quality, it will not do a good job identifying bugs and testing features. Needless to say, it will directly affect the quality of the software.
Generation of Test Data
Since the test data determines the quality of software testing and the development of the software application, you need to keep a close eye on its generation. Generation depends on the type of testing environment and software to be tested. It can be done in different ways:
- Using automated test data generation tools
- Mass copy of test data taken from client systems
- Mass copy of test data from production to testing environment
Please keep in mind that test generation is quite complicated and requires time. In fact, in most cases, test generation takes a lot more time than the testing itself. However, it is mandatory that you pay close attention to this step because it will affect the quality of test data, thereby affecting the software quality.
Ensuring Quality of Test Data:
White Box Testing
For this type of testing, the test data can be extracted from the code directly. The quality can be ensured considering the following:
- All program source code branches should be tested at least once.
- All paths must be tested at least once.
Black Box Testing
Since the tester cannot see the code in this case, certain criteria should be met in order to confirm quality of the test data.
- No data: Do not submit any data (blank data) and see whether the system responds with error messages.
- Valid data: Submit valid data and see if it is being saved. Also check if the system is functioning as expected.
- Invalid data: Submit invalid data to ensure proper behavior for alphanumeric inputs and negative values.
- Illegal data format: The system should generate error messages and refuse to accept the data format.
- Boundary condition data: This will help identify response to lower and upper boundary conditions.
- Performance, load, and stress testing data: This data set will be voluminous and will help you determine the efficiency of the test data.
Security testing revolves around protection against malicious content. The test data designed to test security software must satisfy the following:
Likewise, parameters to ensure the quality of test data vary with the software application. If the applications are complicated, going with automated test generation tools is the safest way to guarantee quality. As aforementioned, it is the quality of the test data that will influence the quality of the software. That is why putting it under the microscope is highly recommended.
Do you really need Test Automation?
Last week we were having a call with one of our Clients about some additional Quality Assurance services that our company has to offer. Test automation was also a part of it. After a fifteen – twenty minutes of detailed discussion on the additional testing services, they asked us a simple question “Do we really need Test Automation?” The question was a straight one and they were expecting a “Yes” or “No” as an answer but for us the answer was not that simple.
As per the data available in public domain, around 63% of all Test automation projects have failed to achieve expected goals and around 50% of procured test automation tools have never been re-used after completion of a project. With these facts in mind, one can easily consider Test automation to be a rather risky proposition. But before we draw any conclusion, we need to consider how the successful projects (remaining 37%) made it work in their favour.
For any successful implementation of Test automation, it is critical that we evaluate and confirm whether test automation is actually required. There are so many myths built around test automation which most organizations believe it’s a panacea to all their quality related ailments. So we need to look beyond those myths and conduct a proper evaluation to determine its usefulness
An example: Most of the time the very objective of starting a test automation activity is to save time, cost and efforts in a project. But on the contrary Test automation involves higher initial investment to procure tools and conduct training, needs specialized resources (hence effort) and additional time to prepare automation scripts. The benefit of test automation can only be realized over a period of time depending on the reusability of scripts and frequency of execution.
Therefore performing a ROI analysis on each automation initiative is an objective way to conclude the type and the extent of automation that is required / beneficial for a project.
A simple analysis can be done by executing these following steps to determine the need of automation for a project. (There are some cool tools available in the market to conduct this ROI analysis, but my suggestion is use an excel sheet with your own parameters. It’s much simpler, quicker and cost-effective way to come to a conclusion)
- Step1: Calculate Potential Effort and Cost savings through Automation
Following points need to be considered while coming up with total potential effort and cost saving figures
- All activities in a Testing life cycle should be considered. Remember, test automation is not limited to test execution. It can also be used to reduce effort for activities like test environment setup and test data creation
- Break down all these activities to combination of independent task groups
- Consider the frequency and number of repetitions for each task group as per Test plan and Test scripts
- Assign a reusability index (i.e. probability of reusing the same task in future projects) for each task group
- Step2: Calculate Total Cost of Automation
Following points need to be considered while calculation total cost
- Technology requirement for automation of tasks
- Cost of Tools that supports automation for these technology
- Skill requirement for implementing test automation
- Overall effort for building automation scripts for the required task groups
- Step 3: Optimize the scope of Automation
- Most of the time organizations end up trying to automate each and every step of Test cycle
- Now if we go by this approach, more often than not we’ll end up having a negative ROI result. Because automation only make sense when it can be reused over a long period of time.
- Therefore we need to prepare multiple automation models and subject them to the ROI calculation process to come up with an optimized automation model which can be implemented for the project.
- Automation models are created by varying the scope of automation from everything to nothing. Different task groups with less frequency, less repetition and less reusability index can be removed strategically from the scope of automation to come up with multiple models.
Based on the above process an optimized Automation model can be derived for the project, which can add value and help to achieve the goal of automation for the client successfully
Now that we’ve decided an optimized automation model for the Project, the next step is to implement the same. Next time we’ll discuss the approach for successful implementation of Test automation in a project.
In the meanwhile please feel free to share your queries and views on the topic through comments in this blog.