As the world is moving faster with emerging new technologies the need of improving Test automation also grew. Now when large data is required for testing within less time latest technologies such as Data science and AI can be helpful to improve Test Automation. Where data science can help in generating data from different sources AI can help in creating codeless automation based on data generated.
What is Data Science and AI?
Data Science is a blend of various tools, algorithms, and machine learning principles with the goal to discover hidden patterns from the raw data. Data Science is primarily used to make decisions and predictions making use of predictive causal analytics, prescriptive analytics (predictive plus decision science) and machine learning.
Artificial intelligence (AI) is an area of computer science that emphasizes the creation of intelligent machines that work and react like humans. Some of the activities computers with artificial intelligence are designed for include: Speech recognition. Learning, observation.
Need of Data Science
Traditionally, the data that we had was mostly structured and small in size, which could be analysed by using the simple BI tools. Unlike data in the traditional systems which was mostly structured, today most of the data is unstructured or semi-structured. This data is generated from different sources like financial logs, text files, multimedia forms, sensors, and instruments. Simple BI tools are not capable of processing this huge volume and variety of data. This is why we need more complex and advanced analytical tools and algorithms for processing, analysing and drawing meaningful insights out of it. Let’s dig deeper and see how Data Science is being used in Banking domains.
How about if you could understand the precise requirements of your customers from the existing data like the customer’s past browsing history, purchase history, age and income. No doubt you had all this data earlier too, but now with the vast amount and variety of data, you can train models more effectively and recommend the product to your customers with more precision. Wouldn’t it be amazing as it will bring more business to your organization?
Let’s take a different scenario to understand the role of Data Science for Fraud Detection. What if customer who always makes transaction less than 500 rupees per day and suddenly his transaction spikes to 10000 a day isn’t is suspicious. In such case Data science can help in early prediction and restrict the account from making such transactions . Based on customers previous transaction history, location history, income history and data from other sources like social media will help data science to predict and act against the fraud.
How Data science and AI will improve Test automation?
Traditional testing techniques still rely on humans to source and analyse data. But let’s just say that humans are not infallible and are quite prone to making poor assumptions.
The less time there is for handling data, the greater the chance that testing will produce skewed results with overlooked bugs in the software. Before you know it, consumers will pick up on these bugs, which usually leads to frustration and undermines the brand’s reputation.
That’s why machine learning, which teaches systems to learn and apply that knowledge in the future, makes software testers come up with more accurate results than traditional testing ever could. Not to mention that the probability of error is not the only thing that gets reduced. The time needed to perform a software test and find a possible bug is also shortened, while the amount of data that needs to be handled can still increase without any strain on the testing team.
In Traditional Testing process human creates the test scripts on the assumption and understanding of the application but with help of Data science System can gather the real-time data from different sources identify the hidden pattern and provide new rare critical scenarios also which could help in improving the quality of Product.
With the help of data generated from Data Science Predictive analysis AI can build the test scripts Based on this AI can start creating test cases on real user data. It is smart enough to identify commonly used actions such as logging in/out of the application and cluster them into reusable components. Then it injects these newly created reusable components into our tests as well. Now, all of a sudden, we already have actual tests written by the AI based on real data, along with reusable components that can be used within other tests as well.
Some of the biggest obstacles keeping companies from moving forward with automation is the amount of time and effort it takes to write and execute tests with the chosen tool or framework and the availability of skilled resources to do this task. There are some AI tools that overcomes this issue. Test that use to take multiple weeks can be done now within few hours of time. This is achieved by creating reusable components, run test quickly integrate CI/CD with different grids.
As the test time is reduced the productivity of the test automation is increased and also non-technical person can perform the automation with help of AI.
One of the most common problems with test automation is maintenance. For example, say we have 100 automated tests running on a daily basis to ensure the main functionalities of the application are still stable. What if the next day we come back to work and find that half of the tests have failed? We would need to spend considerable amounts of time to troubleshoot the failures and investigate what actually happened. This involves figuring out ways to fix the failures and implement the fixes. Then, we re-run the automated tests to ensure everything passes.AI can help here:
Resultant tests are modelled and thus maintained by the combination of an exhaustive and autonomous set of data points, such as the size of element, location on a page, previously known size and location, visual configuration, XPath, CSS selector, and parent/child elements.
Root cause analysis highlights all potential causes for test failure and provides a path for one-click updates.
Selector maintenance should be eliminated by having elements identified by hundreds of data points that are rated and ranked instead of a single selector.
Machine learning gives testers the opportunity to better understand their customers’ needs and react faster than ever to their changing expectations. In addition, testers now also need to analyse more and more data and they are given less and less time to do that, while their margin of error decreases constantly. Data Science and AI offer a way to address these challenges. This approach is set to fill the gaps of traditional testing methods and make the entire process more efficient and relevant to the users’ needs.
Card Management system migrations and conversions are lined up globally across credit card issuers. Credit card domain is witnessing a sharp rise in platform migrations primarily due to:
- Regulatory challenges – Guidelines from national central banks regarding consumer data placement and security
- Financial institution mergers – shifting economies resulting in merger & acquisitions in BFSI sector
- Institutions opting for processor platform instead of in-house processing
- Keeping pace with changing technologies – Migration to more robust platforms
- Product upgrades – Platform upgrades with latest version
All migration initiatives have common factor for success – Testing. A comprehensive testing ensures successful portfolio switch over with No to Minimal impact on Bank and customer’s business as usual activities.
This blog will focus on key migration testing challenges and possible control steps.
Migration testing projects typically have 4 phases:
Let’s list down associated challenges at each stage:
- A medium to large scale migration program may have an all-inclusive timeline ranging from 6-12 months. It is practically impossible to freeze process changes on source platform for such duration due to swift changing customer behavior and industry demands. These changes result in scope extension and also impact the next phases – Planning/Execution.
- Another scope challenge is inadequate documentation. Over the number of years with old platform, a considerable change gets seed in system leaving behind little to No documentation due to repository miss management or resource movement. This creates a scope GAP in certain areas which only surfaces during test execution phase
- A Business requirement traceability matrix must be maintained with a track of changes during migration
- Testing teams should be agile enough to include such changes in planning and execution cycle
- BAU/Business operation teams must be consulted for a broad scope review. This will help to identify any missing process. This approach can act as a supplement of missing documentation.
- Planning timeline: Most clients plans to have a larger execution window rather than spending time on proper test planning. This is done in order to reduce overall migration timeline. Shorter timeline limits the comprehensive test coverage and at many instances, plans are revised during execution to accommodate findings.
- Final parameter set up on migrated platform is generally not available during planning phase and a base parameter oriented planning is conducted. This result in frequent revisit of test plans as soon as final parameters starts flowing in.
- Batch processing, most of the card processing activities like Due calculation/Payments/Collections etc. takes place in batch mode. This puts an extra load on test planning as different functions will have different batch requirements such as Daily batch/Month End batch.
- Discussions with product teams can help to arrive at a near right parameter set up on migration platform. This will reduce the planning update during execution phase
- Always consider batch requirements during planning phase. Cases with specific batch requirements must be clubbed.
- Testing consultants with prior migration expertise and re-usable test repositories can significantly reduce test planning time
Even in ideal scenario, it is not feasible to clear all challenges in scope and planning phase, as a result certain challenges will always flow into execution phase.
- New system readiness – Test environment preparation and readiness sometimes eats up the execution timelines due to interface readiness, correct parameter set up and batch performance
- Change in solution/business process – such changes impacts the test plans followed by execution plan updates
- Blocker defects – This area is extremely crucial for execution. Identification and closure of blocker defects has an important role to play in smooth execution and over all test timeline
- Execution teams must closely monitor system and interface readiness and adjust execution plan as per system availability
- Change is solutions should be scoped in a separate execution plan to minimize impact on existing execution plan
- Execution team must set a function wise execution priority and monitor the business requirement traceability matrix
- All major functions and business process related base parameter set up should be verified at execution start time to identify blocker defects at beginning of execution phase
- At this phase, test activities are at its last phase and a sum up of execution results is in process. This phase identities the system readiness to proceed with production implementation. This phase is not encountered with multiple challenges however; one common challenge is Go/No-Go decision. Test team plays a crucial role to assess project state and recommends on the Go/No-Go decision based on execution test results.
- All open defects must be carefully observed and should be assessed as per severity
- A walkthrough of all open items should be arranged with working group for final assessment
- Test teams must ensure that all stakeholders understand the open defects and associated risk
- A formal agreement should be made for Go/No-Go recommendation
In the end, these challenges may vary on case to case basis. Above expected challenges and its awareness can be helpful to identify associated risks and its mitigation plan.
Over the years the demand is growing for the team be operating in Agile fashion and with Agile Testing methods almost at every project location. Project manager wants model or prototype built quickly and testing to be performed in an iterative manner. Obviously, there are advantages of this iterative short cycle testing & the benefits are enormous.
Agile testing method focuses on an incremental, short term deliveries that allows frequent review responses & continuous feedback mechanism. Last week we have seen how Agile mindset helps in project execution, various roles and terms associated with it. In this blog we are going to see how a testing team performs testing using agile methodology. The quintessential difference is that the testing is not done during only one phase, but testing is done iteratively with every release of the software or the product. This way of testing early and repetitively ensures minimal damage and risk towards the end of project Everyone would have seen the classic curve of time vs cost when defect is identified & cost of fixing it. In Agile approach this is exactly the area that testers can best tackle for you. For them to achieve this project manager and project teams must have an agile mindset.
Let’s see how exactly an agile tester can contribute and participate effectively in an agile project environment.
- Attend Sprint Planning Session.
As a member of Agile Team, quality leads and tester should always attend Sprint Planning sessions. This helps to ensure that they are in sync with the development team from the start and allow QL to identify and highlight possible problem arrears and risk as early as possible. During the Sprint Planning, developers estimate the effort it will take them to write code, similar way testers should estimate effort required for testing code during planning session. So, resulting in a win-win situation for developers and testers both.
- Attend Daily Stand-up meeting.
QA Tester should attend daily stand up meeting with the developers, Scrum Master and the Product Owner. This promotes a strong collaboration amongst the team members and a sense of one-team prevails. Based on the updates received from the Developers, Business Analysis and Testers, product owner stays up to date with how the sprint is going on and teams are able to plan their workload efficiently. In case, if the tester has a blocker, they can bring it up in the daily stand-up meeting and update every team member on known issue which in turn allow developers to speed up testing progress and plan their workload.
- Don’t wait to report bugs, report it immediately & frequently throughout the Sprints:
This is very important, in order to deliver high quality of software in a short period of time your team needs to work efficiently. Any issue that is found during sprint must be reported immediately in the same Sprint. Clubbing all the incidents and reporting only at the end does not help in agile setup. If this happens then closing the sprint becomes difficult and makes it ineffective, given that it is a time boxed activity. By continuous integration of testing and development, it allows both the teams to work together and resolve their issue faster and this help in leading to higher quality of results.
- Be an active participant of Sprint Demo
At the end of each Sprint, tester and developer teams demonstrate the product features that are developed to the product owner and other stakeholders. Any feedback has been received related to look and feels & other features that is incorporated on immediate effects. This help to deliver the product with high quality customer satisfaction and meet the business requirement.
- Document the test cases
Although in agile project where documentation is minimum, due to frequent requirement changes, testing team cannot afford to skip necessary documentation. Documentation is very important for QA. Execution for test cases & pass logs is the proof of successful testing and is enquired to be documented at various levels. This becomes useful for Bank internal or external audit requirements. This also covers the scenarios where a person is replaced by another one.
- Attend Sprint retrospectives
At the end of each sprint, team needs to discuss and provide review comments about what went well & what did not work during the sprint. This helps to improve in future sprint planning and execution. Sprint retrospective is the opportunity to define weaknesses and determine solutions for them. QA need to be involve in these discussions for any concerns that needs to be addressed before the next Sprint can start.
There are certain cautionary warnings that must be taken care during the agile testing:
- Avoid delays in delivering a story point. This will have cascading effects on QA, testing and eventually in achieving the sprint goal.
- The scope and vision must be clear for a sprint. Any changes identified during the sprint should be parked & then incorporated in the next sprint. There is always a temptation to include everything in the on-going sprint. This approach can prove counterproductive and losing the sprint altogether. Hence must be avoided.
- Try to avoid falling in a trap of using Agile as an excuse for missing documents. Agile does not mean no documentation; bust it means minimal documentation that is essential. Ever increasing scope and improvement feature, just because it is agile project one should not keep adding features, wish lists and ideas to the existing project. There must a definite start and an end. Otherwise the project can become an eternal agile project.
The contribution of agile testers in various ways listed above, results in multiple benefits such as:
- Improvement in successive Sprints: By producing frequent builds and conducting testing and reviews in an iterative model, there is an opportunity to fix at every instance. Thus, early detection and fixes result in improvement of product over successive Sprints.
- Early Prototype: Everyone gets to see how the end product will look like, any changes can be quickly incorporated.
- One Team Approach: Due to daily scrums and frequent interactions the team develops a truly one-team bond that delivers better and faster.
- Working software: Getting a working software is the Key. Planning and building for months resulting in an inadequate software is the nightmare for any project team. This risk is very efficiently handle due to agile testing approach
Knowing all these warnings signs & benefits, project teams reap the benefit of successful product as end result. The It is flexible, quick to see results, quick to incorporate and implement changes. Minimal essential documentation and high-quality software / product output. If we compare these things with a traditional approach one can easily see the immense benefits. So, let’s get going and Be Agile!
DevOps!! The new trend coming in the Industry….
Let’s see the whole world of IT Industry with the lens of DevOps. DevOps does not go with only one thing, it has got the combination of Continuous Integration, Delivery, Continuous Deployment and more. DevOps can give the beautiful experience to the Industries handling large customer volume, bringing more enhancements on Automation services and Product Life cycle. As every business needs to stay agile, wants to improve their speed and innovate faster, DevOps methodology supports quality assurance and risk management factors.
DevOps equals Continuous Delivery. DevOps is a (combination of development and operations) software development method in which the operations and the development engineers participate together in the entire service lifecycle, from design through the development process to production support helping an organization rapidly produce software products and services. DevOps practices arena started nearly around 2007.DevOps innovate faster.
Dev and Ops are the two different terminologies which shows that the developers work with Ops to understand the impact of code changes also developer works more closely with production equivalent systems.
DevOps an evolution in the Software Delivery and what does it address?
Several times we have come across conflict conversations like “It’s not the server, it’s your code”, “It works on my machine” etc. What can be the major reason behind this conflict?? There might be the lack of synchronization between the development and the operations or infrastructure team?? Well, the answer to this question is “Yes”. The above conflict can be reduced by adopting DevOps. The development team is responsible for the creating and modifying the code on the other hand the operations team is responsible for creating stability and enhancing services on the developed code. With today’s constant pressures to provide new and innovative services—IT organizations must encourage and institute a culture of continuous learning and improvement. DevOps, a relatively new approach that draws on agile IT development but goes far beyond, can help IT achieve these goals.
DevOps brings out the collaborative mindset between Devs and Ops covering all the aspects of functional and non-functional requirements of an Industry. The complex banking applications takes months or up to a year to develop and launch. However, with the adoption of DevOps methodology and the tools the entire process can be completed in the matter of weeks after rigorous testing.
Before adopting the DevOps methodology, the development and the operations teams used to work individually for the launch of the application. After the introduction of the DevOps the teams collaborated with the single shared objective enabling Agile development and continuous delivery. The API’s introduction made the development process and helped in enhancing the existing services. Performing the non-functional testing approaches from the cloud became a part of it. Thus, DevOps has got great advancements within itself.
Agile and DevOps – How is it connected?
How Agile and DevOps works together is now a question in our mind. Basically, the answer could be “DevOps is an extension of Agile Methodology” Or “Agile Methodology works for DevOps software development processes and product/application delivery enhancements”.
The end to end Automation and the Collaboration between the Dev and Ops teams are the two key ingredients for making Agile and DevOps successful. The question also arises that what can be the use of Testing in DevOps? There are various testing tools (Jenkins a CI tool) and the activities which we need to sink along with the development process in order to help or to speed up the application delivery in a transparent manner. When the developer deploys the build to the QA and the QA has to run the test automatically providing the test results to give an indication where the build can be deployed to the production. As always said finding a bug in an earlier stage is less expensive. Hence, the testing is triggered automatically when the build is available. The diagram below shows how the Agile and DevOps works together hand in hand.
Agile deals with the processes like scrums, sprints etc whereas DevOps deals with the technical practices like CI etc.
How DevOps solves the problems that the Banking Industry is facing????
We all know that the many people are doing the online banking like never before. According to 2016 MX customer survey, clients find it more important to have an easy digital banking experience (67%) rather than a friendly teller or staff (33%) when choosing where to open an account and contact their banks mainly through mobile devices. The 3 main priorities that have been listed by the banking IT executives to enhance the customer service experience and banking software’s: –
- Redesign or enhance the digital experience
- Find ways to reduce operating costs
- Enhance data analytics capabilities to enhance customer needs
In the highly competitive banking industry, we always look to leverage the power of Software. As IT is becoming the heart of the Banks, with DevOps we can have it all (The cost, The Time, The Quality and The Delight). The large automated banking applications and the team have noticed an integration issues with the older codes. The Continuous Integration (Each new application build is automatically deployed and regression tests are triggered against the build) and the automation implementation ideas can bring out the best in the DevOps delivery chain.
Well with all the advancements DevOps is becoming the predominant industry solution. In summary, DevOps is evolving rapidly and each of these pieces provide a slightly different outlook on 2017
“In God we trust, the rest we test” is a very popular line in testing world. Testing team or testers are a very important cog in the wheel as far as product development life cycle is concerned.
A tester essentially conducts prescribed tests on programs and applications for quality, design integrity, and functionality, before they are implemented. A good software tester applies punctilious testing methods and considerable end-user simulations to expose program bugs so that they can be eliminated by the software programmers.
What makes for an excellent tester? Along with possessing a very refined skill set that allows them to do their job well and add value to a development project, a software tester needs to have a few qualities that will make him excel in his field.
Although this is not a complete list, I believe following are traits that an excellent tester should have:
- The ability to analyse data and be curious:
Creating scripts, and executing them numerous times is not a big deal. A good tester does not limit himself to just testing, he analyses and understands the data collected from testing to study the specific behaviour of the application or the product. He goes in depth and fishes out the root cause of bugs and thoroughly analyses the test environment, test data, interruptions etc. He should find out everything about the product under test.
- The knack of good reporting:
A good tester meticulously works for hours together to execute a number of test cases, mark them in management tool, and project a status. An excellent tester along with that, knows exactly how to report it to be understood best. He reports the status to the clients in few crisp sentences as to what bug numbers he found and what will be the next action and not just how many tests were done during the day.
- The will to be a constant and quick learner:
A primary difference between a good software tester and an excellent one is the former is satisfied after gaining expertise while the latter leaves no stone unturned in keeping up with the latest in technology and automation tools. An excellent tester learns to create new ideas, and then form an experience that comes from constant thriving. The ability to grasp the essence and do it quickly is absolutely key.
- The ability to give attention to detail:
Often a software tester only has his eyes set for the glaring issues but fails to look past the obvious and find out small things that can have a trickle-down effect on the whole project. An excellent tester therefore is the one who pays great attention to details and picks out the little issues that affect the big picture of the project.
- The flair to communicate well:
A software tester is often in very close contact with the developers, business analysts, and the major stakeholders in a project. Effective (verbal and written) communication skills therefore are necessary. An excellent tester has great communication skills and therefore he has the ability to understand requirements, describe test criteria and is able to explain how to recreate issues. Apart from good communication skills, a tester also need to have the courage to vocalize concerns and ask doubts.
- Ability to think outside the box:
Any tester can get involved in functional tests and use cases but it takes an excellent tester to push the envelope and strive hard to think outside the box. Users always don’t end up handling the software just as intended. Thinking about situations that aren’t ideal, analysing environmental conditions, and doing case based test on applications are signs of a great tester.
- Highly adaptable to varying situations:
When a project is moving rapidly, requirements sometimes can change in a jiffy, features can shift focus and deadlines can move forward or backward. An excellent tester therefore has to be highly adaptable to different situations.
- Ability to code (programming skills)
An excellent tester will have the capability to at least read and code as to understand the intricacies of product under test. Also the reasonable amount of programming skills will help during automation testing as tester can create tools and snippets of codes needed to automate the manual testing.
The movement of the software industry has been rapid in a short span of time. People’s attention span has become shorter, there is a steep rise in the need for faster deployments and it has become tough to keep tabs on the rapidly changing technology. The industry’s need for quality testers is fast increasing and in this tough game, only the best will survive to do work that is truly worthwhile.
Any important or additional traits that need to be part of this list?
I have been leading Verinite Labs for 6 months now where the mandate has been to develop our own IP by building technology solutions for some of the problems faced by the financial institutions. Being from services background, these 6 months have given me an interesting perspective about lot of things and one of them is about an off-running argument of “Whether today’s testers need programming skills?”.
In a product team, a tester plays a unique and an important role. He is an advocate for the end user while collaborating with his team in maintaining the right agility in its operations and the product development effort.
To be able to understand the product under development more technically, to automate tests, develop test frameworks, and customize tools to meet the team’s testing needs, programming skills come handy.
Testers come under 7 categories:
- User expert
Every tester type has its own value, plus a tester does not have to confine himself to a particular category.
Usually, most testers come with an engineering background, and the least coding skills. They lose touch though if they don’t get sufficient opportunities to code.
Testers can, if they wish, ensure they don’t let their coding skills rust. For that they need to do periodic static code reviews, work on test automation, review existing test automation code etc.
Some argue that since all testers don’t do test automation, all testers need not have programming skills. Because testers who do exploratory testing, bring along a different yet valuable set of skills to the team. They have good analytical skills, investigative skills, and a deep understanding of risk, and other hidden issues along with some technical knowhow like databases, networks, system administration etc. Some of the very best testers don’t know even a bit of programming.
Some others argue that coding and testing skills should not be mutually exclusive.
They say that if a tester has programming skills it will benefit him and the organisation immensely.
Here are a few benefits:
- Ability to code, makes the tester comfortable in owning and preparing test environments. He can pick out the problems, ask for help, and resolve matters more quickly. He is also more confident of dealing with web servers, DBs, Message queues etc., and can write scripts to maintain, monitor, and prepare test environments.
- If the tester is comfortable with coding, it is easier to drive testability. Even a fairly good understanding, of say what might make underlying code more testable, can help a tester discuss test only end points, hooks in the system, and test only configurations with the developers. He can also appreciate the limitations to implementing them if any.
- Programming knowledge helps a tester see the boundaries and layers of underlying systems more clearly. This helps him avoid duplication of test efforts as well as gives him an opportunity to focus on things which are important to test from the system’s point of view.
- The Ability to code gives the tester a better understanding of the release process. He can appreciate the different branching strategies and can see the importance of continuous integration. It can also aid discussions of merits and risks of continuous deployments. It becomes easier for him to own continuous integration and automated testing.
- A tester with programming know how can better understand the complexities of development. They find it easier to spot risks, find problems, and speak the language of developers. They are more in the position of contributing meaningfully in technical discussions.
So there are many reasons why skilled testers should build their programming skills so that they are able to read, write and understand code.
In the end, with areas such as artificial intelligence, big data and interactive computing, manifesting themselves in several domains, technology is growing at a very rapid pace. So in today’s world, the value and the versatility a tester brings with his programming skills will go a long way in differentiating him and his testing team. So it makes more sense to start understanding the value of a tester with programming skills rather than making it mandatory for testers to now programming.
Challenges in Testing the Banking Application
In a continuous endeavor to leverage technology, improve Cost-to-Income ratio, and enhance the services offered to customers, banks keep upgrading their systems and introduce new customer touch points. Innovative products and solutions are launched to make banking even more safe, secure and convenient for customers. With increase in complexity of these systems, testing of the banking applications becomes more and more challenging.
Typically a banking application has to undergo following testing to ensure that it meets the goals:
- Functional Testing
- Security/ Penetration testing
- Performance Testing
- Compatibility / Usability testing (for Applications with Customer facing Interface)
Let’s discuss some common challenges faced while conducting each of these testing.
- To meet the ever changing business need of banking industry, the applications are made highly configurable. Introduction of new parameters increases the number of functional paths that should be verified during testing by manifolds. All these possible scenarios along with the boundary conditions should be rigorously tested. This increases complexity in Test design phase. Covering all the relevant and critical scenarios within the stipulated time of a project can be a daunting task
- With the increase in customer touch points like internet banking, mobile apps, self service kiosks to list a few, testing should make certain that specific scenarios for all access channels are thoroughly covered. Simulation and capturing of test results for all these channels is an eternal challenge
- Compliance and regulatory requirements vary from region to region. Any change in these requires regression testing of the system. Understanding the changes and translating these changes into system impact and test scenarios can be complicated at times
- Lastly, importance of proper data can’t be undermined for a complete and effective functional testing. However many of the times it is difficult to prepare the required input data to simulate all scenarios
Security / Penetration Testing:
- Owing to the sensitivity of data, banking applications are most vulnerable to hacking, spam and other fraudulent activities. As more number of end points are added to an application, more avenues are presented to the hackers through which an application can be attacked. To ensure data security, all these channels and the types of accesses needs to be tested thoroughly
- Different countries have different bank secrecy laws for customer data protection. Moreover, international data security norms are updated regularly to keep the standards abreast with new security threats. This makes the testing even more essential and tough
- Most of the performance testing scenarios can’t be simulated in test environment, therefore a near simulation or simulation through some external tool is used for testing. This limits the effectiveness of testing as the actual issues can’t be traced which might arise in production later
- Normally the test environment does not have the same processing capacity as the production environment. So most of the time some issues are attributed to the limited capacity of the test environment. For a tester to prove that the errors are due to some flaws in application and not due to the limitation in test environment is very difficult
- Moreover, creating and maintaining the extensive range and volume of data similar to production is a tedious task
Compatibility and Usability Testing:
- The ease of usage of any customer facing application decides its fate. The same applies to banking applications as well. The application should be user friendly and should be accessible to bank’s targeted socio-economic segments. In some cases the success of an application can help the bank to acquire, retain and grow their customer base. This aspect should be envisaged and covered in testing
- A customer can access a banking application using different platforms like different browsers, devices or networks. In order to provide an optimal user experience, bank needs to ensure that the look and feel of applications on any of the access medium remains the same. Hence testing should cover compatibility across all platforms while adhering to their security standards. Also, scalability of the design to accommodate new delivery channels should be considered. This makes the scope determination and execution of compatibility testing for any customer facing banking application a massive task
After considering all these challenges, there is no doubt that software testing for banking application is an arduous task, where so many things can go wrong.
But, as per the famous quote from Mosher’s law of software engineering:
“Don’t worry if it doesn’t work right. If everything did, you would be out of a job.”
Test data helps us examine a software application and determine its functionality. The data used to test the features of a newly developed software program is simply, thus, called test data. It helps identify bugs provided it is properly designed. If the test data itself is faulty or of low quality, it will not do a good job identifying bugs and testing features. Needless to say, it will directly affect the quality of the software.
Generation of Test Data
Since the test data determines the quality of software testing and the development of the software application, you need to keep a close eye on its generation. Generation depends on the type of testing environment and software to be tested. It can be done in different ways:
- Using automated test data generation tools
- Mass copy of test data taken from client systems
- Mass copy of test data from production to testing environment
Please keep in mind that test generation is quite complicated and requires time. In fact, in most cases, test generation takes a lot more time than the testing itself. However, it is mandatory that you pay close attention to this step because it will affect the quality of test data, thereby affecting the software quality.
Ensuring Quality of Test Data:
White Box Testing
For this type of testing, the test data can be extracted from the code directly. The quality can be ensured considering the following:
- All program source code branches should be tested at least once.
- All paths must be tested at least once.
Black Box Testing
Since the tester cannot see the code in this case, certain criteria should be met in order to confirm quality of the test data.
- No data: Do not submit any data (blank data) and see whether the system responds with error messages.
- Valid data: Submit valid data and see if it is being saved. Also check if the system is functioning as expected.
- Invalid data: Submit invalid data to ensure proper behavior for alphanumeric inputs and negative values.
- Illegal data format: The system should generate error messages and refuse to accept the data format.
- Boundary condition data: This will help identify response to lower and upper boundary conditions.
- Performance, load, and stress testing data: This data set will be voluminous and will help you determine the efficiency of the test data.
Security testing revolves around protection against malicious content. The test data designed to test security software must satisfy the following:
Likewise, parameters to ensure the quality of test data vary with the software application. If the applications are complicated, going with automated test generation tools is the safest way to guarantee quality. As aforementioned, it is the quality of the test data that will influence the quality of the software. That is why putting it under the microscope is highly recommended.
Do you really need Test Automation?
Last week we were having a call with one of our Clients about some additional Quality Assurance services that our company has to offer. Test automation was also a part of it. After a fifteen – twenty minutes of detailed discussion on the additional testing services, they asked us a simple question “Do we really need Test Automation?” The question was a straight one and they were expecting a “Yes” or “No” as an answer but for us the answer was not that simple.
As per the data available in public domain, around 63% of all Test automation projects have failed to achieve expected goals and around 50% of procured test automation tools have never been re-used after completion of a project. With these facts in mind, one can easily consider Test automation to be a rather risky proposition. But before we draw any conclusion, we need to consider how the successful projects (remaining 37%) made it work in their favour.
For any successful implementation of Test automation, it is critical that we evaluate and confirm whether test automation is actually required. There are so many myths built around test automation which most organizations believe it’s a panacea to all their quality related ailments. So we need to look beyond those myths and conduct a proper evaluation to determine its usefulness
An example: Most of the time the very objective of starting a test automation activity is to save time, cost and efforts in a project. But on the contrary Test automation involves higher initial investment to procure tools and conduct training, needs specialized resources (hence effort) and additional time to prepare automation scripts. The benefit of test automation can only be realized over a period of time depending on the reusability of scripts and frequency of execution.
Therefore performing a ROI analysis on each automation initiative is an objective way to conclude the type and the extent of automation that is required / beneficial for a project.
A simple analysis can be done by executing these following steps to determine the need of automation for a project. (There are some cool tools available in the market to conduct this ROI analysis, but my suggestion is use an excel sheet with your own parameters. It’s much simpler, quicker and cost-effective way to come to a conclusion)
- Step1: Calculate Potential Effort and Cost savings through Automation
Following points need to be considered while coming up with total potential effort and cost saving figures
- All activities in a Testing life cycle should be considered. Remember, test automation is not limited to test execution. It can also be used to reduce effort for activities like test environment setup and test data creation
- Break down all these activities to combination of independent task groups
- Consider the frequency and number of repetitions for each task group as per Test plan and Test scripts
- Assign a reusability index (i.e. probability of reusing the same task in future projects) for each task group
- Step2: Calculate Total Cost of Automation
Following points need to be considered while calculation total cost
- Technology requirement for automation of tasks
- Cost of Tools that supports automation for these technology
- Skill requirement for implementing test automation
- Overall effort for building automation scripts for the required task groups
- Step 3: Optimize the scope of Automation
- Most of the time organizations end up trying to automate each and every step of Test cycle
- Now if we go by this approach, more often than not we’ll end up having a negative ROI result. Because automation only make sense when it can be reused over a long period of time.
- Therefore we need to prepare multiple automation models and subject them to the ROI calculation process to come up with an optimized automation model which can be implemented for the project.
- Automation models are created by varying the scope of automation from everything to nothing. Different task groups with less frequency, less repetition and less reusability index can be removed strategically from the scope of automation to come up with multiple models.
Based on the above process an optimized Automation model can be derived for the project, which can add value and help to achieve the goal of automation for the client successfully
Now that we’ve decided an optimized automation model for the Project, the next step is to implement the same. Next time we’ll discuss the approach for successful implementation of Test automation in a project.
In the meanwhile please feel free to share your queries and views on the topic through comments in this blog.