“The biggest challenge of any data migration project is by far the data itself!” said the product expert. This statement triggered a series of questions in my mind related to the typical challenges faced by the team during a data migration project.
Is it really the ‘Data’ that is an issue? Or is it the ‘Understanding of the Data’ that creates the problem? Are there any ‘other reasons’ which may lead to data issues? I tried to find the answers to these questions during one such recent data migration project.
A data migration project typically follows following stages –
Data Analysis -> Data Mapping -> Data Extraction -> Data Transformation ->Data Loading -> Data Reconciliation
Each stage comes with its unique set of challenges and requires specific corrective actions to ensure a successful migration. This blog is focused around the issues and challenges faced during one such recent data migration project where a mid-sized finance company migrated their credit card portfolio from cards processing environment to an internally hosted licensing card management system solution. Both these card management solutions were industry standard off-the-shelf solutions with multiple customers across the globe. Hence the expectation from senior management was to breeze through the migration without any major hiccups.
But in reality, there were hiccups…. and that too plenty of them!
It all started when the finance company was trying to launch some innovative card products for its customers in the market but faced a lot of resistance from the third-party card processor as the new product features were not readily available in their processing system. Development of the new features would be an expensive proposition and also time intensive. The benefits they were getting from the processor while growing from a small-sized business to a mid-sized business had started to fade away. A decision was made to migrate to a modern system which could be hosted inhouse giving them complete control over the system.
After a detailed evaluation of multiple systems, this finance company decided to migrate to a new system with significant presence in their geography and also with a good support structure. The plan was to complete the migration within a time frame of 4 months as the product vendor had performed multiple migrations from the same processing system to their licensing system during their previous projects. They had readily available migration jobs which could be reused for this project.
I believe it’s easier said than done. Each migration comes with its own set of challenges which everyone involved realized it by the end of this project.
The team faced its first challenge during the data analysis stage. The business team did an analysis of the historic data on the screens to identify all the information that would be required from the old system to have a smooth transition to the new system without affecting any existing functionalities. Little did they know that the data in the screen didn’t match some of the data in the backend files. At that point, they didn’t have an expert in the team who understood how the old system was designed in the backend. The processor also sensed that the partnership with this finance company was coming to an end, so they might not have reviewed the requests thoroughly and shared the data without much further analysis. The quality of data was poor and as a result the first round of data migration was not at all satisfactory, leading to loss of time and energy for the whole team.
The data mapping and data transformation stage brought the next set of challenges for the team. The issue was identified for the Non-Performing Accounts NPA’s which were maintained in a different inhouse system to avoid paying the processing fees for NPAs. During the data mapping stage, it was observed that some of the mandatory data in the new system was not available in the old system for those NPA accounts. This included customer’s statement & transaction history. A lot of processing in the new system was dependent on this data. The only solution available to the team was to rebuild this history data with some assumptions. The data was rebuilt, but the efforts and time consumed for this exercise was huge. It also introduced a lot of additional issues which had to be handled during and after the migration.
The next challenge was completely unrelated to data but required a significant correction of data before migration. As per the original plan, the migration was scheduled before the start of the holiday season on a statement cycle so that the transactions posted in the old system could be cycled in the old system and there was no impact on interest processing. To achieve this, the entire portfolio was updated to have a common statement cycle on which the entire cut over could be done. The planning was good and all testing was performed based on the above understanding. Suddenly three weeks before the planned migration day, the processor informed that the migration could not be continued on the planned date as it coincided with a freeze period. After multiple rounds of discussions, the business agreed to migrate the portfolio 2 weeks after the planned migration date when the freeze period was over. They couldn’t wait till the next cycle date as that would have coincided with another freeze period and festival season. Thus, they ended up selecting a new migration date that was a non-cycle date. The industry accepted strategy for migrating on a non-cycle date was different from the strategy for the ones that migrate on a cycle date. Additional steps had to be introduced in the migration strategy for non-cycled accounts (which was now effectively the entire portfolio!). The new product vendor agreed to avoid these steps and make data corrections in their system during the migration to handle this change. A completely unrelated event had led to significant changes to the data which later caused other issues during the migration.
The last challenge I would like to mention here was related to the data reconciliation stage. One critical aspect of reconciliation was identified as Reconciliation of Balances. The balance reconciliation would ensure that there was no leakage of balances during the migration. According to the reconciliation strategy, it was decided to perform the reconciliation at each Product Level. The reconciliation went fine for couple of products but started giving major differences from the third product onwards. What had gone wrong all of a sudden from the third product? After some analysis it was identified that the products after the third product supported cards with split credit limit. It was also identified that some 10 years back these split limit cards were mysteriously changed to combined credit limit and then changed back to split limit cards. No one had an idea about why and how this exercise was performed as the old processing system did not support such a change. Without wasting further time, a decision was taken to correct such cards in the old system before migration. As part of balance reconciliation additional information was extracted to identify each account with the above issue and provided to the business for manual corrective action.
This brings us back to the question whether the biggest challenge of any data migration project is the data itself? Having this recent migration project experience fresh in my mind; I would like to conclude that data need not always be the biggest challenge of a migration project. A proper planning for the project, detailed analysis of the data and understanding of the source and destination systems could bring down the data issues by significant level. Correctness of the data is relative to the system that processes it. While migrating it proper care should be taken to make it relevant for the new system.
In summary all these elements matter a lot, meticulous planning, detailed design, deep understanding of source and destination system architecture, experience & expertise to tackle data exception during migration cut-over all go hand in hand and a long way. Reading about migration challenges and executing a migration is altogether a different experience. Getting the people with right skillset and expertise right from inception will help execute such complex and important projects for any financial banking institution.
In case you are thinking about embarking on one such excursion, Verinite is happy to help you succeed on your journey. For more information on our offered services in this space you can contact us by sending an e-mail to “[email protected]”
So here continues few more tips that will help in managing projects:
6) Follow the process and keep an eye on improvement: Follow the sequential set of activities that will help you bring out the specific output of the project. Skipping steps in order to meet short project timelines may backfire and end up taking more time and rework. Follow the set process to avoid such pitfalls.
An inherent characteristic of processes is that it loses efficiency over a period of time, some processes become obsolete as well since at some point it worked but now it doesn’t. If you see something that fits this description, question it and either a) remove it to save everyone time or b) improve it so it fills the actual needs.
7) Accept Delays Gracefully, Manage Risks: Delays and bad news are part and parcel of project management, what matters the most is to know early. The team should never be afraid to give bad news and project manager shouldn’t panic because the timeline could change. Establish the critical path and assess how much delay it will cause to the final timeline, look for work arounds to address the delays as much as possible. Inform the stakeholders about the same and justify with genuine reasoning.
Every change or delay invariably brings with it inherent risk and challenges associated. Analyze it, Understand the impact and take necessary actions to mitigate it.
8) Ensure that testing is done properly: Many a times there comes a situation that project started late, there was some change, additional scope was added etc. and then project Build is delayed or activities start to run in parallel when it is not intended. Once you get in to this phase its difficult to ensure that the testing has been done enough and doesn’t need to be repeated. Another round of testing means additional cost, impact on timeline and threat to final Go-Live date. No matter how hard pressed you or your team is; please ensure that proper testing is done on pieces of SW delivered at various points in time. Successful testing is key to a successful deployment.
9) Create a positive vibe, empower people, led by example: If you foster negative attitudes, you will end up bringing your team down with you. As challenging as some projects can be, always look for the clichéd bright side. Defuse tense environments with humor. Focus on what’s being accomplished and the success rather than the hiccups. Make the people in the team feel responsible. Empower them with authority on the tasks allocated to them. Whenever possible, lift as much work as you can off them so their path is cleared for achieving the project goals.
One position is not greater than the other, they’re just different roles. All roles are equally important as far as the project is concerned since without any one of them the final objective might be jeopardized. A feeling of subordination might break the unity of the team.
10) Project Closure and evaluation: Once the project is deployed successfully often the project closure reports or gathering lessons learnt are side tracked. This may happen due to number of reasons. But important thing is to take time out and perform the project closure activities and get some useful insights and lessons learnt for future references. You may think some things to be trivial that you solved during project but could prove useful to someone else. You may have solved some critical problems or challenges with your project management skills; make sure this experience is handed over to next project or project manager.
As they say Project management is sometimes a lonely job but a very important one. Project manager is the Leader who Navigates the project & is responsible for success of the project. Do it right and victory is yours!
In present world, filled with certified and degree-wielding project managers it is expected that they know the rules of successful project management. It is expected they know prevalent methodologies like Waterfall, Agile, RUP and all the project management tools like Gantt charts, MPPs, Vision, Project requirements, specifications, and almost everything! All those wonderful bits and pieces that seem to be the second nature to the project management world. Realistically a project manager’s life circles around endless status meetings, updating tickets, filling out paperwork, answering nonstop emails, ensuring sign-off, managing timelines and communicating with stakeholders almost every day. If he/she fails to do this, the fragile framework tracking the project timeline would come crashing down.
A Project manager tends to feel like a juggler in a circus. He/She has to manage multiple projects simultaneously with clients spread across the country across different geographies. Co-coordinating with multiple teams, multiple vendors, multiple team members like architects, designers, developers, DBAs, SMEs, testers, Quality Assurance, change management, release management and taking care of stakeholders at the same time could be strenuous.
If you have been there, done that, experienced it or would like to be a part of it someday then here are some tips. This will better equip you handling projects and juggling multiple responsibilities successfully.
1) Know Your Subject: It is difficult to effectively manage anything without truly understanding how it works and the complexities involved. It is not necessary to be an expert in technology to understand how code is written, standards of programming, how servers work, how various frameworks work but you need to know basics and principles behind it. Similarly, it is very important to know the concepts of project management.
One can search the Internet for information about project management, get familiar with the various methodologies and tools, read a few books. Even consider getting that very popular project management certification.
2) Understand client’s expectations, requirements and deliverables follow: It is important to understand the expectations of the client before beginning the project. For this, ask clear questions during initial interactions with the client. This will help the client identify and express their specific requirements and help you understand the objectives.
All doubts and queries about the project needs be clarified in the initial stages before starting the project to avoid confusion in the later stages. This will also help you while drafting the project scope document. Understanding the expectations of clients will enable you to clearly define the deliverables of the project and take their approval, which you can communicate to your internal development team. This will ensure the deliverables will meet their expectations.
It is necessary to know the people responsible for signing-off on the various deliverables. Getting to know them in the early stages and involving them in discussions to take their inputs ensures there are no last-minute surprises or new demands (after the submission of deliverables).
3) Communicate with clients on regular basis: According to a study by the Project Management Institute (PMI), “ineffective communication leads to fewer successful projects; significantly fewer projects meet original goals, finish on time, and within budget.” With so much dependent on proper communication, a project manager has to ensure communication is constant with the client.
Learn how your clients like to communicate. Mode of communication, frequency of communication are important attributes to keep in mind to avoid obstacles in the flow of information. Always share project updates with all relevant people on a regular basis through status reports.
4) Define roles clearly, pitch in whenever required: The role of each member in the team should be defined clearly. This will help avoid confusion and overlap of responsibilities. When the team members are clear about their roles, their energy and focus will be directed toward reaching their individual targets. To do this, provide team members the right information at the right time.
Defining clear roles doesn’t always guarantee smooth flow of work. Often you encounter situations where there is a delay in task execution or question mark on quality of deliverables. A project manager cannot draw a hard line between team roles. In difficult times he has to help team members and sometimes take up tasks which is not meant to be done by him. The important part is to contribute as a member of the team, not as somebody who is overseeing an operation.
5) Update changes in the scope document: The scope document, as you know, defines what the project is supposed to achieve and what it cannot accomplish. The scope document is locked-in (baselined) before the project begins. However, as seen in major projects there are additions or changes to the scope document during the course of the project. For instance, the client may ask for additional deliverables which may alter the budget and deliverables initially agreed upon. This will change the estimates of cost, effort, and duration which will become the approved target.
For this, changes have to be made in the scope document and approved by the management and stakeholders. It is a difficult tradeoff since clients would want to get more with same cost and within same timeline, but it will be foolish on the project manager not to get the changes approved as it will put pressure on the entire project team and resources which might be catastrophic.
Stay Tuned for more Tips!!!
The number of service providers for banks is increasing massively and banks now have the choice to pick considering numerous factors. With growing need, expectations are also growing and banks are really making informed decisions when it comes to picking the right service providers or vendors.
Let’s take a step further and see what their expectations are in 2018 and what can really help them surpass their competitors:
- Advisory Consulting services: IT giants usually have a fixed set of services, which are provided to banks. Additions such as advisory consultancy services may not be a part of that package and that makes things harder because banks may not be sound in the technologies or processes that are being employed. Banks would, hence, really like to be guided through the process and any service provider assuring that would get an automatic edge.
- That one extra value-added offering: Is it only the said service that banks are looking for? No. They would definitely like some value adds from their service providers. This could be as simple as providing suggestions wherever necessary to optimize the entire process or helping out with test case repositories or some sort of automation, or guidance on certification with schemes etc. The vendor that can help them train in-house or create process documentation for banks end users will be added advantage. So, they would prefer a vendor meeting these criteria after checking their past success record.
- Flexibility: As aforementioned, IT giants in the name of ‘standards’ have some fixed set of processes they follow, services they provide, and the amount they charge. Not all banks may be able to afford this kind of rigidity or being dictated or intimidated by them. That’s why if there are service providers that are flexible with their approach and are willing to go extra mile for the client are always preferred.
- Technological help: Getting services from one vendor and hiring another vendor to help with them technological products or advancements can prove to be hassle-ridden. If one vendor can do it for them, a lot of time will be saved. Also, banks will be comfortable working with their existing vendors because of the trust factor. This convenience will lead to confidence, which will ultimately increase the probability of a successful outcome.
- Adaptability to new banking trends: Recently banking industry is facing challenges and some competition from non-banking institutions in the same space. With open banking or Payment Services Directive (PSD2) coming into the picture, banks have to take additional steps to stay ahead of the new players in the market. Open banking is pretty much what the name describes: Banks are now obligated to be transparent. This massive change, which came into effect in January this year, has made banks more proactive and attentive. They have to reassert their existence as well as their position in the market. Thus, if vendors provide banking API support they will choose them over the others. Here, we must keep in mind that open banking may not be all bad for banks. While technical challenges will arise, due to which the pressure on vendors will go up, the number of strategic opportunities such as partnerships and collaborations are abundant.
We have indeed seen a great shift in momentum in the banking industry due to which we can expect the mode of operation of banks to change as well. During this shift, service providers really need to adapt and ensure they serve banks to their potential because banking, regardless of the number of new players, will remain the core of the financial ecosystem.
Having our presence in this industry and getting the footprint stronger we offer these as a part of our services to our clients. It’s always great to build up a friendly and healthy relationship that delivers successful projects. Verinite is committed to achieving its client’s requirements but also helps identify what they ‘need’ more than the ‘wants’ or ‘wish lists’. We believe we can address this area on our project engagements and thus delivering the best results!
“Kites rise highest against the wind, not with it” — Winston Churchill
To make the most of your career, weave the above words into your everyday life. When you stick to safe and comfortable in your life, your potential stays unfulfilled and you risk stagnation. Anything that is safe and comfortable never pressurizes you to rise to the occasion and deliver your best. Without the resistance, you won’t be able to discover all the new possibilities in a situation.
This is especially true in the world of software development and testing. With the IT sector expanding at such a drastic rate, the technology used is becoming a vital part of our lives and a significant driving force behind the global economy.
Technical knowledge is easy to acquire through academical learning but the proficiencies that are required for establishing your expertise are achieved over time when you push yourself and rise to challenges. The tools that are basic requirements to help you deliver your best are a problem-solving mindset and a disciplinary approach to resolve a problem.
When it comes to a tough challenge, here are a few pointers to help you rise to the occasion at hand.
- Keep “For Action” mindset: Exploring the advantages and the disadvantages of any new challenge before delving into it is a must. Go through the necessary material and consult the best people on the subject to get some advice from them. It is easier to perform a task once you are aware of what it is that you’re dealing with. This is simplified if your bias toward any challenge shifts to “YES”. Not only will you be open to learning new things in a shorter span of time, but you will also grow and develop more as an individual. Also, your career skills will increase by a lot, thanks to your open attitude toward challenges. Saying yes to new challenges makes you visible in a crowd of people who prefer to stay in their comfort zone. It also allows you to interact with more people, giving you a much greater opportunity to learn.
- Ingredients to Success: There will always be a certain number of things that need to be present, or satisfied, to improve your chances of success. Learn to identify those and ensure that you are in control of those aspects. Having knowledge of these aspects make it easier for you to successfully complete the challenge along with best interests of your organization. Before you start working on a new challenge, remember to have a clear perspective about what is considered as success in the challenge and don’t be afraid to ask for the resources you think you will need to achieve success.
- Don’t overwork: You have already taken the challenge, and no one will be expecting you to complete it in a day. So, don’t try to overwork yourself for the challenge. Proper management of time and resources is crucial to succeed in any challenge. But work wiser and smart.
- Calm and Focused: When you’re facing a challenge, your mind is the most important resource you have. Then it goes without saying that keeping calm and focused is one of the first and most important steps in dealing with a challenge. When your brain is in control you can observe the things around you better, and respond to them appropriately. Staying calm also allows you to be more creative and channel your energies sin the right direction.
- Work Professionally:When everything around you is new, it’s easy to fall prey to what is being said about you. However, it is best for you to ignore such kinds of situations and find peace in yourself and your work. It will also allow you to be more focused and complete your work more efficiently. In workplace situations, all that ultimately matters is the end goal, so as long as you don’t let things affect you and keep working, your results will show.
- You learn faster under pressure: Remember the times during your college days when preparing for exam on the last night? The concentration is immense and suddenly things begin to fall in right place, it makes sense to you and you have higher focus on the things that are important. Always prioritization and focus will help you achieve the goal.
Now that you know the best way to face new challenges, look for any new opportunity that is floated around in your organization. Better yet, accept the challenges that have already made their way to you. Taking up new challenges and delivering your best is the ultimate way to keep learning, keep growing, and keep moving forward. It also boosts your confidence to a great extent.
The service industry has been growing leaps and bounds in the last decade and is continuing to do so, steadily, the product industry too has been trying to steadily catch up. We have more and more business organizations coming up offering better products and better services and catering to larger customer base.
The product industry and the service (in this case the software) industry have a lot of similarities though their differences are more apparent. Many manufacturers now offer their own service operations and both require skilled people to create a profitable business.
Let’s take a closer look at how these industries function and what makes them similar or different.
The product and the service development cycle:
The first step: (Ideation)
Product: Generating a fresh idea for the product forms the first step. Customer surveys and feedback on existing products and then examining the industry to mark areas where useful products do not exist, helps generating these ideas. Pros and cons of each idea are discussed with the management team to narrow down on the best product ideas among the rest. The list is then whittled after discussions with employees and partners and considering customer’s opinions.
Service: Determining requirements forms the first step. Meetings with managers, stakeholders, and users are held in to determine basic requirements like who is the user of the system? How is the user going to operate the system? What data inputs and data outputs are required? Etc. After the necessary requirements are gathered, they are analysed for their validity. Then the possibility of incorporating these requirements in the system to be developed is studied.
At the planning stage, motivation, self-assessment and Adjustment to changing thought process is extra necessary. Deadlines need to be taken into account at each stage and suitable management of the same should be done.
The second step (Analysis):
Product: The product idea that is finally narrowed down is analysed from a business perspective. If competition for that product exists, it is calculated. Then determination of the demand for the product is made and costs affiliated to the product including developmental and operational costs are estimated.
Service: Along with the assistance of client focus groups, the project team gauges the end user requirements. Often this is done with the assistance of client focus groups, which provide an idea of the expectations for the finished product and how it will perform. The project team then documents all of the user requirements and gets a clearance from the client and management to move forward with system design.
The third step: (design)
Product: a prototype of the product is made initially after a thorough brainstorming. This prototype is then sent across to key partners and a few loyal customers, asking for their opinion and feedback. Based on the feedback given, the product may or not be evaluated and modified.
Software design specifies hardware and system requirements and also helps define the basic layout. These design specifications serve as an input to for the next phase of the model.
The fourth step (Build)
The build stage follows the prototyping stage. The engineering team focuses on developing the modules, sub-modules, functions, sub-functions to come up with a first version of the product that is in line with the prototype.
Developers follow the coding guidelines set by their organization, they use programming tools like compilers, interpreters, debuggers etc. are used to generate the code. The programming language is chosen with respect to the type of software being developed and then the software product is actually built.
All effort should be made to keep the product/ service as close to what the customer demands as possible because eventually the responsibility of the builder, isn’t just limited to warranty.
At every step of the way, whether developing a product or a service, it is essential to employ real- time driven resource management for the project. This process is dynamic and should be continued consistently to avoid schedule delays and cost overruns.