In continuation with one of my previous posts , I am sharing some of my experiences with Globalization testing.
Software Globalization (abbreviated as G11n), simply put is a combination of 2 activities- Internationalization and Localization
Internationalization is planning and implementing products and services so that they can easily be localized for specific languages and cultures. So for any product to be successfully localized in to different languages, it is imperative that it has to be internationalized properly. Internationalization is abbreviated as I18n.
Localization is the aspect of development and testing relating to the translation of software and its presentation to the end user. Localization is abbreviated as L10n.
The extent and the quality of Internationalization and Localization are tested by means of Globalization (G11n) testing. G11n testing is a specialized field in itself and it requires specific skills and expertise to be successful at this form of testing. Because of surface level similarity this testing has with Functional testing, it is often wrongly compared leading to false conclusions. In this write-up, I intend to uncover some of the common myths around the G11n testing.
Myth 1: G11n testing is not technical enough
This is a myth because it is more based on Ignorance rather than a thoughtful observation. As G11n testing is a separate form of testing altogether, there is a lot of scope of learning new and specific technologies. Organizations which are serious about developing and selling the products for an International market usually have a separate group which takes care of the Globalization aspects of the product. A person testing internationalized version of the Software needs to be detailed in his knowledge in the following areas-
- An I18N tester needs to have to have an in-depth knowledge of features and functionalities of the application under test. He/she needs to understand the product under test as well as Functional tester does.
- An I18N tester needs to be well versed with the technical concepts of Internationalization such as Single base binary, Unicode, Multi-lingual support, different encoding systems, code-pages, Locale awareness etc. Without the knowledge of these concepts, a tester will never be able to find good I18N bugs.
- Troubleshooting the I18N related issues requires additional skills and knowledge of the Internationalization concepts.
- An I18N tester needs to have a fair bit of knowledge about the Technical architecture of the product under test. Without this knowledge, he/she will not be able to accurately assess the impact any change in software product would have from the Internationalization perspective.
- An I18N tester is usually an interface to the Translation experts. It is very likely that the Translation experts because of the nature of their job may not be technically proficient. Usually, I18N test teams help the Translation experts in their unit testing involving complex product setups.
Apart from the purely technical knowledge needed by an I18N tester, it is imperative for a person to have knowledge about the cultural considerations for different countries including (but not limited to)- understanding the date/time formats, number formats, preferred printing options, currencies etc. This makes the job of an I18N tester even more interesting and challenging.
Myth 2: G11n testing is majorly about testing the UI
This is a myth because- to the majority of people, the translation of UI text and labels etc. are the only changes that get done when a product is internationalized.
There is no doubt to the fact that whenever an application gets localized (aka translated) to any other languages, the UI changes are the most prominent in the product. All the text/labels that were originally displayed in the base language (which are usually English), now gets shown in the translated language. Though UI changes are prominent but technically these are only one of the changes that get done as a part of Globalizing the Software. A few examples of activities involved in internationalizing the software include-
- Externalizing the UI.
- Design UI so that strings can expand.
- Ensuring that the product can support international character sets.
- Eliminating strings corruption.
- Ensuring that Regional settings are read from OS.
- Support for bidirectional text.
And many more..
A typical G11n testing cycle might have more UI issues logged than the actual functional issues depending upon the quality of Globalization but that doesn’t mean that majority of testing efforts were spent to find UI issues as there is a considerable testing effort spent to verify the other aspects of Internationalization as well.
Myth 3: G11n Testing can start only after the base product is translated
This is one of the common misconceptions for the people completely unaware of Software Globalization concepts. To give a brief background, Globalization of software happens appropriately in different phases when the base product (The English language product) is being developed. In an Ideal development scenario, the software development consideration from Internationalization perspective begins alongside with base product development. So, when the feature complete build gets released for testing, it has all the Internationalization features built-in but point to note here is that the Localization of software hasn’t happened yet. Localization of the software product usually happens after the User Interface for the base product is frozen. Before UI Freeze, the Globalization test team's focus is testing the Internationalization aspect of the product. So, the testing for Globalization starts much before the translation of the Software happens.
Watch out for more such myths being uncovered in this space!
Saturday, June 14, 2008
Sunday, June 8, 2008
Designing effective Test status reports
"You are the headlights of the project"- Sounds something familiar? Yes, it is the very first lesson of the landmark Software Testing book- "Lessons learnt in Software Testing". The intent of this lesson is simple- Testing is done to find information . This is infact the essence of testing group. The project stakeholders always rely on the results of the testing effort to get a feel of how the project is coming along. And it is this information that helps executives take important decisions about releasing and positioning the product under test in the market. That is probably one of the important reasons why organizations invest in an independent testing group.
There is no doubt that finding information is one of the premier role of the Testing group in an organization but what is equally important is how effectively the data is actually presented. I was reading an article on Metrics by Mr. L.R.V Ramanna and the below mentioned quote took my attention-
Test metrics are like corporate balance sheets. What they show is important but what they hide is vital.
So Test status, depending upon the way it’s presented- can help take Critical decisions with ease or rather than answering questions, raise more questions. Consider, a project is 2 day away from the release and the testing status report tells only about the Open defects and the defect trends and misses vital information such as Test coverage etc. may cause decisions to be made without enough considerations to the facts unless somebody raises the hidden questions
Here i am sharing some tips based on my experience on how the status reports can be made more effective.
Identify the audience of status report:
Knowing the audience who will read the report is an important exercise. More often, i have seen that the Test status reports are sent to distribution lists containing large number of people without a thought to who the recipients are. Having knowledge does not mean to know the people personally, which may not always be possible if you are working in an organization distributed world wide but it means to understand their role in the project. This always helps to anticipate the expectations people have from the status report.
It is always preferable to list the role of people receiving the status report which may be as diverse as the following-
- Product Manager
- Project Manager
- Development Manager
- Development Director
- Test Team
- Development team
- Vice President etc.
If the status reports are sent without knowing who is receiving it, then it often leads to majority of recipients ignoring the report upon receiving it and you take the blame of unnecessarily spamming their inboxes.
Anticipate what questions the recipients would have:
This step is the basis of Test status reports.
The key purpose of Test project status reports is that it answers the wide variety of status related questions that the project stakeholders might have. An effective status report would always answer the diverse questions clearly. Depending upon the phase the project is, the stakeholders have different status requirements. In an example below, i have listed a few questions that the different project stakeholders might have at Test execution phase.
Phase: Test Execution
Development Manager:
- What is the incoming bug rate ? Is the bug find rate declining or increasing?
- Which components have the most high severity bugs?
- Which components have the bugs that are blocking the testing?
etc.
Project Manager:
- Is the testing currently happening on track?
- Are there any major blockers to testing?
- Is there any risk to meeting test deliverables and milestones?
- If there are risks, what are the plans to mitigate the same?
etc.
Product Manager:
- Are there any defects that need Product Management attention?
- What is the list of deferred defects?
etc.
Test Manager:
- What is the percentage of test coverage completed?
- What is the percentage of defects found based on Test methodology- Scripted testing vs. Exploratory testing?
- What is the reason if the defects are trending up?
- What is the defect fix rate?
etc.
The above list is definitely not an exhaustive list but gives some pointers to varying status needs. There may also be overlap of questions that different stake holders needs answer for. These questions vary as the context and the project phase changes but it makes report more effective if these are anticipated in advance.
It saves a lot of time in the meetings when the status report answers the questions appropriately.
Answer each question in an objective manner:
Once the list of questions are thought of and finalized, the next step is to design the metrics. Pick up each question, assess each with the help of the data. It is important that answer to each question should be as fact based as possible. Refrain from adding emotional responses to the questions.
Position most important information at the beginning of the report:
The positioning of data in the Test status report is quite vital. Sometimes, i have observed the status report begin with all the positive information and has the risks listed somewhere at the bottom of the report. Such an organization of the report may cause oversight of important information. Effective status report should alert the recipients with any important information within one glance.
It’s always good to Summarize the status at the beginning of the report before getting into the details section.
Don’t get intimidated with mentioning the Risk items-
Test teams should not fear giving bad news. If the project is in the bad state, it should stated in a way that everyone understands the risks and the situation properly. Use forceful language if needed to pass the message properly but the language used should be professional.
Any more thoughts ?
There is no doubt that finding information is one of the premier role of the Testing group in an organization but what is equally important is how effectively the data is actually presented. I was reading an article on Metrics by Mr. L.R.V Ramanna and the below mentioned quote took my attention-
Test metrics are like corporate balance sheets. What they show is important but what they hide is vital.
So Test status, depending upon the way it’s presented- can help take Critical decisions with ease or rather than answering questions, raise more questions. Consider, a project is 2 day away from the release and the testing status report tells only about the Open defects and the defect trends and misses vital information such as Test coverage etc. may cause decisions to be made without enough considerations to the facts unless somebody raises the hidden questions
Here i am sharing some tips based on my experience on how the status reports can be made more effective.
Identify the audience of status report:
Knowing the audience who will read the report is an important exercise. More often, i have seen that the Test status reports are sent to distribution lists containing large number of people without a thought to who the recipients are. Having knowledge does not mean to know the people personally, which may not always be possible if you are working in an organization distributed world wide but it means to understand their role in the project. This always helps to anticipate the expectations people have from the status report.
It is always preferable to list the role of people receiving the status report which may be as diverse as the following-
- Product Manager
- Project Manager
- Development Manager
- Development Director
- Test Team
- Development team
- Vice President etc.
If the status reports are sent without knowing who is receiving it, then it often leads to majority of recipients ignoring the report upon receiving it and you take the blame of unnecessarily spamming their inboxes.
Anticipate what questions the recipients would have:
This step is the basis of Test status reports.
The key purpose of Test project status reports is that it answers the wide variety of status related questions that the project stakeholders might have. An effective status report would always answer the diverse questions clearly. Depending upon the phase the project is, the stakeholders have different status requirements. In an example below, i have listed a few questions that the different project stakeholders might have at Test execution phase.
Phase: Test Execution
Development Manager:
- What is the incoming bug rate ? Is the bug find rate declining or increasing?
- Which components have the most high severity bugs?
- Which components have the bugs that are blocking the testing?
etc.
Project Manager:
- Is the testing currently happening on track?
- Are there any major blockers to testing?
- Is there any risk to meeting test deliverables and milestones?
- If there are risks, what are the plans to mitigate the same?
etc.
Product Manager:
- Are there any defects that need Product Management attention?
- What is the list of deferred defects?
etc.
Test Manager:
- What is the percentage of test coverage completed?
- What is the percentage of defects found based on Test methodology- Scripted testing vs. Exploratory testing?
- What is the reason if the defects are trending up?
- What is the defect fix rate?
etc.
The above list is definitely not an exhaustive list but gives some pointers to varying status needs. There may also be overlap of questions that different stake holders needs answer for. These questions vary as the context and the project phase changes but it makes report more effective if these are anticipated in advance.
It saves a lot of time in the meetings when the status report answers the questions appropriately.
Answer each question in an objective manner:
Once the list of questions are thought of and finalized, the next step is to design the metrics. Pick up each question, assess each with the help of the data. It is important that answer to each question should be as fact based as possible. Refrain from adding emotional responses to the questions.
Position most important information at the beginning of the report:
The positioning of data in the Test status report is quite vital. Sometimes, i have observed the status report begin with all the positive information and has the risks listed somewhere at the bottom of the report. Such an organization of the report may cause oversight of important information. Effective status report should alert the recipients with any important information within one glance.
It’s always good to Summarize the status at the beginning of the report before getting into the details section.
Don’t get intimidated with mentioning the Risk items-
Test teams should not fear giving bad news. If the project is in the bad state, it should stated in a way that everyone understands the risks and the situation properly. Use forceful language if needed to pass the message properly but the language used should be professional.
Any more thoughts ?
Subscribe to:
Posts (Atom)
