Presented a Workshop on Software Globalization Testing

On 14th-Sept, I got to present in the STeP-IN Forum's event in Hyderabad. The event was Software Testing Conference. It was primarily a 2 day conference, in which the first day on 14th-Sept was the Pre-Conference Tutorials.

Tutorials were different than the sessions on the normal conference day as these went in-depth into the topic and were more or less like a Workshop of about half-a-day. I had a good experience overall as i presented on the topic of Demystifying Globalization Testing .

I could cover the different aspects of Globalization testing both via presentation and live demonstration, which apparently was decently received (though i am still expecting a formal feedback update from the Organizers).

The content about Globalization Testing that i have written in this blog purely from my experience was always as helpful in conducting the session of this magnitude.
The best part that i liked was that the session eventually turned out to be quite interactive and there were lots of discussions and meaningful interactions with the audience during the entire session.

Will upload the presentation soon.

************************Update on 23rd-Sept-2011************************
I have uploaded the Presentation i delivered at this session here

Presented a Corporate Techtalk on Globalization Testing

Earlier in the month of July, i was involved in presenting a TechTalk at Aditi Technologies in Bangalore. The TechTalk was on the topic- "Uncovering myths around Globalization Testing".
This talk was as a result of Invitation by one of the Testing community members Ravisuriya . Thank you Ravisuriya.

The presentation for this talk can be found here .

The overall Presentation setup was good and there were quite a few questions in the end about the topic discussed. One of the questions that caught my attention was-
"Is it possible to eliminate Globalization testing altogether?”
Although as a passionate professional, i may be biased in my views and opinions around this but this one is good to think further and investigate. I did respond with a "No" with some reasons around external dependencies and Product architecture and lot of other variable factors but this one certainly deserves a little bit more thought.
There was one more good question on what is the right ratio of Globalization testing vs English Testing.
I would write about my perspective to these questions sometime soon.

One Effective way to learn about Software Globalization Engineering

If you want to Learn about Software Globalization Engineering, one of the practical tools that i have come across is "World Ready Software Example", which can be found here .

Originally designed to support only Windows 2000 and Windows XP, this tool actually works on Windows 7 to a good extent. You can actually try out several features of a fully Globalized application.

What can you do with this tool ?
- You can check the Local formats like Date, Time, Currency, Calendar, Numbers from the loads of different locales.
- You can input any Unicode characters and see how a truly Globalized application processes the characters,\.
- You can check the Multilingual User Interface feature and change the current language to any listed language at the run-time.
- You can simulate Pseudo Translation.
- You can simulate Pseudo Mirroring. Mirroring is the process of simulating the RTL (Right-to-Left languages like Arabic etc.).
- And much more.

How does this tool look like ?

Myths about Globalization Testing

I think i havent contributed to the Myths about Globalization Testing Series in a while. I do intend to write more about this and soon. In the meantime, i thought to just go back in time a little and list all myths that i have worked to uncover in the previous blogs. Here's the list-

Uncovering Myths about Globalization testing -1
Myth 1: G11n testing is not technical enough
Myth 2: G11n testing is majorly about testing the UI
Myth 3: G11n Testing can start only after the base product is translated

Uncovering Myths about Globalization testing -2
Myth 4: A person who doesn't know French cannot test the French version of the Software
Myth 5: A tester only needs to follow the test cases executed for Base language in order to thoroughly test the internationalized applications
Myth 6: There is no scope of exploratory testing while testing internationalized applications
Myth 7: The language verification of User Interface can be done by comparing the text on screen with translation outputs of any freely available Online translator.

Uncovering Myths about Globalization testing -3
Myth 8: If a test case works fine in French language, it will work fine in German language as well
Myth 9: If the Foreign text input in application text fields work fine by using the Soft keys, then it means the data input through respective Foreign language key board would also work fine.
Myth 10: Globalization testing doesn't require the same test setup as is required to do the Base language testing. Globalization testing can be done with a minimum test setup.

Uncovering Myths about Globalization testing -4
Myth 11: Localization - means Localized product on a localized Operating System, Internationalization- means Localized product on English Operating System

Uncovering Myths about Globalization testing- Demystifying MUI Packs
Myth 12: Testing International applications using "Microsoft's MUI Pack" or "Localized OS installation" means one and the same thing

Uncovering Myths about Globalization testing- Input validation testing 2
Myth 13- A tester can perform tests specific to text inputs for Localized applications using the similar approaches as the English language testing

Uncovering Myths...."Security Testing is from Mars and Globalization Testing is from Venus"
Myth 14- Security Testing is from Mars and Globalization testing is from Venus

Uncovering Myths about Globalization testing- English version on Localized setup
Myth 15: There is no use testing the English version of a product on Localized Operating systems

Uncovering Myths about Globalization testing- MUI Packs in Win XP and Win Vista
Myth 16: There is no difference between MUI technology being used in Win XP and Win Vista

Uncovering myths about Globalization Testing- Reusing Test Automation
Myth 17: The test scripts meant for English language automated tests cannot be reused for Internationalization testing


Uncovering myths about Globalization testing- Context driven planning
Myth 18: There is one standard way of Test Planning the Globalization testing, which is applicable to all the contexts.

Uncovering Myths about Globalization Testing- Websites with localized addresses
Myth 19: There is no need to include localized web addresses as a part of your test data as Web addresses are always in English

Uncovering Myths about Globalization Testing- Knowledge of Native Language
Myth 20: If i dont know German at all, i can still effectively test a German application

Uncovering myths about Globalization Testing- Finding Localization bugs before actual translation takes place
Myth 21: It is not possible to find the Localization bugs before actual translation takes place

Uncovering myths about Globalization Testing- Testing MUI feature
Myth 22 : A tester testing Globalization features need not pay any special attention towards testing MUI feature.

Uncovering myths about Globalization testing- Approach to generate Localized test data
Myth 23: It is possibly the right strategy to randomly pick the test data specific to the localized language you are testing


Hope you enjoyed going over these again. Would appreciate your comments, suggestions!

Presented at SoftTec 2010 Testing conference

I got to present at the SoftTec 2010 Testing conference this weekend (on 3rd-July-2010). The conference details are available here .

It was after quite a while that i presented on a public forum and realized that i should be doing it more often.
This conference was different from the other Software Testing conferences (QAI, STeP-In Forum) that i attended in the past. Some of the key things were-
- Test Professional track and Test Leadership tracks were different and the presentations ran parallely.
- The conference was attended by around 700 Software Testing professionals right from Interns to CEOs, the audience consisted of one and all. Infact, this is the largest gathering of Software Test Enthusiasts that i have seen so far.

Coming to the topic i presented, it had to be something i eat, sleep and live (as i had only 2 days to prepare) and it was- Globalization Testing- Getting your Software World Ready .
I have posted the presentation here

I think I was able to complete the presentation just-in-time in the allocated time and credit to audience for creating meaningful conversations by asking some challenging questions.

Some Questions that got asked in Q/A session:
- How many languages have you been involved in testing as a part of Globalization testing ?
- I have heard that Sanskrit is the best language for Software Globalization. Are you aware of any Indian organization who are involved in researching about Sanskrit Localization ?
- What considerations we need to be putting towards Database side when we are talking about multilingual applications ?
- I am working to test application mostly in around 10 Indian languages. I am under pressure to devise Test strategy in such a way that i dont spend too much testing effort. How do i do it ?
- Can you tell something about Automation Strategy to be used for multilingual applications ?
- If i am involved in testing multiple languages, Do i test everything (as i do for Enhlish language) in all the languages ?
- If i am testing say login of the application say Japanese application, one Japanese character may occupy more space than English character- how do i plan for such a test ?
- I am involved in testing an application and sometimes i am required to copy labels so that i can check the meaning using some Online translator. Can this be done ?

I think this covers all the questions that i could recollect. I will be taking answers to some of these questions (some are already explictely covered in my previous posts) in the upcoming blog posts.

Overall, it was a hugely satisfying experience.

Update on 16th-Oct-2010:
TheSmartTechie magazine in their Aug 2010 edition ran an article on this conference. It had some comments for my presentation as well. Read below-

Uncovering myths about Globalization testing- Approach to generate Localized test data- II

This post is in continuation of one of my previous posts on the subject of Localized test data generation. To recap the suggested approach, here it is-
- Come up with the appropriate classes of the data as per the structure of the language.
- Once the classes are identified, enture to pick up test data in a way that each class has a representation in the text data.

In the previous post, i gave an example about classes specific to Spanish language. I though to extend this further to include other European languages German and French as below.

French Language Input characters classes:
Capitals:
À,Â,Ä,È,É,Ê,Ë,Î,Ï,Ô,Œ,Ù,Û,Ü,Ÿ (commas are only used as separators)

Lower case:
à,â,ä,è,é,ê,ë,î,ï,ô,œ,ù,û,ü,ÿ

Punctuation
Ç,ç,«,»,€

Special French representations:
HTML entity codes (HTML entity codes are the codes which allow browsers and screen readers to process data as the appropriate language) e.g. for the character á, the HTML entity code is á
HTML entity codes for French language are listed here .

English Lower case characters:
a-z

English Upper case characters:
A-Z

Numeric representations:
1,2,3,4....

Special characters (EN):
~`!@#$%^&*()_+-={}[]|\:;"'<,>.?/

Any Known problematic French characters (not included above):

Some Real French Test data:
The purpose of this class is to simulate some of the real time test data as might be used by customers for some fields. Microsoft Terminology Translations offer a good source to simulate real strings that are used in Microsoft Terminologies.

**********************************************************************************************************

German Language Input characters classes:
Capitals:
Ä,Ö,Ü,ß,€ (commas are only used as separators)

Lower case:
ä,ö,ü

Punctuation
«,»,‹,›,„‚“‘”’–,—

Special German representations:
HTML entity codes (HTML entity codes are the codes which allow browsers and screen readers to process data as the appropriate language) e.g. for the character á, the HTML entity code is á
HTML entity codes for German language are listed here .

English Lower case characters:
a-z

English Upper case characters:
A-Z

Numeric representations:
1,2,3,4....

Special characters (EN):
~`!@#$%^&*()_+-={}[]|\:;"'<,>.?/

Any Known problematic German characters (not included above):

Some Real German Test data:
The purpose of this class is to simulate some of the real time test data as might be used by customers for some fields. Microsoft Terminology Translations offer a good source to simulate real strings that are used in Microsoft Terminologies.

Watch out this space further for Test data classes for Japanese, Chinese, Korean and other complex languages.

Published- An article in Testing Experience Magazine

Uncovering myths about Globalization testing- Approach to generate Localized test data

Myth 23: It is possibly the right strategy to randomly pick the test data specific to the localized language you are testing

While performing Internationalization testing, a tester largely deals with testing the product on the languages not known to him. Imagine a tester from India testing a product on Spanish Operating system. While it is largely possible to do the Internationalization testing without knowing the language
but there are certain traps related to this ignorance of the underlying language that a tester should be aware of. One of such trap is usage of test data.

Very often the lack of knowledge of language prompts the tester to just use a "few" input characters of the language under test while ignoring the rest. This is obviously a dangerous situation as it leads to some holes in the test coverage. The below example illustrates what i was mentioning here-

E.g. considering that a tester is testing Spanish langage, he is most likely to include Spanish specific input characters such as- á,é,í,ó,ú,ñ,ü and may be more investigative ones will use there above list in CAPS, or in combination with English characters, numbers etc. But the basic question that most of testers resist to ask is whether my test data is covering all the possible Input characters for the language under test ?
May be one reason this question is not asked is because of lack of complete knowledge of the language under test. Some may argue the validity of this question with an argument- Is it really necessary to test all the possible Input characters that a langauge offers ? This is a fair questionconsidering say while testing English language- if a tester uses "a" as a test data or if he uses "z" as a test data, wouldnt the result be the same all the time ?
May be Yes.
But when we are considering the test data generation of Localized characters, there are some more crucial points to be considered. This is what i intend to cover in the upcoming sections with a idea on how the test data of "foreign" languages can be better classified and used while testing.


How do i know which Input characters should be tested for a particular language ?
I have generally found this website quite good and informative. This has separate web pages for different languages and the tester can easily see which Input characters define each language. More on this in upcoming examples.

Is there a reasonable way to ensure that the localized test data gets appropriately covered ?
This website would not only allow you to know the characters used in a particular language but also the appropriate ways of classifying the same.

In my experience, dividing the Input characters of a particular language in to appropriate classes is always a better approach. Let me try and explain using an example.

Consider Spanish language for example-

Webpage listing all the chatacter inputs:
http://tlt.its.psu.edu/suggestions/international/bylanguage/spanish.htm

E.g. the below section represent Input character classes for Spanish language characters in a boarder sense. And while coming up with the test data for a particular field say- Password, depending on the rules and the length accepted one or more characters from each of these classes can be used as a test data.

Spanish Language Input characters classes:
Capitals:
Á, É, Í, Ó,Ú,Ñ,Ü (commas are only used as separators)

Lower case:
á,é,í,ó,ú,ñ,ü

Punctuation
¿,¡,º,ª,«,»,€:

Special Spanish representations:
HTML entity codes (HTML entity codes are the codes which allow browsers and screen readers to process data as the appropriate language) e.g. for the character á, the HTML entity code is á
The reason to include these codes as a separate class is because if these codes are used separately as the Test data in an input fields, the application might interpret this as a single letter. This possiblity is espacially true for web based applications.

English Lower case characters:
a-z

English Upper case characters:
A-Z

Numeric representations:
1,2,3,4....

Special characters (EN):
~`!@#$%^&*()_+-={}[]|\:;"'<,>.?/

Any Known problematic Spanish characters (not included above):

The reason to include English Input characters in Spanish test data is that users generally use English along side with Spanish characters with some commonality being there in characters usage.
Further in our example- say is password is accepted to be 20 or less character length, some of the test data can be (notice that test data has one or more characters from each identified class).

Less than 20 characters
Áñº«gz24@

=20 characters
Üó€»sH224&ÜÍ¿¡

Greater than 20 characters
Ó᪫»tyKL7845`!@º

Far Greater than 20 characters
(If there are no limits on the UI)

0 characters
(blank)

Important point to note that each test data above includes atleast one character from the Input data classes identified for Spanish language above. That is the key as this approach helps ensure that all the classes of data is utilized appropriately and no Input character is actually left to chance.

Some questions related to this that i would be working to address in upcoming posts-
- The strategy to divide the Input characters into classes seems good for the languages which has limited set of characters like European lanaguages. Can this approach be used for Asian languages like which Japanese, Chinese, Korean etc. which deals with multiple writing scripts as well as thousands of characters ?
- In larger context, what could be the test straregy totest Unicode feature ? Unicode being one of the key Globalization features that is built in the product and should be tested comprehensively from the point of view of Globalization testing.

For a complete list of Globalization testing myths uncovered in the past, please visit here .

Uncovering myths about Globalization Testing- Testing MUI feature

Myth 22 : A tester testing Globalization features need not pay any special attention towards testing MUI feature.

I had touched up the nuances of Multilingual user interface (MUI) feature in this post as well as this one in the past. These posts primarily focused on how MUI technology is implemented in Windows Operating systems and spoke about the differences in MUI technology in different versions of Windows OS. What these posts did not cover specifically about MUI was- if the software application you are testing supports Multilingual user Interface, then what are the aspects that you can look at from testing perspective to ensure that MUI is tested properly. In my experience, this is an often neglected area of Globalization testing. May be because supporting MUI is one of the basic features and it is often "expected" to be working fine just because few common user cases work. The purpose of this post is to correct this myth and look at the finer details of a typical MUI implementation and the areas the could be generally tested.

MUI at a basic level:
What does it actually mean why you stare at a requirement that says- the application you are testing supports MUI ?
At a very basic level, it means that-
1. In the application under test, the strings are separated from the source code. What does "Strings" include in an application ? Strings may include any text on User Interface (or even sometimes Command Line interface) such as Menu names, Text labels,Control names etc. All these are generally static texts, which are usually the candidates for translation into any given language. These strings are separated from the source code and included in the resource files.
2. The second part- at a higher level, of being an MUI supported application is that the application has the ability (built programatically) to load the separated resource files dynamically at the run time.

So MUI software application is the one which is language neutral, has the translatable strings available separately in the resource files and finally has the ability to load the resource files dynamically.

Lets further look into some questions that might come into mind at this time-

What is the need to separate the strings from the source code ?
If the strings are not separated from the source code, then it will reside in the source code. What that means is- the language experts who translate the strings to the target language wont have access to those strings as they usually get the strings in separate files (called resource files) to carry out the translation. In this situation, when the strings are still in source code- it will result in that text being shown in the base language even when the application is localized. e.g. suppose the application's original language is English and there is a requirement to translate in German. So, the strings residing in source code will not get translated in German and we will have part English and part German text because of some English strings still being "hard-coded".

The second question that might arise is-

What is the need to dynamically load the resource file ?
Few examples here-
- Suppose an application supports 3 languages- English, German and French. If user installs the application on German OS, then at the time of installation, the application should recognize the Germaan OS and load the German resource files. If it does not then the German customer may end up seeing English or French text.
- Suppose the organization has a support group in Germany and they deal with English customers. The German support engineer might prefer the User interface to be in German and as he is dealing with English customer, there may be a need for him to switch between German and English User interfaces. This is where dynamically loading the resources and hence, the MUI features comes in real handy.

What are the questions i should ask before testing MUI feature ?
Asking questions helps! and it does always help to ask quite a lot before testing and of course, while testing. Some questions below might help understanding the scope of MUI testing-

- Is the Installer of my application support MUI i.e. does the application Installer installs different supported languages' resource files as a part of Installation ? i.e. if product supports English, German and French languages- does the respective resource files gets into the system during installation ?
- Does the installer recognize the OS locale settings and installs appropriate resources files as per locale settings ?
- Does the application installer provides an option to select the language before proceeding with Installation ?
- Is the Fallback logic implemented ? i.e. say if supported language is German and the application is installed on Dutch OS, does the application falls back to a common language say English ?
- After the application is installed, does it provide the option to the user to switch language at a run time ? i.e.
- If the computer has 2 users with different languages (its possible in Windows OS!) i.e. say English and German and the user logs on to German user settings, does the application desplays the German UI ? This should be taken care of by the application's dynamic resource loading logic.
- If the application happens to be a web application, then too all the above questions will hold good. One additional question to ask, though can be- What is user's browser language ? This is a typical case- when OS language is different from the browser language. In this case, what is the fallback logic ? How should it behave ? It can become more complex say in a situation when the web application runs on French OS with French Canadian browser settings and at the application logon page, the user selects English language as his preferred language. In this case, what language should be shown to the user ?
- How does the MUi logic works in Mac, Linux, Firefox, Mozilla etc. ? Are there any implementation differences that i need to be aware of ?

These questions (and may be more) can help clarify MUI requirement in detail and accordingly testing strategy can be formed. Asking questions is required because Test engineers may never get the requirements in as much detail as might be required for them to test effectively.

Why is testing MUI feature important ?
For an International application, MUI is one of the key features. It is a feature that has helped Microsoft add the required scalability to scale their products across the world. The reason i used Microsoft's example here is because probably Microsoft is the most Global software organization in the entire world. Consider this- Windows 98 supported 30 languages, Windows XP supported 42 languages and Windows Vista- a whopping 97 languages. One of the prime reasons, Microsoft has been able to scale up so much in Windows Vista is because of advancement in MUI technology during the Vista release and this technology is further used in Windows 7 too. So, for an application to be truly global, it has to have a core global arhitecture in place which is what MUI helps achieve.

What are typical Test ideas of Testing MUI feature in the application ?
At the core level, testing MUI means-
- Testing for Language neutral core binary
- Testing for single MUI Installation
- Testing for switching language without Reinstallation
- Testing for the logic of dynamically loading resources
- Testing for language Fallback mechanism
- Testing for Single base binary

Depending upon the answer to the questions asked in the section above, the right testing strategy can be formed.
I think with this information, i will be good to continue the discussion further on what parts of MUI can be specifically tested in your applications.

Please do share your ideas!

Uncovering myths about Globalization Testing- Finding Localization bugs before actual translation takes place

Myth 21: It is not possible to find the Localization bugs before actual translation takes place.

This is a myth because it is possible to find Localization bugs before actual translation takes place using an approach called- Pseudo Translation.

To know more visit my article on Pseudo Translation

Visit here to know about other Globalization Testing myths uncovered in the past.

Uncovering Myths about Globalization Testing- Knowledge of Native Language

Myth 20: If i dont know German at all, i can still effectively test a German application

Referring to Globalization Testing Myth# 4 i talked about why "A person who doesn't know French cannot test the French version of the Software" is a myth.
There is one relevant thought as i was going through Bj Rollison’s interesting blog on Localization testing.
In one of my past experiences, while testing a consumer application- there was a very critical bug that got overlooked resulting in a lot of noise generated by the customers. Basically, the team was involved in testing an update to an existing version for different languages. As none in the team was an expert in the language ,the pure focus of this update testing was to test the changed areas (and none of the changed areas included UI specific changes) with least focus on UI regression as nothing had changed.
After a while the update was released, it caused immense confusion to the customers based out of Germany. The prime reason for confusion was that they were now seeing the User Interface in a "different" language than German, which needless to say offended many customers. And as this was a consumer application, it did affect many home users. The corrective update with right language pack was soon delivered to the customers.
The Root cause analysis of this issue result in crucial learning for all the teams involved. While this issue was mostly because of wrong resource file being referenced for German build, which could ideally have been caught during unit testing. From Testing team's perspective, this issue could have been found earlier by some sort of a checklist that would have been in place which could help a test engineer distinguish between languages by means of checking the presence of language reserved characters etc. This would be highly effective while testing "similar" looking languages.

So, even though one is not a language expert, it does add value to know the nuances of languages enough so as to help test engineers make more informed decision on what he/she is testing.

Uncovering Myths about Globalization Testing- Websites with localized addresses

Myth 19: There is no need to include localized web addresses as a part of your test data as Web addresses are always in English

This was a truth till a little while back. The news is that Internet Corporation for Assigned Names and Numbers (ICANN) has given a go ahead to International domain names.
As per the news details- Currently, domain names can only be displayed using the Latin alphabet letters A-Z, the digits 0-9 and the hyphen, but in future countries will be able to display country-code Top Level Domains (cc TLDs) in their native language. ccTLDs are those that have a two-letter country designation at the end of a domain name.

Source: http://www.pcworld.com/article/181052/international_domains_get_icann_thumbs_up.html

So, Globalization testers- get those fancy web addresses with localized characters added to your test data. There are more bugs waiting to be found.

Uncovering myths about Globalization testing- Context driven planning

Myth 18: There is one standard way of Test Planning the Globalization testing, which is applicable to all the contexts.

I have been looking at the alternate approaches on how different Software Organizations are planning for Globalization testing. Its quite an intriguing topic because the way different Software product organizations go for strategizing Globalization testing is quite unique and yet yield successful results within their own sphere of influence. In the quest to gain knowledge in this area, i came across an interesting presentation at the below URL.

www.localisation.ie/resources/conferences/2007/presentations/TCallanan/Risk-based-L10N.pps

I had an interesting conversation with the Author of this PPT- Mr. Tim Callanan and below is the excerpt of the conversation i had with him-
[Please ensure to go through the above PPT to appreciate the questions and responses below]

A question regarding Test case prioritization and Test Prioritization-
1. The High Risk=1 category shows the Risk as "New Product Feature" and this feature is also on High Importance category (=1). Suppose the Core team has defined 1000 test cases to test this new feature-

a. Will Globalization (G11N) team pick up all 1000 test cases in its execution ?
[Tim] At the start of a project, we will define a number of core modules that we must test. These core modules again will be based on importance. So, for
example, Product A, would have Feature A and Feature B as critical functions, this will be reflected in the importance value that will be associated with this function. Then say if 1000 test cases was associated with this function, G11N team will quickly review all 1000 test cases and will then sample some of those test cases from an G11N view point to ensure that the most critical of these, based on importance and risk to determine priority will be covered.


b. If at all G11N team decides to not use all 1000 test cases (as execution would be costly), what factors would be considered to cut down on number of test cases here ?
[Tim] Again, this will be based upon importance and risk using historical bug analysis, newly written areas of code, potential customer impact etc. We reviewed all core test cases for all products and found a lot of them did not directly effect G11N, the reason being that core testing involves a lot of performance, compatibility, hardware etc. testing that may not effect the Globalized product.

c. What kind of test cases other than these 1000 or subset of 1000 (depends on approach a. or b. above) would be created from G11N perspective to test the particular feature ?
[Tim] The main areas we would need to look at would be tests that would effect G11N but would not be covered by core. This would involve particular 3rd party applications that would only be available in the regions e.g. Japan use a lot of e-mail clients that are just not sold outside Japan, compatibility with these 3rd party apps would be critical for L10N. Also some type of I18N testing i.e scanning a DBSC or extended character directory for example or compatibility with a particular piece of hardware that is only sold in the regions. Also the insertion of non EN characters and lots of UI issues such as char. corruption, clipped strings, etc.

The approach defined in your presentation seems to be very effective. What kind of deliverables are expected of Core development team to ensure that Risk based L10N testing works ?
[Tim] The most important deliverables is that core team adhere to what was agreed at the milestones of the Software Development Lifecycle. By including G11N criteria at each stage of SDLC and ensuring that the Core team deliver these will ensure that nothing is overlooked that could potentially make any piece of software 'unlocalisable' or that would resulting major L10N or I18N defects being found at a very late stage in a process that would cause delay in the I18N releases.

Deliverables like-
The code base is same for all Single byte languages.
The code base is same for all double byte languages.
etc.
[Tim] Yea, there must be only one code based for all language versions. So here I18N considerations are critical. For this reason i18N testing is included as a critical functional of either the core or L10N teams (depending on what can be agreed between the two groups) to be execute as early as possible in SDLC. I have seen in the past, in other companies, how the code base was split and a separate code based used to address i18N defects that was found late in the project to ensure that the En-US version could ship on time. It turned out to be a real nightmare trying to maintain to separate code bases and merge code later in the project. This is definitely something to be avoided if at all possible.
The key I think to ensure a good Localised and Internationally enabled product is to ensure that there is 'Functionality parity'; between software products running on EN OS and all other Localised OSs. In other words the EN product on En-US OS becomes your base line and all Languages version of software running on any non-En-OS will functional in exactly the same way as the English product. If we don't have this we have big problems.

Its a well known fact that most of the tests that Globalization team does are derived from English test cases. Another fact is that it is not efficient to run all test cases on all the languages.
Is there a standard that you know or a norm that you follow in testing that defines-
- How much %age of English test cases should be run on the localized languages ?
[Tim] To answer your question, we usual base this upon risk and importance, this was the focus of the LRC paper that I wrote a couple of years back
http://www.localisation.ie/resources/conferences/2007/programme.htm. It is hard to sat exactly the percentage of test cases we would cover as this would be based upon product maturity, feature set, degree of internationalization testing already conducted, the level of pseudo translation testing already undertaken etc. etc.
Based on projects that we have done in the past, L10N would involve testing roughly 30% of the cases that are conducted by the core team but this is a very general estimate and we would use the risk based testing method to ensure that all necessary tests are carried out. If this comes out at 30% then that would be our coverage but it will depend on the risk that is assessed.

All Globalization Testing myths uncovered so far....many more to come

Its been a while that i have been writing about uncovering various myths associated with Globalization testing. While there is a lot of relevant information on Globalization testing already listed in this blog, i am inclined to think about and work towards consolidating this information and much more in the form of an ebook in the time to come. Though i dont have definitive timelines in mind at the moment, this is something i have started working towards already. For the benefits of the readers, while i work to uncover more myths around Globalization testing, i have consolidated all the previous posts on this topic below.
[Just to remind that all these myths are the real time myths that i have experienced while working on Globalization testing]

Uncovering Myths about Globalization testing -1
Myth 1: G11n testing is not technical enough
Myth 2: G11n testing is majorly about testing the UI
Myth 3: G11n Testing can start only after the base product is translated

Uncovering Myths about Globalization testing -2
Myth 4: A person who doesn't know French cannot test the French version of the Software
Myth 5: A tester only needs to follow the test cases executed for Base language in order to thoroughly test the internationalized applications
Myth 6: There is no scope of exploratory testing while testing internationalized applications
Myth 7: The language verification of User Interface can be done by comparing the text on screen with translation outputs of any freely available Online translator.

Uncovering Myths about Globalization testing -3
Myth 8: If a test case works fine in French language, it will work fine in German language as well
Myth 9: If the Foreign text input in application text fields work fine by using the Soft keys, then it means the data input through respective Foreign language key board would also work fine.
Myth 10: Globalization testing doesn't require the same test setup as is required to do the Base language testing. Globalization testing can be done with a minimum test setup.

Uncovering Myths about Globalization testing -4
Myth 11: Localization - means Localized product on a localized Operating System, Internationalization- means Localized product on English Operating System

Uncovering Myths about Globalization testing- Demystifying MUI Packs
Myth 12: Testing International applications using "Microsoft's MUI Pack" or "Localized OS installation" means one and the same thing

Uncovering Myths about Globalization testing- Input validation testing
Uncovering Myths about Globalization testing- Input validation testing 2
Myth 13- A tester can perform tests specific to text inputs for Localized applications using the similar approaches as the English language testing

Uncovering Myths...."Security Testing is from Mars and Globalization Testing is from Venus"
Myth 14- Security Testing is from Mars and Globalization testing is from Venus

Uncovering Myths about Globalization testing- English version on Localized setup
Myth 15: There is no use testing the English version of a product on Localized Operating systems

Uncovering Myths about Globalization testing- MUI Packs in Win XP and Win Vista
Myth 16: There is no difference between MUI technology being used in Win XP and Win Vista

Uncovering myths about Globalization Testing- Reusing Test Automation
Myth 17: The test scripts meant for English language automated tests cannot be reused for Internationalization testing

Hope you enjoyed going over these again. Would appreciate your comments, suggestions!

Uncovering myths about Globalization Testing- Reusing Test Automation

Myth 17: The test scripts meant for English language automated tests cannot be reused for Internationalization testing

The idea of making a Software test Software without any human intervention is something that is the basis of automated testing. In my experience, i have seen fewer things that catch the fancy of management other than a well crafted Test automation strategy. I was recently studying one of the Surveys listing the success of Software projects across different Software development models. While it is interesting to notice how the project success ratio varies from different Software development methodologies but one important point to note here is that irrespective of the Software development methodology used, the success ratio is far less than maximum. Before i sound like deviating further in this post, the idea here is not to compare how different Software development methodologies fare in the real projects but certainly to draw an analogy that like the success ratio of Software projects, the success rate of Test automation projects (the lifecycle of which is quite similar to Software development projects, though on a smaller scale) is also quite far from ideal. I don’t have a real data to quote here and this inference is quite based on my observations and experience.

While the reasons of Test automation projects' failure can be plenty and its not my intention to dive deeper into those here but in this post i would be focusing on one of the main reasons automation efforts fail to scale up for International languages' testing.
To start with, It’s actually a colossal myth that the test scripts meant for English language automated tests cannot be reused for testing with International languages. I call it "colossal" because the root cause of the automated test scripts for any projects not working on International platform is no less than a design flaw or a lack of forethought at the time of scripts’ design. In my belief, the Software organizations (which went global quite late) have spent sizable costs due to rework relating to introduction of Internationalization quite late in the overall life cycle. On the similar lines as this, Test automation projects have also suffered great deal of rework because the test automation scripts were not Internationalized upfront. Let me try and elaborate what i mean here-

- For a Software product to be properly Internationalized i.e. ready to be sold in International markets- the product design should-
• be Unicode enabled i.e. the product should be able to process and display data in any language
• have all the strings externalized i.e. none of the strings that show up on User Interface should be hard coded.
• have the capability to know under what language OS is it running on i.e. if the product is installed in Japanese, the product logic should recognize that.
• Should be able to read locale specific details from underlying OS
• be robust enough so that the same code base can be used across all the languages that the product support.


- On the similar lines, for Test automation script to be fully internationalized i.e. working successfully on different languages,
• The Test tool should have a capability of being installed and running on different language OS.
• The Test tool and hence, the automated test scripts should have the capability to process the Unicode i.e. any language text and characters. This is because any testing that's done on International test environments makes use of International character sets(which is what the customers would do)
• The Test scripts should have a capability to understand under what language OS is it running.
• The Test data is not hard coded in the Test automation scripts and is Externalized to a file or any database. This helps because, once the Test scripts detects the language of the OS, the "right" language Test data files can be loaded.
• There should not be separate Test scripts per supported language but a single version of Test scripts should be capable to run on multiple languages.

I must admit the above is not a comprehensive list of features required for enablement of Internationalization in automated scripts but i feel it is good enough list to give an idea on things that need to be kept in mind while designing automation scripts which are scalable in multiple languages. Traditionally, the automated scripts are designed keeping in mind that they have to be run on English environments only. To me, that is a sort of cardinal sin because it robs the Software organizations of reaping multi-fold benefits from automation efforts. Consider this- if by automating the English language Build verification tests, the team handling English testing saves 1 man month of effort, it is just a simple mathematics to judge the benefits the same automated scripts will reap if they are designed to run on 10 languages.

In today's global market place, if your product currently doesn’t support International markets that does not mean that it would not in future also. If it is not already, Globalization of Software products would eventually be required for survival of organizations. In such a dynamic scenario, it is imperative that like smart individuals- we think of future possibilities upfront and include Internationalization in our Test automation design right from the word go.

I would be exploring this area further in detail in the upcoming posts and I am keen to hear your views and experiences around Test automation in general and specifically on leveraging Test automation scripts in International environment.

Uncovering Myths about Globalization testing- MUI Packs in Win XP and Win Vista

Myth 16: There is no difference between MUI technology being used in Win XP and Win Vista


This myth has a little bit of background in the Globalization testing Myth# 12 . This post is rather an extension to the before-mentioned post.
While performing the Globalization testing, one of the key test environment specific decisions is whether to use native language Operating system or make use of MUI packs as Microsoft provides both the options for its users. The previous post does touch upon this aspect. As i delved more into this topic, i had a realization that MUI packs do not actually behave the same as the different flavors of Windows Operating systems. With a bit of research, i was able to arrive at the below table listing the differences between the way MUI technology works in Windows XP and Windows Vista. This information has largely been derived from the below mentioned sources.

.

. Category Win XP Win Vista . MUI architecture Some part of Win XP code is language dependent (e.g. country specific device drivers) Separates the language resources for the user interface completely from the binary code of the operating system. All installations of Windows Vista contain at least one language pack and the language neutral binaries that make up the operating system. . Mode of Installation MUI Packs can be installed over English version of OS only. Language Packs can be installed on any language version of Windows Vista. E.g. a user having Spanish Vista Ultimate can install German Language packs and switch between these 2 languages on the same system. . Maintenance updates Windows XP usually releases one Service pack per language Being completely built o language neutral code base, Windows Vista can be updated by a single software update anywhere in the world . Upgrades The Multilingual User Interface Pack only supports upgrades from English versions in case of Windows XP. All upgrades must be performed in the same language in case of Windows Vista. Vista Home Basic (EN)-> Vista Ultimate (JA) not allowed. . Licensing requirements For non Win Vista Ultimate, Vista Enterprise installations: Only single language installation. For these editions, Windows will automatically remove all non-default languages from the computer after the end user completes the Windows Welcome. . File and folder names created at setup In case of Windows XP, the files and folder names created at setup such as "Program Files", "Documents and Settings" etc. remain in English language. In case of Windows Vista, the corresponding Files and Folder names created during setup changes to the language of the MUI pack. . Multiple Languages on same Installation Windows XP allows having multiple languages on the same computer system provided the original language of OS is English. Available with any language . Languages support 42 97 .

Source: http://technet.microsoft.com/en-us/library/cc721887(WS.10).aspx
http://msdn.microsoft.com/hi-in/goglobal/dd218461(en-us).aspx

Uncovering Myths about Globalization testing- English version on Localized setup

Myth 15: There is no use testing the English version of a product on Localized Operating systems

This myth has a little bit of background in the Globalization testing Myth# 3 i.e. Globalization testing actually should start much before the International product is translated i.e. the User Interface, Documents etc. start showing up localized. The question here is how is the Internationalization testing done when the translated product is not available. I will just attempt to explain this in various points below-

Does English version of product and International version of product gets released for testing at the same time ?
In a more matured Software development life cycles, the code complete specific to English version of the product and the International version of product gets submitted at the same time i.e. one single English build have the code changes specific to International version as well.
In some Software development life cycles, the Internationalization specific changes gets introduced only after English build is released.

Can translation of product happen before English product's User Interface is frozen ?
Also, it is a well known fact that the in an ideal scenario, the Translation of product User interface into supported Localized languages happens only after English product's User Interface is Frozen.
* English product's User Interface is Frozen only after some cycles of testing in which entire User Interface is tested in different setups (OS, browsers etc.) to ensure that there all the User Interface specific issues are found and fixed before the different texts are translated. And even after User Interface freeze, the translation activity acually takes quite a bit of time because it is usually a manual process and has its own cycles of reviews before translation gets finalized.
* It is quite evident that since the time first build with Internationalization specific code changes is introduced to User Interface Freeze milestone to actual Translation of the product, there is a lot of time that gets elapsed. If Internationalization testing does not start when the first build (usually English build) is received, then many of the Internationalization specific changes will not get found until the Test team receives the translated build.

How can Internationalization testing happen when there is no translated software available ?
* The answer to this question is testing English version of the product on Localized setup e.g. Say a product is supporting German and Japanese languages and supports Windows XP, Windows Vista, Mac 10.5 OS- Internationalization testing in this case would involve testing English product on German Windows XP with German Internet Explorer 7.0, testing English product on Japanese Windows Vista with Japanese Firefox etc.

What kind of bugs is this type of testing (Testing English product on Localized Operating systems) helping to find ?
One may always argue that testing English product on a Localized version of the Operating system will always result in English specific issues because what we are essentially testing is the English product. This may be true to some extent but not entirely. Consider the below situations-
* The Product installation works fine when English product is installed on English Operating system. The Product installation fails when English product is installed on Spanish language. The reason- the Install Path name is hard coded in the product. The product usually gets installed in "Program Files" folder in Windows. "Program Files" folder is called as "Archivos de programa" in Spanish language.
* The data input using English characters e.g. Writing the name as "Anuj" works fine in English Operation system. When using the same build on Japanese Operating system and using the name as "廃れる" fails. The reason- the product does not recognize the Japanese language data.
These are just a few basic examples and there can be many such instances of unique bugs that can be found (i will cover this aspect in more details in upcoming blogs)

Keep testing passionately and do provide your feedback to me!

Uncovering Myths...."Security Testing is from Mars and Globalization Testing is from Venus"

This post is a continuation of my previous post based on the real time myths about Globalization testing as i have experienced.

Myth 14- Security Testing is from Mars and Globalization testing is from Venus

Introduction:
One of the intriguing areas in the sphere of Software Globalization testing is planning/performing testing of the international application from the security perspective. One of the popular myths or rather assumptions when talking about Globalization testing is that the software applications usually do not have any impact as far as Software Security is concerned and thus the Security testing is not required to be done on an International application. While this may be true in certain contexts but there is also a large possibility that system security gets compromised with incorrect assumptions about the topic. Security related bugs usually have a high business impact and are (in most cases) costlier to fix than the related functional bugs.
Without doubt this is a much broader topic to be reasonably covered in one article alone, this article is primarily an attempt to put together a "perspective" of how different security related aspects may have impact on International Software applications.

A background in possible Security considerations for International applications:
Unicode provides a unique number for every character, no matter what the platform, no matter what the program, no matter what the language. In many ways the emergence of Unicode standard has changed the way Software Internationalization has been perceived and carried out in product design. It has been one of the significant advancement over the past encoding systems. Unicode 5.1.0 contains over 100,000 characters and encompasses of large number of different writing systems of the world, the faulty usage of the same may result in potential security attacks. Unicode consortium defines the major security threats related to International applications in 2 major categories-
1) Visual security issues
2) Non Visual security issues

Visual security threats:
The threats under this category are nothing but the Visual spoofs. Since Unicode consists of myriad of characters, there is a good probability of a layman user coming across visually confusing strings. There are no hard-and-fast rules for visual confusability- many characters look like others when used with sufficiently small sizes, in different font, considering different sequences of characters e.g. In some cases sequences of characters can be used to spoof: for example, "rn" ("r" followed by "n") is visually confusable with "m" in many sans-serif fonts.
As security expert Eric Johanson mentions in an advisory, a security weakness in a standard for handling special character sets in domain names could let an attacker spoof Web sites. There are now many ways to display any domain name on a browser, as there are a huge number of character sets which look very similar to Latin characters. The advisory demonstrates the attack using the domain for PayPal, but using an alternate Unicode character for the first "a." That gives an address that looks like "http://www.pàypal.com," but with a "à". This can enable an attacker to create a fake Web site for a phishing scam.

Non Visual security threats:
Non Visual security threats primarily deal with how the Unicode data is interpreted by the system. There are different security flaws that can be exposed by indifferent use of Unicode data by the system. Some of such attacks include-
UTF-8 Exploits, Text Comparison, Buffer Overflows, SQL Injection Vulnerabilities, Cross-Site scripting, Format String vulnerabilities etc.

Potential Security tests on International applications:
For the sake of simplifying the usage of the tests, the Security tests on the International applications can be divided as-

1) Security tests based on Functional requirements
2) Security tests based on Non-Functional requirements

Security tests based on Functional requirements:
These tests validates the Security sensitive functional portion of the system. The tests that would primarily come under these categories would pertain to applications'-
a) Authentication
b) Authorization

Authentication and Authorization tests usually go hand in hand.

If the application is going to be used in International markets, then the relevant Security tests here would be-
- To use the International characters in the supported languages. From the Security perspective, depending upon the reach of the product - the test characters can be of languages not supported by product but are supported by Unicode.
- If the product authorization is dependent upon presence or absence of an dependenant application, for an international application it will make sense to use Localized version of the applications.

Security tests based on Non-Functional requirements
Non-functional security requirement can be something like “System should not be compromised.” As is clear from the wordings, this requirement is not associated with a specific feature and it’s a very generic but an important requirement. In order to look at the Security aspects of an application accurately, it is necessary to have a holistic view of the situation. The most predominant challenge is that if the requirement is as vague as the one mentioned above, there is no one simple test be performed to make sure that the security requirement is met.
One of the possible approach that can be used to find such vulnerabilities is to generate the Flaw specific test ideas. The obvious question is which flaws to consider. Below listing has a mention of few flaws that can potentially have an impact on Internationalized applications and some ideas on how the tests can unveil these. This, by no means is the complete list of Security tests for International applications.

a) Buffer Overflow vulnerability:
o About the Buffer overflow vulnerability-
Buffer overflows occur in the software written in programming languages that do no strictly enforce bounds checking on arrays. The basic concept of a buffer overflow is that we provide an application with more data to be stored in a particular variable than the programmer setup the space for. When this happens, it is likely that the application writes past the bounds of the variable buffer, allowing an attacker to change the value of other data stored in memory and even execute the malicious commands. It is easiest for the attacker to perform buffer overflow attacks on the stack. These attacks can happen over heap but considering the dynamic nature of heap, these are usually difficult to simulate.

o How can this vulnerability impact Localized applications-
One of the possible ways this vulnerability can impact Localized applications is that the application may have typical checks built-in regarding ASCII text but the entering the Unicode data may expose the buffer overflow vulnerability.

o Possible tests on Localized applications to unveil Buffer overflow vulnerability-
1. Identify the areas of the application that are potentially vulnerable. Majorly, it would all the areas where the application accepts user inputs and particularly the areas that are exposed to wider audience. Potential candidates for this type of vulnerability would be any text fields within application that do not have any input validation check.

2. The Localized application might have done the Input validation on the text fields by virtue of number of characters supported e.g. Say, an Input field is programmed to accept maximum of 45 characters for the name field. If the character input is in English, the 45 characters will amount to 45 bytes in case of UTF-8 encoding. But if the character input is in Japanese, then depending upon the character input one character might take 3 bytes- which will amount to 45*3= 135 bytes of "acceptable" data input. Such an application is potential candidate for buffer overflow vulnerability attacks, as this gives an opportunity for the attacker to input malicious code along with the input text.

3. Depending upon the underlying encoding system in use, number of bytes a character occupies varies e.g. the character "は" occupies 2 bytes in UTF-16, 3 bytes in UTF-8 and 5 bytes in UTF-7. Thus, by using character driven Fuzz techniques the test data can be generated to simulate a situation when the character data exceeds the available buffer space.

4. Converting strings between different character encodings (such as SBCS, MBCS, Unicode, UTF-8, and UTF-16) may produce a buffer size mismatch. Being aware of the areas where such conversion is happening in application may aid the test team to focus to find this vulnerability.

5. If the application reads certain text or data embedded in the communications protocol, this source can be populated with localized text to simulate the buffer overflow attacks. e.g. by some internal operation of the program, the string may expand- which will result in enlargement of the buffer. Strings may expand in casing: Fluß → FLUSS

b) SQL Injection vulnerability:
o About the SQL Injection vulnerability-
SQL injection is an attack in which malicious code is inserted into strings that are later passed to an instance of database server for parsing and execution. The primary form of SQL injection consists of direct insertion of code into user-input variables that are concatenated with SQL commands and executed.

o How can this vulnerability impact Localized applications-
This vulnerability can be tested in the applications with an interface to the database. In case the localized applications are using the database with localized schema, then this vulnerability (if existing) might be exposed by entering alternate encodings for the potentially problematic characters such as Apostrophe, Quotation mark, Comma, Bracket etc.

o Possible tests on Localized applications to unveil SQL Injection vulnerability-
1. Make a note of all the user input fields that commit the data to the database.
2. Generate the test data that includes the data changing or even schema changing commands. There are a lot of publically available SQL vulnerability cheat sheets available that can help generate the relevant test data depending upon the database being used.
3. One of the key changes that could be made to the test data is to use the equivalent characters in the different supported languages.

c) Other Security vulnerabilities:
One of the more methodical ways while considering the Security testing for International applications is to commence with the creation of appropriate threat model. A threat model is a description of a set of security aspects; that is, when looking at a piece of software (or any computer system), one can define a threat model by defining a set of possible attacks to consider.
There are other notable Security vulnerabilities that can have potential impact on the security of International applications (and can be considered in threat modeling) such as Format String vulnerability, canonicalization exploit and with more Software vulnerabilities being found with each passing day, the list of vulnerabilities having impact on International applications can never be fixed but would always be ever growing.
Local governments may have their own specific security requirements. For example, any product that either uses or implements cryptography for confidentiality must obtain necessary approvals from the French government prior to shipping to France.

Epilogue:
It is quite evident that the International application does bring out the unique challenges from Security perspective. There is a certain intersection between Security testing and Globalization testing, something that cannot be ignored. The adage "Security testing is from Mars and Globalization testing is from Venus" is possibly not quite right! and this is certainly one area that is waiting to be explored further and researched.

References:
http://www.unicode.org/
http://www.isecpartners.com/files/iSEC_Scott_Stender_Blind_Security_Testing.bh07.pdf

Uncovering Myths about Globalization testing- Input validation testing 2

This post is a continuation of my previous post on the same topic and is based on the real time myths about Globalization testing as i have experienced.
In my previous post, i talked about how the byte count varies across different Unicode representations. Any tester reading this may further have a few questions here-
- What is the right approach to come up with the localized test data ?
- Once i have the test data, how do i know which localized character occupy how many bytes considering i know the type of Unicode representation (UTF-8, UTF-16 , UTF-32 etc.) ?
The answer to the first question is in itself a very broad topic and i do plan to cover this in my future posts.
The second question i.e. decoding how many bytes a character occupies for a given encoding is equally interesting. I have found one of the General testing tool created by Bj Rollison known as String Decoder quite useful.
The Bj Rollison's website has great amount of details/user guides etc. about this great utility for Internationalization testing.

Uncovering Myths about Globalization testing- Input validation testing

This post is a continuation of my previous post on the same topic and is based on the real time myths about Globalization testing as i have experienced.
Myth 13- A tester can perform tests specific to text inputs for Localized applications using the similar approaches as the English language testing

Introduction:
The testing specific to Input field validation is an important form of testing in any Software application. An example of such testing can be suppose an application having a text field to input Credit card details and a tester can test the same by including various possible inputs to ensure that the valid data is being accepted by the application and the user is presented with a valid message indicating that the input is incorrect.

There are a several techniques that can be used to test this aspect of the application properly. Some available resources listed below-
http://www.testingeducation.com/BBST/Domain.html
Book: Lessons Learned in Software Testing Chapter-3 Testing Techniques Section- "How to create a Test Matrix for an Input field"

What is an encoding system ?:
Though the known techniques do talk about usage of various types of inputs including Language reserved characters e.g. the characters specific to any language that a Software application may be supported such as German, Japanese etc. as these languages do have their own writing systems and character sets. It is of utmost importance to test a Localized application with the language specific characters as any user in any of the product's supported countries would expect the application to support data processing in their own native language e.g. a Japanese user using an email client would expect the application to support writing emails in the Japanese language, otherwise the customer may not find the application worthwhile at all.
One of the important aspects specific to Localized data processing that the known techniques do not specifically talk about is the dependency the Localized data has on the underlying encoding system in the application. If you are new to the term- "encoding system", please read below mini description from www.unicode.org-
Unicode provides a unique number for every character, no matter what the platform,no matter what the program, no matter what the language.
Fundamentally, computers just deal with numbers. They store letters and other characters by assigning a number for each one. Before Unicode was invented, there were hundreds of different encoding systems for assigning these numbers.

Does Unicode have different representations ?:
Unicode is actually an encoding system that encompasses virtually all the known character sets from different languages. There are several possible representations of Unicode data, including UTF-7, UTF-8, UTF-16 ,UTF-32 etc. Each of these different representations have its own advantages and disadvantages depending upon the context e.g.
UTF-8 is most common on the web. UTF-16 is used by Java and Windows. UTF-32 is used by various Unix systems.

Does encoding system representation affect the test data size ?:
One important fact to consider when testing the Input character set for an localized application is to know what type of encoding system is being used beneath. The reason why is it so important to know the underlying encoding system is that no. of bytes occupied for a certain character varies depending upon the encoding system used. Lets take a closer look at this statement by means of an example-
Take into consideration the following character from German language "ä". The byte count of this character depending upon the encoding system used is as follows-
UTF-16 Byte count for "ä"= 2
UTF-8 Byte count for "ä"= 2
UTF-7 Byte count for "ä"= 5

The above example shows that the encoding system do have an dependency on the no. of Bytes for a particular test character.

Different ways of Input text validation- No. of Bytes vs. No. of characters ?:
The next important factor before performing the Input validation testing in Localized applications is to know whether the validation logic is done as per no. of Bytes or the validation is done with no. of characters. Lets take a closer look at this statement by means of an example-
Suppose there is an application with a text field say Username. The usual assumption is that the validation will be done by no. of characters say the "Username" field will support maximum of 10 characters and a minimum of 3 characters.
Suppose a tester uses test data for "Username" as "ääääääääää" and the application is using encoding system as UTF-7. If the validation is done as per No. of characters, then the above is a valid test data as it represents 10 characters. In case the validation is done as per No. of Bytes, then it may not be a valid data (depending upon the Byte limit set), as the test data in the above example may amount to 50 bytes.

Thus, it is important to ascertain before you test to ensure the validation rules.

Summary:
So, before you consider performing the Input validation testing or even generate test data for testing for Localized application ensure that you know about the following-
- Encoding system used by the application
- Validation rule- does the application validates the data as per Bytes or by no. of characters ?