How safe am I, if I use your application?

Mission:

Find security information which makes the application to loose competition for competitor's similar application.

Application under Test: File Uploading Application.

Session ID: FUA-1

Start Time: 27th March 2010 12:08 PM IST.

End Time: 27th March 2010 3:25 PM IST.

OS: Windows XP SP3.

Browser: Firefox 3.6.2

Anti Virus: Symantec Endpoint Protection.

Context:

Tester has been given just an application release and asked to test it, which is available in market for users. Application owner is expecting information from testing in few hours. The application has demand in market and being purchased by users to upload files to desired FTP sites. In view of the sustenance in competitive market, application owner decides to test for security threats. A tester is assigned with a job to find security loopholes information and to identify how their competitor's similar application withstands the identified security loopholes.

Tester starts learning:

• What the application is?
• What platforms does the application support?
• Why should user this application? What are the benefits of using this application?
• Who are the users of this application and for such applications?
• Who the competitors are, with similar applications?
• How the application is used (or can be used) when available?
• Why they use this owner's application in particular, when other such similar application are available in market?
• What type of files can be uploaded? What is the minimum and maximum file size that can be uploaded?
• When, how and why the application "does not do what it appears to do"?
• How the application "does what it does" and "should do what it is expected to do"?
• How the application is built?
• Any limitation known currently in the way how application appears to work?
• Any defects reports not fixed or deferred or not released with present build? If yes, why?
• Do the users of the application are satisfied using it today and earlier? If no, about what they are concerned in using application?
• Are users looking for new things in application today? if yes, how quick they need it and how they value application if it is available and if it is not available?
• Does application was cracked for license key earlier? If yes, how frequently the license key or file was cracked or generated? If no, did the team at development site tried to do it so and what was the result? What the decision maker(s) decision when license file was cracked or when it was not possible to crack?
• Does application server at owner place was hacked earlier? If yes, how frequently it was hacked? What was the cost of these hacks? If no, did the team at development site tried to do it so and what was the result? What was the decision maker(s) decision when server was hacked or when it was not possible to hack?
• How the sales and revenue for the application has been along with the similar application of competitor's?
• Does all claims of application are met? If not, which claims are not met and for whom? How bad the cost is to them for not meeting the claims? What claims have been met?
• Any records or feedbacks from the users of application are maintained? What those records or feedbacks say?
• What does the customer support feedback say for service being given for application users?

My mistakes in this session:

I was not able to test with all security test ideas I had in test coverage model in given time. Invested time in investigating behavior of application. Need to concentrate and achieve mission in given time by exercising application using test coverage model. Test report can be found here.

I learned whenever we are using anti virus, firewalls, routers, port scanners or any other threat identifying tool, we are using others program code built as an application. These application can have defects within them. The questions that remains now in me is, can an application be easily accessed when unexpected happens to security providing application? If did so, what will it cost the user? If security is understood as freedom from fear, then why many software applications get hacked or cracked? Do software application have no freedom?

European Weekend Testing 09 -- Test and Experience Report

Mission:

You are moving from lovely Europe with measurements based on the metrics system to the US with imperial units. Test Converber v2.2.1(http://www.xyntec.com/converber.htm) for usability in all the situations you may face. Report back test scenarios for usability testing until 4.30pm GMT on bug repository.

Application Under Test: Converber 2.2.1

Actual Event Date: 13th March 2010, 09:00 PM IST

Start Time: 14th March 2010, 09:03 AM IST

End Time: 14th March 2010, 10:15 AM IST

Test Machine: Windows XP SP2

Tester: Ravisuriya

Modeling of context:

I am a common man with elementary school education and a very rare computer user. Working for a mining firm in Europe now transferred to US for few years on contract job. Came to know by a colleague that measurement unit changes in Europe and US. With the help of a person in Internet Center browsed for the measurement information. Found few details of Imperial and US customary measurement systems. Later used tool Converber V2.2.1 to note measurement unit I needed.

Understanding of term ‘Usability’:

Usability: How comfort is it for a user to use AUT? How quick can the user learn to use the application for desired purpose? What efforts by the physically challenged people need to put for using the application? Does application supports the Operating Systems accessibility features? What efforts does the user need to put while using it?

Session Report:

I did not participate in the EWT 09; but I practiced it offline on the next day. Report from EWT team is here. Chat transcript of discussion session is here. PDF document of my report is here. The usability scenarios that I identified during the practice session:

1. User launched the AUT. Was user able to use AUT with the GUI objects available in it when AUT launched? Did the GUI objects help or retarded the speed of using AUT? Was the naming convention of GUI objects was self explanatory to user?
2. User entered values for a selected unit. How to identify the value seen is of type ‘Imperial’ or ‘US customary unit’?
3. Users want AUT to have a default option of either ‘Imperial’ or ‘US’ units.
4. User selected units are same; what message is displayed now?
5. User not aware whether the calculation of unit value for desired, is correct or not. How to test this by user?
6. User entered variant value which is not acceptable by the selected unit. How such instances are handled?
7. User need to convert of area, distance, volume, speed, power, pressure, luminance, temperature and other common used units in industries and daily life. Is there any option to see most common units converted or used across the globe?
8. Whether all Imperial units are available in AUT for conversion to US customary values?
9. User feels it was difficult to learn using AUT. Searches for help manual. Is a Help manual available feature, limitations, bugs and contact details?
10. User wants to enter new units that do not exist in AUT. If added, how the conversion procedure for added units can be added in AUT?
11. How simple the words and contents are available in AUT, so that user with no much school education can use application with ease learning of units and conversion?
12. User does not understand English well and can use AUT in Farsi. Does AUT support of Farsi or other languages? Does all the displayed and available contents in AUT are shown in Farsi?
13. User changed language in AUT to ‘English’. Did the AUT have any Farsi words displayed now?
14. User did not know much information about the Imperial and US Customary Values. Did the help manual have that information for user to understand them?
15. User wanted to have print outs of the converted unit values. Does AUT support the print? What are the ways the user can save the converted unit values for using it later? Does AUT support all of these or few among them? Which is most commonly used option to store the converted unit values by user?
16. User entered the value for a selected unit. The converted unit showed scientific notation. How user can convert that scientific or mathematical notation to values which is understood by user?
17. User entered a value which was not valid for a chosen unit. Can the user identify the displayed pop up dialog is for invalid value entered and it is from AUT? Closing the dialog will it allow user to continue using the AUT? Does the dialog appear in foreground or in background of AUT? If appeared in background which user cannot identify it, will user be able to continue using AUT? What options are available in displayed dialog to assist user?
18. User is visually challenged. How the AUT does help user now to know the converted unit or to convert the values into US units from Imperial?

It was a good exercise for me to identify the usability scenarios. At end of this session, had a feel that I did not do well enough in identifying the usability scenarios. When I am finding few more such usability ideas after the session, I could have done better in identifying scenarios; need more such practice.