Cloud computing creates software testing challenges

The "cloud" promises to create new opportunities for enterprise developers as well as for suppliers offering services and tools for this new paradigm. For testing organizations, there will be both new challenges and new tools for answering what Soasta CEO Tom Lounibos calls the one key question: Can I go live?






The impact on testing is that the end-user experience is being influenced by the cloud provider and all other parties involved.
Vik Chaudhary
VP of product management and corporate development, Keynote Systems Inc.











"Testing all the layers — from your application to the cloud service provider — is something testers will have to become efficient in," said Vik Chaudhary, vice president of product management and corporate development at Keynote Systems Inc. in San Mateo, Calif.
According to market research firm IDC, spending on IT cloud services is expected to grow nearly threefold, to $42 billion by 2012. Cloud computing will also account for 25% of IT spending growth in 2012 and nearly a third of the IT spending growth in 2013, IDC projected.
IDC makes a distinction between "cloud services" and "cloud computing." Cloud services, according to the market research firm, are "consumer and business products, services, and solutions that are delivered and consumed in real-time over the Internet." In contrast, cloud computing as defined by IDC is the infrastructure or "stack" for development and deployment that enables the "real-time delivery of products, services, and solutions over the Internet."
Chaudhary explains the shift: "Enterprises like Schwab, Travelocity, etc. have been deploying their own data centers for years. The challenge was to manage highly scalable applications and how to ensure the best experience. Legions of people were employed by these companies to monitor/test/add servers, etc."
What's happening more recently with new cloud infrastructure like Google App Engine, he said, is that organizations can run their applications on Google's infrastructure.
"That means the bar to deploy applications in the cloud is so much lower. You don't have to have data centers or ops teams; you can focus on building the application and the functionality. It's a paradigm shift in application development," he said.
It's a shift for the tester, too. For example, Chaudhary said, "If you build an application and you use the BlackBerry to access a manufacturing application hosted by a cloud company like Salesforce, Salesforce does a certain amount of testing, to ensure the server is available, etc. But when it comes to the application itself, does it run on two phones or 50 phones? Do you have a long page to load?"
In addition, the cloud hosting company may use a third-party service to speed performance. "The impact on testing is that the end-user experience is being influenced by my company, by the cloud provider, and all other parties involved," he said.
Reducing testing costs
While Lounibos said Mountain View, Calif.-based Soasta Inc. has a growing group of customers that don't own servers and do everything in the cloud, "the majority are still more traditional; they use managed service providers and are dabbling in the cloud." However, he said, cloud-based testing is a way for organizations to learn the cloud and reduce the costs of testing at the same time.
"Traditional customers see testing as a money pit. They're looking for ways to reduce costs. The [main] argument for cloud computing for the enterprise is, is it reliable enough," he said. "This is not so for testing. Testing [in the cloud] just replicates the real world; it doesn't have the issues associated with production, but it has the benefits of cost reduction."
With cloud computing, Lounibos said, testers "have access, availability, and affordability to enormous amounts of computing power, which is what's needed in testing. The idea of being able to provision 125 servers in 5 to 8 minutes and only pay for the hours you test is so compelling. You no longer have to have huge test labs for Web applications."
Soasta's CloudTest, for example, is available as an on-demand virtual test lab in the cloud or as an appliance. It supports load, performance, functional, and Web UI/Ajax testing. According to Lounibos, "We were built on top of the cloud for the cloud."
For its part, Keynote offers KITE (Keynote Internet Testing Environment) for testing and analyzing the performance of Web applications across the Internet cloud. KITE offers instant testing from the desktop as well as from a variety of geographic locations.
For Internet applications in particular, Chaudhary said performance testing needs to move to the cloud.
"When it comes to performance, you're not depending just on the application but on all the providers [involved]. And do you [the user] have DSL or a dialup line, or a mobile device? Performance testing by nature is environmental," he said.
For mobile applications, Chaudhary said both performance and functional testing should move to the cloud.
"For mobile applications, the functional testing also depends on the providers. Say you've got a screen for login. The size of the page and the screen on phone and the provider can all affect if the application works," he said.
By testing in the cloud, Chaudhary added, organizations can more easily and cost-effectively test for hundreds of devices.
With applications that run on the cloud, "you need to test network performance, server performance, database performance, software performance on the application, and how it's cached in the client," said Dennis Drogseth, a vice president at market research company Enterprise Management Associates Inc., based in Boulder, Colo. "If you have a single application that runs in one place, you can test it geographically in one place. What you have with an Amazon or Facebook, for example, is all kinds of pieces coming in from different geographies, and you can't know ahead of time where they'll be. It's definitely more complicated than running a test script on a single server-based application."
The challenge is to run tests across all the diverse components and geographies to identify problems, he said, and organizations that develop an application "typically don't have access to those types of environments. So [a company like] Keynote is giving those testers a working environment where they leverage the Internet cloud and all the vagaries and look at real networks and desktops."
New testing tools needed
Drogseth said new types of testing tools will be needed. "You can't do cloud computing with application development testing tools for a LAN or a single server. You need tools to allow you to understand the network and desktop implications and all the pieces. You need to bring the network to the developer."
"I suspect over the next five years every testing vendor will try to come to the cloud. I think we will have a new generation of testing companies," said Lounibos. "This [cloud computing] is a big market coming down road; it's just the way we'll consume services."

Software Testing Advice for Novice Testers

Novice testers have many questions about software testing and the actual work that they are going to perform. As novice testers, you should be aware of certain facts in the software testing profession. The tips below will certainly help to advance you in your software-testing career. These ‘testing truths’ are applicable to and helpful for experienced testing professionals as well. Apply each and every testing truth mentioned below in your career and you will never regret what you do.
Know Your Application
Don’t start testing without understanding the requirements. If you test without knowledge of the requirements, you will not be able to determine if a program is functioning as designed and you will not be able to tell if required functionality is missing. Clear knowledge of requirements, before starting testing, is a must for any tester.
Know Your Domain
As I have said many times, you should acquire a thorough knowledge of the domain on which you are working. Knowing the domain will help you suggest good bug solutions. Your test manager will appreciate your suggestions, if you have valid points to make. Don’t stop by only logging the bug. Provide solutions as well. Good domain knowledge will also help you to design better test cases with maximum test coverage. For more guidance on acquiring domain knowledge.
No Assumptions In Testing
Don’t start testing with the assumption that there will be no errors. As a tester, you should always be looking for errors.
Learn New Technologies
No doubt, old testing techniques still play a vital role in day-to-day testing, but try to introduce new testing procedures that work for you. Don’t rely on book knowledge. Be practical. Your new testing ideas may work amazingly for you.
You Can’t Guarantee a Bug Free Application
No matter how much testing you perform, you can’t guarantee a 100% bug free application. There are some constraints that may force your team to advance a product to the next level, knowing some common or low priority issues remain. Try to explore as many bugs as you can, but prioritize your efforts on basic and crucial functions. Put your best efforts doing good work.
Think Like An End User
This is my top piece of advice. Don’t think only like a technical guy. Think like customers or end users. Also, always think beyond your end users. Test your application as an end user. Think how an end user will be using your application. Technical plus end user thinking will assure that your application is user friendly and will pass acceptance tests easily. This was the first advice to me from my test manager when I was a novice tester.
100% Test Coverage Is Not Possible
Don’t obsess about 100% test coverage. There are millions of inputs and test combinations that are simply impossible to cover. Use techniques like boundary value analysis and equivalence partitioning testing to limit your test cases to manageable sizes.
Build Good Relations With Developers
As a tester, you communicate with many other team members, especially developers. There are many situations where tester and developer may not agree on certain points. It will take your skill to handle such situations without harming a good relationship with the developer. If you are wrong, admit it. If you are right, be diplomatic. Don’t take it personally. After all, it is a profession, and you both want a good product.
Learn From Mistakes
As a novice, you will make mistakes. If you don’t make mistakes, you are not testing hard enough! You will learn things as you get experience. Use these mistakes as your learning experience. Try not to repeat the same mistakes. It hurts when the client files any bug in an application tested by you. It is definitely an embracing situation for you and cannot be avoided. However, don’t beat yourself up. Find the root cause of the failure. Try to find out why you didn’t find that bug, and avoid the same mistake in the future. If required, change some testing procedures you are following.
Don’t Underestimate Yourself if Some of Your bugs Are Not Fixed
Some testers have assumptions that all bugs logged by them should get fixed. It is a good point to a certain level but you must be flexible according to the situation. All bugs may or may not be fixed. Management can defer bugs to fix later as some bugs have low priority, low severity or no time to fix. Over time you will also learn which bugs can be deferred until the next release.

SQL Injection – How to Test Web Applications against SQL Injection Attacks

Security testing of web applications against SQL Injection, explained with simple examples - By Inder P Singh.

Many applications use some type of a database. An application under test might have a user interface that accepts user input that is used to perform the following tasks:

1. Show the relevant stored data to the user e.g. the application checks the credentials of the user using the log in information entered by the user and exposes only the relevant functionality and data to the user

2. Save the data entered by the user to the database e.g. once the user fills up a form and submits it, the application proceeds to save the data to the database; this data is then made available to the user in the same session as well as in subsequent sessions

Some of the user inputs might be used in framing SQL statements that are then executed by the application on the database. It is possible for an application NOT to handle the inputs given by the user properly. If this is the case, a malicious user could provide unexpected inputs to the application that are then used to frame and execute SQL statements on the database. This is called SQL injection. The consequences of such an action could be alarming.

The following things might result from SQL injection:

1. The user could log in to the application as another user, even as an administrator.

2. The user could view private information belonging to other users e.g. details of other users’ profiles, their transaction details etc.

3. The user could change application configuration information and the data of the other users.

4. The user could modify the structure of the database; even delete tables in the application database.

5. The user could take control of the database server and execute commands on it at will.

Since the consequences of allowing the SQL injection technique could be severe, it follows that SQL injection should be tested during the security testing of an application. Now with an overview of the SQL injection technique, let us understand a few practical examples of SQL injection.

Important: The SQL injection problem should be tested only in the test environment.

If the application has a log in page, it is possible that the application uses a dynamic SQL such as statement below. This statement is expected to return at least a single row with the user details from the Users table as the result set when there is a row with the user name and password entered in the SQL statement.

SELECT * FROM Users WHERE User_Name = ‘” & strUserName & “‘ AND Password = ‘” & strPassword & “’;”

If the tester would enter John as the strUserName (in the textbox for user name) and Smith as strPassword (in the textbox for password), the above SQL statement would become:

SELECT * FROM Users WHERE User_Name = ‘John’ AND Password = ‘Smith’;

If the tester would enter John’– as strUserName and no strPassword, the SQL statement would become:

SELECT * FROM Users WHERE User_Name = ‘John’– AND Password = ‘Smith’;

Note that the part of the SQL statement after John is turned into a comment. If there were any user with the user name of John in the Users table, the application could allow the tester to log in as the user John. The tester could now view the private information of the user John.

What if the tester does not know the name of any existing user of the application? In such a case, the tester could try common user names like admin, administrator and sysadmin. If none of these users exist in the database, the tester could enter John’ or ‘x’=’x as strUserName and Smith’ or ‘x’=’x as strPassword. This would cause the SQL statement to become like the one below.

SELECT * FROM Users WHERE User_Name = ‘John’ or ‘x’='x’ AND Password = ‘Smith’ or ‘x’=’x’;

Since ‘x’=’x’ condition is always true, the result set would consist of all the rows in the Users table. The application could allow the tester to log in as the first user in the Users table.

Important: The tester should request the database administrator or the developer to copy the table in question before attempting the following SQL injection.

If the tester would enter John’; DROP table users_details;’—as strUserName and anything as strPassword, the SQL statement would become like the one below.

SELECT * FROM Users WHERE User_Name = ‘John’; DROP table users_details;’ –‘ AND Password = ‘Smith’;

This statement could cause the table “users_details” to be permanently deleted from the database.

Though the above examples deal with using the SQL injection technique only the log in page, the tester should test this technique on all the pages of the application that accept user input in textual format e.g. search pages, feedback pages etc.

SQL injection might be possible in applications that use SSL. Even a firewall might not be able to protect the application against the SQL injection technique.

I have tried to explain the SQL injection technique in a simple form. I would like to re-iterate that SQL injection should be tested only in a test environment and not in the development environment, production environment or any other environment. Instead of manually testing whether the application is vulnerable to SQL injection or not, one could use a web vulnerability scanner that checks for SQL injection.