Custom Search
What is difference between Performance Testing, Load Testing and Stress Testing?
Q. What is difference between Performance Testing, Load Testing and Stress Testing?1) Performance Testing:Performance testing is the testing, which is performed, to ascertain how the components of a system are performing, given a particular situation. Resource usage, scalability and reliability of the product are also validated under this testing. This testing is the subset of performance engineering, which is focused on addressing performance issues in the design and architecture of software product. Performance Testing Goal: The primary goal of performance testing includes establishing the benchmark behaviour of the system. There are a number of industry-defined benchmarks, which should be met during performance testing. Performance testing does not aim to find defects in the application, it address a little more critical task of testing the benchmark and standard set for the application. Accuracy and close monitoring of the performance and results of the test is the primary characteristic of performance testing. Example: For instance, you can test the application network performance on Connection Speed vs. Latency chart. Latency is the time difference between the data to reach from source to destination. Thus, a 70kb page would take not more than 15 seconds to load for a worst connection of 28.8kbps modem (latency=1000 milliseconds), while the page of same size would appear within 5 seconds, for the average connection of 256kbps DSL (latency=100 milliseconds). 1.5mbps T1 connection (latency=50 milliseconds) would have the performance benchmark set within 1 second to achieve this target. For example, the time difference between the generation of request and acknowledgement of response should be in the range of x ms (milliseconds) and y ms, where x and y are standard digits. A successful performance testing should project most of the performance issues, which could be related to database, network, software, hardware etc… 2) Load Testing:Load testing is meant to test the system by constantly and steadily increasing the load on the system till the time it reaches the threshold limit. It is the simplest form of testing which employs the use of automation tools such as LoadRunner or any other good tools, which are available. Load testing is also famous by the names like volume testing and endurance testing. The sole purpose of load testing is to assign the system the largest job it could possible handle to test the endurance and monitoring the results. An interesting fact is that sometimes the system is fed with empty task to determine the behaviour of system in zero-load situation. Load Testing Goal: The goals of load testing are to expose the defects in application related to buffer overflow, memory leaks and mismanagement of memory. Another target of load testing is to determine the upper limit of all the components of application like database, hardware and network etc… so that it could manage the anticipated load in future. The issues that would eventually come out as the result of load testing may include load balancing problems, bandwidth issues, capacity of the existing system etc… Example: For example, to check the email functionality of an application, it could be flooded with 1000 users at a time. Now, 1000 users can fire the email transactions (read, send, delete, forward, reply) in many different ways. If we take one transaction per user per hour, then it would be 1000 transactions per hour. By simulating 10 transactions/user, we could load test the email server by occupying it with 10000 transactions/hour.
3) Stress testingUnder stress testing, various activities to overload the existing resources with excess jobs are carried out in an attempt to break the system down. Negative testing, which includes removal of the components from the system is also done as a part of stress testing. Also known as fatigue testing, this testing should capture the stability of the application by testing it beyond its bandwidth capacity. The purpose behind stress testing is to ascertain the failure of system and to monitor how the system recovers back gracefully. The challenge here is to set up a controlled environment before launching the test so that you could precisely capture the behaviour of system repeatedly, under the most unpredictable scenarios. Stress Testing Goal: The goal of the stress testing is to analyse post-crash reports to define the behaviour of application after failure. The biggest issue is to ensure that the system does not compromise with the security of sensitive data after the failure. In a successful stress testing, the system will come back to normality along with all its components, after even the most terrible break down. Example: As an example, a word processor like Writer1.1.0 by OpenOffice.org is utilized in development of letters, presentations, spread sheets etc… Purpose of our stress testing is to load it with the excess of characters. To do this, we will repeatedly paste a line of data, till it reaches its threshold limit of handling large volume of text. As soon as the character size reaches 65,535 characters, it would simply refuse to accept more data. The result of stress testing on Writer 1.1.0 produces the result that, it does not crash under the stress and that it handle the situation gracefully, which make sure that application is working correctly even under rigorous stress conditions. |
Database Testing – Practical Tips and Insight on How to Test Database
| Database Testing – Practical Tips and Insight on How to Test Database Database is one of the inevitable parts of a software application these days. It does not matter at all whether it is web or desktop, client server or peer to peer, enterprise or individual business, database is working at backend. Similarly, whether it is healthcare of finance, leasing or retail, mailing application or controlling spaceship, behind the scene a database is always in action. Moreover, as the complexity of application increases the need of stronger and secure database emerges. In the same way, for the applications with high frequency of transactions (e.g. banking or finance application), necessity of fully featured DB Tool is coupled. Currently, several database tools are available in the market e.g. MS-Access2010, MS SQL Server 2008 r2, Oracle 10g, Oracle Financial, MySQL, PostgreSQL, DB2 etc. All of these vary in cost, robustness, features and security. Each of these DBs possesses its own benefits and drawbacks. One thing is certain; a business application must be built using one of these or other DB Tools. Before I start digging into the topic, let me comprehend the foreword. When the application is under execution, the end user mainly utilizes the 'CRUD' operations facilitated by the DB Tool. C: Create – When user 'Save' any new transaction, 'Create' operation is performed. It does not matter at all, which DB is used and how the operation is preformed. End user has no concern if any join or sub-query, trigger or stored-procedure, query or function was used to do what he wanted. But, the interesting thing is that all DB operations performed by user, from UI of any application, is one of the above four, acronym as CRUD. As a database tester one should be focusing on following DB testing activities: What to test in database testing:1) Ensure data mapping:Make sure that the mapping between different forms or screens of AUT and the Relations of its DB is not only accurate but is also according to design documents. For all CRUD operations, verify that respective tables and records are updated when user clicks 'Save', 'Update', 'Search' or 'Delete' from GUI of the application. 2) Ensure ACID Properties of Transactions:ACID properties of DB Transactions refer to the 'Atomicity', 'Consistency', 'Isolation' and 'Durability'. Proper testing of these four properties must be done during the DB testing activity. This area demands more rigorous, thorough and keen testing when the database is distributed. 3) Ensure Data Integrity:Consider that different modules (i.e. screens or forms) of application use the same data in different ways and perform all the CRUD operations on the data. In that case, make it sure that the latest state of data is reflected everywhere. System must show the updated and most recent values or the status of such shared data on all the forms and screens. This is called the Data Integrity. 4) Ensure Accuracy of implemented Business Rules:Today, databases are not meant only to store the records. In fact, DBs have been evolved into extremely powerful tools that provide ample support to the developers in order to implement the business logic at DB level. Some simple examples of powerful features of DBs are 'Referential Integrity', relational constrains, triggers and stored procedures. So, using these and many other features offered by DBs, developers implement the business logic on DB level. Tester must ensure that the implemented business logic is correct and works accurately. Above points describe the four most important 'What Tos' of database testing. Now, I will put some light on 'How Tos' of DB Testing. But, first of all I feel it better to explicitly mention an important point. DB Testing is a business critical task, and it should never be assigned to a fresh or inexperienced resource without proper training. How To Test Database:1. Create your own QueriesIn order to test the DB properly and accurately, first of all a tester should have very good knowledge of SQL and specially DML (Data Manipulation Language) statements. Secondly, the tester should acquire good understanding of internal DB structure of AUT. If these two pre-requisites are fulfilled, then the tester is ready to test DB with complete confidence. (S)He will perform any CRUD operation from the UI of application, and will verify the result using SQL query. This is the best and robust way of DB testing especially for applications with small to medium level of complexity. Yet, the two pre-requisites described are necessary. Otherwise, this way of DB testing cannot be adopted by the tester. Moreover, if the application is very complex then it may be hard or impossible for the tester to write all of the needed SQL queries himself or herself. However, for some complex queries, tester may get help from the developer too. I always recommend this method for the testers because it does not only give them the confidence on the testing they have performed but, also enhance their SQL skill. 2. Observe data table by tableIf the tester is not good in SQL, then he or she may verify the result of CRUD operation, performed using GUI of the application, by viewing the tables (relations) of DB. Yet, this way may be a bit tedious and cumbersome especially when the DB and tables have large amount of data. Similarly, this way of DB testing may be extremely difficult for tester if the data to be verified belongs to multiple tables. This way of DB testing also requires at least good knowledge of Table structure of AUT. 3. Get query from developerThis is the simplest way for the tester to test the DB. Perform any CRUD operation from GUI and verify its impacts by executing the respective SQL query obtained from the developer. It requires neither good knowledge of SQL nor good knowledge of application's DB structure. So, this method seems easy and good choice for testing DB. But, its drawback is havoc. What if the query given by the developer is semantically wrong or does not fulfill the user's requirement correctly? In this situation, the client will report the issue and will demand its fix as the best case. While, the worst case is that client may refuse to accept the application.
Conclusion:Database is the core and critical part of almost every software application. So DB testing of an application demands keen attention, good SQL skills, proper knowledge of DB structure of AUT and proper training. In order to have the confident test report of this activity, this task should be assigned to a resource with all the four qualities stated above. Otherwise, shipment time surprises, bugs identification by the client, improper or unintended application's behavior or even wrong outputs of business critical tasks are more likely to be observed. Get this task done by most suitable resources and pay it the well-deserved attention.
|
Failover and Recovery Testing
Failover and Recovery Testing testing verifies product in terms of ability to confront and successfully recover from possible failures, arising from software bugs, hardware failure or communication problems (eg network failure). The objective of this test is to check the system restore (or duplicate the main functional systems), which, in the event of failure, ensure the safety and integrity of the data product being tested.
Testing for failure and recovery is very important for systems operating on the principle of "24×7". If you create a product that will work, such as the Internet, without this kind of test you just can not do. Because Every minute of downtime or data loss in case of equipment failure can cost you money, losing customers and reputation in the market.
The technique of this test is simulating various fault conditions and subsequent study and evaluation of the reaction of protective systems. During these inspections it turns out, was it achieved the desired degree of recovery after the crash occurred.
For clarity, we consider some variants of this test, and general methods for their implementation. The object of testing in most cases are highly probable operational problems, such as:
- Denial of electricity on a computer server
- Denial of electricity on the client computer
- Incomplete data processing cycle (interruption of data filters, interrupt synchronization).
- Announcement or introduction into arrays of data are not available or erroneous elements.
- Refusal data carriers.
These situations can be played as soon reached a point in development when all the system restore or duplication are ready to perform its functions. Technically, to implement the tests in the following ways:
- Simulate the sudden failure of electricity on the computer (disconnect the computer).
- Simulate the loss of communication with the network (turn off the power cord, disconnect the network device)
- Simulate the failure of carriers (disconnect the external storage medium)
- Simulate the situation in the presence of invalid data (a special test kit or a database).
Upon reaching the appropriate conditions of failure and performance-based recovery systems, we can estimate the product in terms of testing to failure. In all the cases listed above, upon completion of the recovery to be achieved some desired state of the data product:
- Data loss or corruption within an acceptable range.
- Report or reporting system, indicating the processes or transactions that were not completed because of errors.
It is worth noting that testing for failure and recovery - is very product-specific testing. Development of test scripts shall be subject to all the features of the system under test. Taking into account the rather harsh methods of influence, we should also evaluate the usefulness of this type of testing for a particular software product.
Design-Based Test Case Design an Effective Software Testing Technique
Software design errors and faults can be discovered and software designs validated by two techniques like:
1) Requirements-based test case design being the primary technique
2) Another technique being the early design-based test case design.
In design-based test case design the information for deriving them is taken from the software design documentation.
Design-based test cases focus on the data and process paths within the software structures. Internal interfaces, complex paths or processes, worst-case scenarios, design risks and weak areas, etc. are all explored by constructing specialized test cases and analyzing how the design should handle them and whether it deals with them properly. In software testing effort, requirements-based and design-based test cases provide specific examples that can be used in design reviews or walkthroughs. Together they provide a comprehensive and rich resource for design based software testing.
Design Testing Metrics:
Increasingly, formal design reviews are adopting metrics as a means of quantifying test results and clearly defining expected results.
The metrics (measures that are presumed to predict an aspect of software quality) vary greatly. Some are developed from scored questionnaires or checklists. For example, one group of questions may relate to design integrity and system security.
Typical Integrity Questions can be like the following
Q.1: Are security features controlled from independent modules?
Q.2: Is an audit trail of accesses maintained for review or investigation?
Q.3: Are passwords and access keywords blanked out?
Q.4: Does it require changes in multiple programs to defeat the access security?
Each reviewer would answer these questions, and their answers would be graded or scored. Over time, minimum scores are established and used as pass/ fail criteria for the integrity metric. Designs that score below the minimum are reworked and subjected to additional review testing before being accepted.
Another example of a metric-based design test that can be used effectively is a test for system maintainability. An important consideration in evaluating the quality of any proposed design is the ease with which it can be maintained or changed once the system becomes operational. Maintainability is largely a function of design. Problems or deficiencies that produce poor maintainability must be discovered during design reviews; it is usually too late to do anything to correct them further along in the cycle of software testing.
To test the maintainability we develop a list of likely or plausible requirements changes (perhaps in conjunction with the requirements review). Essentially, we want to describe in advance what about the system we perceive is most apt to be changed in the future. During the design review a sample of these likely changes is selected at random and the system alterations that would be required are walked through by the reviewers to establish estimates for how many programs and files or data elements would be affected and the number of program statements that would have to be added and changed. Metric values for these estimates are again set on the basis of past experience. Passing the test might require that 80 percent of the changes be accomplished by changes to single programs and that the average predicted effort for a change be less than one man-week. Designs that score below these criteria based on the simulated changes are returned, reworked, and re-subjected to the maintainability test before being accepted. This is just one example of an entire class of metrics that can be built around what-if questions and used to test any quality attribute of interest while the system is still being designed.
Design for Testing:
In addition to the testing activities we perform to review and test the design, another important consideration is the features in the design that simplify or support testing. Part of good engineering is building something in a way that simplifies the task of verifying that it is built properly. Hardware engineers routinely provide test points or probes to permit electronic circuits to be tested at intermediate stages. In the same way, complex software must be designed with "windows" or hooks to permit the testers to "see" how it operates and verify correct behavior.
Providing such windows and reviewing designs to ensure their testability is part of the overall goal of designing for testability. With complex designs, testing is simply not effective unless the software has been designed for testing. Testers must consider how they are going to test the system and what they will require early enough in the design process so that the test requirements can be met.
Design Testing Tools and Aids:
Automated tools and aids to support design testing play an important role in a number of organizations. As in requirements testing, our major testing technique is the formal review; however there is a greater opportunity to use automated aids in support of design reviews.
Software testing tools in common use include design simulators (such as the data base and response time simulators; system charters that diagram or represent system logic; consistency checkers that analyze decision tables representing design logic and determine if they are complete and consistent; and data base dictionaries and analyzers that record data element definitions and analyze each usage of data and report on where it is used and whether the routine inputs, uses, modifies, or outputs the data element.
None of these software testing tools performs direct testing. Instead, they serve to organize and index information about the system being designed so that it may be reviewed more thoroughly and effectively. In the case of the simulators, they permit simplified models to be represented and experimentation to take place, which may be especially helpful in answering the question of whether the design solution is the right choice. All the tools assist in determining that the design is complete and will fulfill the stated requirements.
Documentation Errors
The purpose of error reporting is fixing it. About how to describe the error, what it consists of a description of the error and how it might look like the example tells this story.
So you found a bug. Not shelving them to start to write a bug report (not to procrastinate, then you can forget to write a report, to forget in what place was a mistake to miss part or even misquote the situation).
The first step is desirable to calm down and not make any sudden movements, not to press extra buttons, etc. We must remember the sequence of actions that have been made and try to reproduce the situation. Better to do it in a new browser window (if it is a web-application). Come and write data input / command buttons are pressed, in any menu jump to, what kind of reaction system to these actions, what error message is displayed.
Now we need to write their own actions. Record should be brief, but clear and understandable. Find a middle ground. If you write a memoir, the programmer will not read them or think that the error is very difficult and would defer until later, and ultrashort report no one will understand. As a consequence, error correction (bug) will hang in the air and sent back to you marked "not play" or ask clarifications, thereby simply and your spending and your time. Also, do not enter into one report more than one error. Motive is the same.
The report is written not only for ourselves but for others. So, it should be written so as to understand everything, but had no idea that you would like to say, do not ask again. Ask yourself a question: whether to repeat your actions the person who first sees the product?
If possible, try different options to express exactly the problem.
It is also desirable to avoid jargon or expressions which may be difficult to understand others.
In no case does not need to pass an oral report bugs, write an e-mail, icq, etc.! In most cases, forget about them, not treat them seriously and, if not corrected, especially in this case will blame you. You need this? All errors must be observed and described and have its own unique number. Then, for uncorrected errors will be the responsibility on the programmer.
These records will need another bug testers to work with you, the managers for them to see that you work and work productively, testers, who will come for you, as well as for writing reports.
Error Description
We now turn to the description of the error. Depending on what the company bagtrekingovaya system (accounting system errors), there will be different input fields.
At the beginning of opening a new bug report. It is possible that you will see a lot of lines for the filling, but it is possible that they do not all have to fill. It is better to consult with other testers, a manager or head of the group testing. But most likely have to fill out the following fields:
- Priority (as a serious mistake and a speed of execution requires. Must be corrected quickly or you can wait)
- Designate (who will deal with an error)
- Class (this is what kind of error-serious, minor, typo …)
Header Error
The headline should concisely and fully describe the problem. We spend a lot of time leafing through the bugs database and browsing the headlines errors. Much time can be saved if the headlines are clear and not have to open the error description to understand what was meant.
Problem Description
Describe the problem better by using "arrows". With them, the text of the report is discarded many unnecessary words that interfere with understanding the essence.
Example: I opened www.aaa.ru -> introduced in the word bbb ccc -> clicked ddd -> get errors: ddd
An example from the life
Title: The problem with the menu "forgot password"
Problem Description: Go to the login page -> click "Forgot Password" - "in the" Personal Account "enter 2389 -> in the« e-mail »enter test@test.com -> System says:" Error sending message . (# 1) "
If necessary, the data in the operating environment, configuration, logs. Is there a dependence on the configuration, installation, condition, options, settings, version, etc.
Attachments
To make the report more detailed and vivid can and must resort to:
- links
- screenshot
- video recording
Link
Well here everything is clear. Popped up an error - is taken link to this page, and inserted into the report. It is desirable also with screenshots. (Assuming that the tested Web application - approx. Editor)
Screenshots
A very useful thing to visualize the problem. Make a screenshot of the problematic area. (The simplest thing - it's on the keyboard to find the button Print Screen, then press it to open the program from Paint (if we are in the operating system of family Windows - approx. Editor), which is automatically installed in Windows and in her press Ctrl-V, then cut out unnecessary , store (preferably in the format JPG))
Although there is a more professional program that are more adapted to this kind of action and have a lot of very useful features, such as SnagIt, HyperSnap, HardCopy, RoboScreenCapture, FullShot 9, HyperSnap-DX 5, TNT 2. Screenshot want to attach to the bug report.
Videos
If the error difficult to describe, it is the most appropriate method. Program: SnagIt, CamStudio.
See Also
Subscribe to:
Posts (Atom)
