Open source QA tool for automated Web application testing

Q- Could you recommend a quality assurance (QA) tool for automated regression/functional testing (open source or free tools are preferred) for testing a Web application that contains a lot of JavaScript for opening pop-up windows, redirects, etc.? I'm using HTTPUnit and Selenium but these tools are not handling pop-up windows and redirects well. Thanks

A- My preferred automation solution for Web applications is a combination of *Unit and Wati* -- for instance, JUnit plus Watij. I like Watij over Selenium RC simply because it seems a little more object-oriented than Selenium. But this is totally personal, I've used both tools successfully.

Handling pop-ups and other interactive display changes can be a challenge, regardless of the tool. You might consider a couple of approaches. First, be active in the tool's user group, seeking solutions. There are several groups available on the Internet -- just pick one or two. Be polite: post your question once, rather than blasting across multiple groups. If you post to the Selenium user's group, you will definitely encounter other testers who have faced similar challenges in the past -- they'll probably have tried-and-true solutions for you. Secondly, pay close attention to your implementation. If your Web app uses a lot of rich Internet applications (RIAs), your might get away with the use of divs rather than handling each window (in RIA, developers can "pop up" windows which are, in fact, just hidden divs being exposed). Experiment with different programming solutions and see which is more reliable. There is generally more than one way to accomplish what you're trying to do. You're looking for the way which is 1) feasible, 2) most reliable and 3) most performant (in that order).

The third approach may be the most challenging and, in the short term, costly. However, if you're working on an ongoing, long-term project it will have payoff. If the current implementation is not very testable, propose alternative implementations to the development team. For instance, if the current project creates multiple pop-up windows (rather than using Ajax to expose elements), ask that the team take time to change this, implementing a solution which you can test more reliably and in a shorter amount of time. You need to be very, very detailed in your reasoning -- you will need to include schedule and cost savings. Point to gains down the road when your automated regression and functional tests run more reliably. By implementing testability, you will be addressing what Agile teams call "technical debt." You take a short-term hit on schedule, with the outcome being a long-term improvement in effectiveness and efficiency.

Will penetration testing be replaced by preventative tools?

by michaeldkelly

I recently read the article "Penetration Testing: Dead in 2009" by Bill Brenner. In the article Mr. Brenner follows a small debate around the idea that over time penetration testing will be largely replaced by preventative checks.

The debate opens with some quotes from Brian Chess from Fortify Software. Fortify creates code analysis tools that scan for security concerns and adherence to good secure coding practices. That potential bias aside, I suspect that Mr. Chess' statement — that "Customers are clamoring more for preventative tools than tools that simply find the weaknesses that already exist [...]. They want to prevent holes from opening in the first place" — is absolutely true. I know I clamor for those tools, and I'm just a lowly test manager.

I'm a big fan of the work companies like Fortify, IBM and HP are doing in this space. If my project team can find a potential issue before we deploy the code, I'm all for it. It can save us time and helps us focus on different and potentially higher-value risks. However, I've yet to see a tool that can deal with the complexity of a deployment environment (setup, configuration, code, etc…) and while I'm a big believer in doing everything you can up front (design, review, runtime-analysis, etc.), I believe there will always be a roll for a skilled manual investigation of what gets deployed.

Testing (penetration or other) is about applying skill and judgment to uncover quality-related information about the product. That's not just code — it's more than that. Your typical penetration tester today covers more than today's automated tools can cover. While there are different tools to test various components (some that focus on code, some that focus on the network, etc.), and they should absolutely be used, those tools will never be able to uncover all the potential issues with a system. And, what's sometimes worse, is they can lead to a false sense of security.