Detailed testing method

3.2.1. Test configuration

Figure 3.0 Proxy based content filter test configuration

View the larger figure (New window)

Figure 3.1 Bridged content filter test configuration (Filter B only)

View the larger figure (New window)

3.2.2. Test Facility

A total of six computers (Intel P4 2.8Ghz with 512MB RAM) all running on a 1Gbps switched network were used to create a load and simulate multiple client computers running simultaneous HTTP transactions using a web performance test utility, Ziff Davis WebBench⁸ (see 3.2.1). Each of these computers ran up to ten virtual simultaneous connections therefore simulating sixty computers connected to the server.

There was also one relatively powerful (dual processor) web server configured, on the network, to receive the transactions generated by the six client computers.

A master controller system was used to coordinate testing scripts (ramp up-ramp down and duration of testing as well as collect the data feedback).

Simulating sixty virtual users connected to the network was enough to show that each of the tested filters had reached their maximum or peak throughput. This was evident in the flat line graphs that were generated in the baseline tests (Graph 3.0).

While having no pre-conceived ideas about how the filters would perform under testing the test team did know that there would be some performance impact between the test rig being filtered and the test rig being unfiltered.

The test rig that was created was capable of up to 1Gbps data rates. However, only one vendor (the appliance vendor) configured their filtering system with gigabit network support, the others all used 100Mbps networking; therefore the test rig was more than capable of testing and accurately recording the performance throughout the testing.

3.2.3. Test procedure

A baseline test was run six times with just the test rig itself (i.e. without a filter included in the network) and the results averaged to ensure that the performance results were consistent.

Once the baseline was recorded, the filtering application/appliance was connected to the network (in line with the server) to filter all traffic transparently. The test routine was then run a further six times to show the ‘filtered’ results. Once the results were recorded and checked for consistency the filter was removed from the network and the server re-tested to ensure that the baseline test results were consistent. This test was repeated with each filter selected for the test.

Once all the baseline and filtered test results were collected, the data was collated, averaged and the individual filter’s performance was measured against that of the baseline.

The Test Plan recommended that engineers for each content filter vendor be involved in the initial setup and configuration to ensure the filter was tested in its optimal configuration and that any performance issues that may have arisen could be solved via configuration changes with minimum time loss.

There were ‘legitimate’ sites which vendors attempted to tune their filters against or block before the WebBench tests were run with filtering enabled on each of the filters. The set of ‘legitimate’ sites used were a subset of the URLs used for the NetAlert joint study into the ‘Effectiveness of Internet Filtering Software Products’. The sites were divided into a number of different category groups (refer to Accuracy Testing for more information on the Category list). This created a scenario of a business/ enterprise looking to block content according to their employee acceptable usage policies.

The test scripts were written to provide 12 mixes of scenarios containing differing numbers of users and engines or processors per user. The values for mixes 1 through 12 resulted in a range from 1 through to 60 virtual users, as shown as follows:

3.2.4. Test tools

WebBench

WebBench 5.0 is a PC Magazine benchmark program that measures the performance of Web servers. WebBench was developed by VeriTest⁹.

WebBench uses PC clients to send page requests to the server which maintains a static web site installed on the server during the installation of the benchmark.

When a WebBench test suite is started, the clients issue a combination of secure and unsecured requests for static and dynamic data. These clients simulate web browsers. When the server replies to a client request, the client records information such as how long the server took and how much data it returned and then sends a new request. When the test ends, WebBench calculates two overall server scores (requests per second and throughput in bytes per second) as well as individual client scores.

WebBench can be used to:

• Measure the performance of different web server software packages by running WebBench against them on the same server hardware,
• Gauge how well different hardware systems perform as web servers by running a given web server package and WebBench on those systems.

WebBench provides a number of templates for test suites however the sample test suite had to be edited to reflect the web server environment created.

To run WebBench, a minimum of three computers were required on the network:

1. A web server - This computer runs the web server software. The web server receives the requests from the clients and sends responses back to the clients. Based on the request that the clients send, these responses can be the HTML, .GIF, or executable test files that were placed in the document root or data produced when the server executed the dynamic executable.
2. A controller - This is a computer running Windows NT, Windows XP, or Windows 2000 and the WebBench controller program. This computer sets up, starts, monitors, and stops the WebBench tests. When the tests end, the clients send their results to the controller, where the results can be viewed using Excel workbooks. The controller, unlike the clients, does not affect the server’s overall score.
3. Clients - These are the computers running the WebBench client program. WebBench supports Windows 95/98, Windows NT, Windows XP, or Windows 2000 clients. The clients execute the WebBench tests and send requests to the server.

The following list contains a short summary of WebBench's major points:

• WebBench measures the performance of web server software and hardware,
• Clients issue requests to the server using HTTP GET requests,
• WebBench requires a server networked to a computer running the controller program and one or more computers each running the client program. WebBench provides both a controller and client program; it does not run a server application. However, WebBench does require that the server be running a third-party web server program,
• WebBench provides static test suites which access only HTML, GIF, and a few sample executable (.EX) files. They do not run any programs on the server,
• The controller must be running Microsoft Windows NT, Windows XP, or Windows 2000 so that it can talk to all the clients that might exist on multiple network segments. The clients, however, can be running Windows 95/98, Windows NT, Windows XP, or Windows 2000.

⁸ WebBench: WebBench is a PC Magazine benchmark program that measures the performance of Web servers by using virtual users to send page requests to the server.

⁹ http://www.veritest.com/benchmarks/webbench/home.asp

Back to Introduction to performance testing | Table of contents | Forward to Test results