Skip to content

CMS performance testing. Part I

February 10, 2011

Hi.
This post opens a series of articles describing popular CMS performance research. Original report was about 35 pages so here I’ll try to make it a bit shorter and less formal.

1. Introduction

This document defines the necessary resources and describes the approach, methodology and scenario used by the testing team to carry out the 3 Content Management Systems (CMS) performance testing. Below we tested Kentico CMS, DotNetNuke CMS, Sitefinity CMS.
CMS applications Web-applications based on SQL 2003 server as backend. CMS’s realizes a number of use cases. We just took to review CMS Blog module.
Performance testing load types:

  • Ramp-up run. The users in a ramp-up run are staggered (adding a few new users every x seconds). The ramp-up run does not allow for accurate and reproducible averages because the load on the system is constantly changing as the users are being added a few at a time. Ramp-up tests are valuable for finding the ballpark in which you think you later want to run flat runs.
  • Flat run. To have truly reproducible results, the system should be put under a high load with no variability. To accomplish this, the virtual users hitting the server should have 0 seconds of think-time between requests. The only way to accurately get performance counters though is to load all the users at once, and then run them for a predetermined amount of time.

2. Scope of work

The scope of work includes performance testing of 3 Content Management Systems. It is Kentico CMS, Sitefinity CMS, dotNetNuke CMS.

3. Objectives

  • Measure end-to-end response time for testing application.
  • Measure hardware resource usage on all application and database servers.
  • Identify bottlenecks.
  • Define possible break points and determine the maximum load the server can handle before degradation in response time occurs.
  • Response time per page load must meet the service level.
  • Compare CMS’s and found better one.

4. Resources

4.1. Tools

HP Load Runner Software 9.10
SQL Server Profiler 9.00.1399.06

4.2. Testing Scenario

  • Blog page. Open CMS Blog page to see user posts.
  • Blog post comment. Open random post comments.

4.3 Test Environment

Application & Database Server
Resource CPU RAM Software configuration
EPRUSARSD0021 OS: Windows Server 2003 Standard Edition SP2 

Database server: MS SQL Server  2003 SP2

EPRUSARSD0047 2x Intel 2,8 GHz 2 Gb OS: Windows Server 2003 Standard Edition SP2 

App. server: IIS 6.0

Test Control Console & Measurement Station
Resource CPU RAM Software configuration
EPRUSARW0129 2xIntel  3.0 GHz 1 Gb OS: Windows XP SP2, HP Load Runner, SQL Server Profiler
Advertisements
4 Comments
  1. Hello very nice web site!! Man .. Excellent .. Superb .. I’ll bookmark your web site and take the feeds also?I am glad to find a lot of useful info right here within the submit, we’d like develop more techniques in this regard, thank you for sharing. . . . . .

Trackbacks & Pingbacks

  1. CMS performance testing. Part III (Kentico) « QA Questions
  2. CMS performance testing. Part II (DotNetNuke) « QA Questions
  3. CMS performance testing. Part IV (Sitefinity ) « QA Questions

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: