Skip to content

CMS performance testing. Part I

February 10, 2011

This post opens a series of articles describing popular CMS performance research. Original report was about 35 pages so here I’ll try to make it a bit shorter and less formal.

1. Introduction

This document defines the necessary resources and describes the approach, methodology and scenario used by the testing team to carry out the 3 Content Management Systems (CMS) performance testing. Below we tested Kentico CMS, DotNetNuke CMS, Sitefinity CMS.
CMS applications Web-applications based on SQL 2003 server as backend. CMS’s realizes a number of use cases. We just took to review CMS Blog module.
Performance testing load types:

  • Ramp-up run. The users in a ramp-up run are staggered (adding a few new users every x seconds). The ramp-up run does not allow for accurate and reproducible averages because the load on the system is constantly changing as the users are being added a few at a time. Ramp-up tests are valuable for finding the ballpark in which you think you later want to run flat runs.
  • Flat run. To have truly reproducible results, the system should be put under a high load with no variability. To accomplish this, the virtual users hitting the server should have 0 seconds of think-time between requests. The only way to accurately get performance counters though is to load all the users at once, and then run them for a predetermined amount of time.

2. Scope of work

The scope of work includes performance testing of 3 Content Management Systems. It is Kentico CMS, Sitefinity CMS, dotNetNuke CMS.

3. Objectives

  • Measure end-to-end response time for testing application.
  • Measure hardware resource usage on all application and database servers.
  • Identify bottlenecks.
  • Define possible break points and determine the maximum load the server can handle before degradation in response time occurs.
  • Response time per page load must meet the service level.
  • Compare CMS’s and found better one.

4. Resources

4.1. Tools

HP Load Runner Software 9.10
SQL Server Profiler 9.00.1399.06

4.2. Testing Scenario

  • Blog page. Open CMS Blog page to see user posts.
  • Blog post comment. Open random post comments.

4.3 Test Environment

Application & Database Server
Resource CPU RAM Software configuration
EPRUSARSD0021 OS: Windows Server 2003 Standard Edition SP2 

Database server: MS SQL Server  2003 SP2

EPRUSARSD0047 2x Intel 2,8 GHz 2 Gb OS: Windows Server 2003 Standard Edition SP2 

App. server: IIS 6.0

Test Control Console & Measurement Station
Resource CPU RAM Software configuration
EPRUSARW0129 2xIntel  3.0 GHz 1 Gb OS: Windows XP SP2, HP Load Runner, SQL Server Profiler
  1. Hello very nice web site!! Man .. Excellent .. Superb .. I’ll bookmark your web site and take the feeds also?I am glad to find a lot of useful info right here within the submit, we’d like develop more techniques in this regard, thank you for sharing. . . . . .

Trackbacks & Pingbacks

  1. CMS performance testing. Part III (Kentico) « QA Questions
  2. CMS performance testing. Part II (DotNetNuke) « QA Questions
  3. CMS performance testing. Part IV (Sitefinity ) « QA Questions

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: