Unit Tests

Just to be sure that I write this up the other basic part of my project methodology is unit testing.  Each set of tests I run are run on the same day, one after the other.  Exact replicas (in terms of HTML output) are being compared and the code behind is as similar as possible.

Unit testing implies targeting a small subsection of a system and testing it alone.  I am carrying out unit testing because if I weren’t doing that it would be impossible to separate the result sets.  It is assumed to treat each page of a website as being completely independent of all other pages.  In reality the pages could in fact be executed each on a separate server so this should be a valid assumption.

Sometimes it will be necessary to draw inferences between batches of unit tests.  When this is done I do some simple, undocumented testing to ensure the results of the older test still ring true.  There should be no significant difference because my server configuration is static and my systems are isolated from external networks.

Which way round?

I can test by timing a specified amount of time (i.e. 60 seconds) and counting the number of requests fulfilled in that period of time.  I can also do the reverse by setting a specific number of requests and timing how long it takes to fulfil them.

Both are valid but the second is easier to program and easier to control.  For now that is what I will do.

Initialising multiple client apps

In order to test my prototypes from multiple computers it is necessary to start the test application on multiple computers at the same time.  I will also need to synchronise the task that the test app is performing because the app will be multi purpose.

To start the app at the same time on multiple computers I have several options.  I could use psexec or runas to remotely launch the task on multiple machines.  After experimenting with this option I found it to be difficult to control.  Alternatively I can use some kind of timer to initiate each at a specified time.  I think that the best way I can do this is by using windows scheduled tasks – manually configured on each client.

Requests Per Second

A useful metric in determining the usefulness of a language is the request per second count.  Requests per second should hold steady no matter the load.  I suspect however that beyond a certain load the number of requests per second will fall.  This needs to be taken into account when designing the test schedule.

Testing Trials

While I obviously haven’t completed any database applications yet I ran some preliminary tests against a basic PHP file just to get a feel for what I can expect later in the project.  Initially I tested under XP and found that Google could respond faster that my local IIS.  Moving to Windows 7 and IIS7.5 I found a great improvement with local responses averaging 3ms while Google takes around 300ms.  For a distance trial I ran a test on the BBC site – it takes around a second to respond to each request.

The little bit of testing that I have now done has helped me to gauge the parameters that I will use later in the project.  From what I have seen so far I don’t think I will need more that one or two PC’s to simulate a very heavy server workload.

A Basic Testing Tool

After a cursory search I determined that it is going to be far easier for me to develop a simple testing tool than to find one that suits my purpose.  Yesterday I did just that and produced a small program which lets you hit a specified web address with a specified number of requests.  To simulate multiple users I implemented multi threading.  Currently the program outputs results in milliseconds to a file called output.txt.