This is a quick shoutout on the release of version 0.4 of SuperBenchmarker, a Web and/or Web API performance benchmarking command line tool for Windows.
You might have heard about and used Apache Benchmark (ab.exe) in the past which is a very useful tool but on Windows it is very limited (e.g cannot make POST, PUT, etc requests and only supports GET). SuperBenchmarker (sb.exe) supports PUT, DELETE, POST or any arbitrary method and allows you to parameterise the URL and headers using a data file, a .NET DLL plugin and the new feature is the randomisation feature which removes the need for any setup when all needed is random data.
Getting started
The best way to get SuperBenchmarker is to use awesome Chocolatey which is Windows' equivalent of apt-get tool on Linux.To get Chocolatey, just run this command in your Powershell console (in Administrative mode):
iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))
c:\> cinst SuperBenchmarker
c:\> sb -u http://google.comNote: if you are using Visual Studio's command line shell, you cannot use ampersand character (&) and you have to escape it using hat (^).
Using SuperBenchmarker
Normally you would define total number of requests and concurrency:c:\> sb -u http://google.com -c 10 -n 2000
Status 503: 1768
Status 200: 232
TPS: 98 (requests/second)
Max: 11271.1890515802ms
Min: 3.15724613377097ms
Avg: 497.181240820346ms
50% below 34.0499543844287ms
60% below 41.8178295863705ms
70% below 48.7612961478952ms
80% below 87.4385213898198ms
90% below 490.947293319644ms
In case you need to dig deeper, a log file gets created in the current directory with the name
run.log which you can change using -l parameter:c:\> sb -u http://google.com -c 10 -n 2000 -l c:\temp\mylog.txt
Sometimes when running a test for the first time, something might not have been quite right in which case you can make a dry run/debug using -d parameter that makes a single request and the body of the response will be shown at the end. If you want to see the headers as well, use -h parameter.
c:\> sb -u http://google.com -c 10 -n 2000 -d -h
Supplying request headers or a payload for POST, PUT and DELETE
In order to pass your tailored request headers, a template file needs to be defined which is basically the HTTP request template (minus the first line defining verb and URL and version):c:\> sb -u http://google.com -t template.txt
User-Agent: SuperBenchmarker
MyCustomHeader: foo-bar;baz=biz
For using POST, PUT and DELETE we need to supply the verb parameter:
c:\> sb -u http://google.com -v POST
User-Agent: WhateverValueIWant
Content-Type: x-www-formurlencoded
name=value&age=25
Parameterising your requests
Basically you can parameterise your requests using a CSV file containing values, your plugin DLL or by specifying randomisation.You would define parameters in URL and headers (payload not yet supported but coming soon in 0.5) using SuperBenchmarker's syntax:
{{{MyParameter}}}c:\> sb -u "http://myserver.com/api/customer?customerid={{{customerId}}}^&ignore=false"
customerId
123,
245,
c:\> sb -u "http://myserver.com/api/customer?customerid={{{customerId}}}&ignore=false" -f c:\mypath\values.csv
c:\> sb -u "http://myserver.com/api/customer?customerid={{{customerId}}}&ignore=false" -p c:\mypath\myplugin.dll
IValueProvider interface which has a single method:public interface IValueProvider
{
IDictionary<string, object> GetValues(int index);
}
Now we have a new feature that in most cases alleviates the need for CSV file or plugin and that is the ability to setup random value provider in the definition of the parameter itself:
c:\> sb -u "http://myserver.com/api/customer?customerid={{{customerId:RAND_INTEGER:[1000:2000]}}}&ignore=false"
Possible value types are:
- String: using RAND_STRING. Will output random words
- Date: using RAND_DATE (accepts range)
- DateTime: using RAND_DATETIME (accepts range)
- DateTimeOffset: using RAND_DATETIMEOFFSET which outputs ISO dates (accepts range)
- Double: using RAND_DOUBLE (accepts range)
- Name: using RAND_NAME. Will output random names
