Benchmarking process / IO / latency
links:
apt-get install contest dbench stress tiobench latencytop
Unix Benchmarking
see: http://www.webhostingtalk.com/showthread.php?s=&threadid=308055
Download: UnixBench v4.1.0 - WHT Variant
# gunzip -dvc unixbench-4.1.0-wht.tar.gz | tar xvf - # cd unixbench-4.1.0-wht # make # ./Run
Please download and run it on your dedicated or VPS. Then post your result here for discussion. Thanks!
IO test
- script available here, mirrored copy is down below.
/--(a@tbox)-(Tue Oct 21 15:28:24 CEST 2008)-(0)-- \--(#!:~/src)-- python iotest-2008-10-15 /dev/sda3 /dev/sda3, 23 GB, 512B blocks: 66.1 IOs/s, 33 kB/s /dev/sda3, 23 GB, 1024B blocks: 65.7 IOs/s, 65 kB/s /dev/sda3, 23 GB, 2 kB blocks: 64.9 IOs/s, 129 kB/s ...
f iotest-2008-10-15.py
#!/usr/bin/env python
#
# Copyright (c) 2008 Benjamin Schweizer. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the
# distribution.
# * Neither the name of Benjamin Schweizer nor the names of its
# contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
# IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
#
# Abstract
# ~~~~~~~~
# Benchmark disk IOs
#
# Authors
# ~~~~~~~
# Benjamin Schweizer, http://benjamin-schweizer.de/contact
#
# Changes
# ~~~~~~~
# 2008-10-16, benjamin: initial release
#
# Todo
# ~~~~
# - we'll see
#
import sys
import random
import time
def usage():
print """Copyright (c) 2008 Benjamin Schweizer. All rights reserved.
usage:
iostat <device> [time]
device := some block device
time := time in seconds
example:
iostat /dev/sda
"""
def greek(size):
"""Return a string representing the greek/metric suffix of a size"""
# Copyright (c) 1999 Martin Pohl, copied from
# http://mail.python.org/pipermail/python-list/1999-December/018519.html
_abbrevs = [
(1<<50L, ' P'),
(1<<40L, ' T'),
(1<<30L, ' G'),
(1<<20L, ' M'),
(1<<10L, ' k'),
(1, '')
]
for factor, suffix in _abbrevs:
if size > factor:
break
return `int(size/factor)` + suffix
def iotest(dev, blocksize=512, t=10):
"""io test"""
fh = open(dev, 'r')
bof = 0
fh.seek(0,2)
eof = fh.tell()
io_num = 0
start_ts = time.time()
while time.time() < start_ts+t:
io_num += 1
pos = random.randint(bof,eof)
fh.seek(pos)
blockdata = fh.read(blocksize)
end_ts = time.time()
total_ts = end_ts - start_ts
print "%s, %sB, %sB blocks: %.1f IOs/s, %sB/s" % (dev, greek(eof), greek(blocksize), io_num/total_ts, greek(int(blocksize*io_num/total_ts)))
return io_num/total_ts
if __name__ == '__main__':
if len(sys.argv) < 2:
usage()
raise SystemExit
device = sys.argv[1]
t = 10
if len(sys.argv) == 3:
t = int(sys.argv[2])
blocksize = 512
try:
iops = 2
while iops > 1:
iops = iotest(device, blocksize, t)
blocksize *= 2
except IOError, (err_no, err_str):
raise SystemExit(err_str)
except KeyboardInterrupt:
print "caught ctrl-c, bye."
# eof.
Measuring Disk IO Performance (alternative)
see: http://benjamin-schweizer.de/measuring-disk-io-performance.html
files: http://benjamin-schweizer.de/files/iops/
Performance on AWS
EBS blocks .. root@api:~# ./iops-2010-08-12 --num_threads 1 --time 2 /dev/sdf /dev/sdf, 107.37 GB, 1 threads: 512 B blocks: 24.9 IO/s, 12.5 KiB/s (102.2 kbit/s) 1 KiB blocks: 84.3 IO/s, 84.3 KiB/s (690.9 kbit/s) 2 KiB blocks: 107.6 IO/s, 215.1 KiB/s ( 1.8 Mbit/s) 4 KiB blocks: 96.9 IO/s, 387.6 KiB/s ( 3.2 Mbit/s) 8 KiB blocks: 97.0 IO/s, 776.3 KiB/s ( 6.4 Mbit/s) 16 KiB blocks: 55.5 IO/s, 888.5 KiB/s ( 7.3 Mbit/s) 32 KiB blocks: 9.0 IO/s, 287.7 KiB/s ( 2.4 Mbit/s) 64 KiB blocks: 44.4 IO/s, 2.8 MiB/s ( 23.3 Mbit/s) 128 KiB blocks: 53.3 IO/s, 6.7 MiB/s ( 55.9 Mbit/s) 256 KiB blocks: 47.4 IO/s, 11.9 MiB/s ( 99.5 Mbit/s) 512 KiB blocks: 29.0 IO/s, 14.5 MiB/s (121.5 Mbit/s) 1 MiB blocks: 24.0 IO/s, 24.0 MiB/s (201.3 Mbit/s) 2 MiB blocks: 20.9 IO/s, 41.7 MiB/s (350.0 Mbit/s) 4 MiB blocks: 12.2 IO/s, 48.7 MiB/s (408.7 Mbit/s) 8 MiB blocks: 8.4 IO/s, 67.6 MiB/s (566.8 Mbit/s) 16 MiB blocks: 2.1 IO/s, 33.9 MiB/s (284.3 Mbit/s) 32 MiB blocks: 0.8 IO/s, 25.4 MiB/s (213.4 Mbit/s) MNT default device [root@process3 ~]# ./iops-2010-08-12 --num_threads 1 --time 2 /dev/sdb /dev/sdb, 440.20 GB, 1 threads: 512 B blocks: 1259.6 IO/s, 629.8 KiB/s ( 5.2 Mbit/s) 1 KiB blocks: 1216.9 IO/s, 1.2 MiB/s ( 10.0 Mbit/s) 2 KiB blocks: 1441.9 IO/s, 2.8 MiB/s ( 23.6 Mbit/s) 4 KiB blocks: 927.2 IO/s, 3.6 MiB/s ( 30.4 Mbit/s) 8 KiB blocks: 1384.7 IO/s, 10.8 MiB/s ( 90.7 Mbit/s) 16 KiB blocks: 1109.9 IO/s, 17.3 MiB/s (145.5 Mbit/s) 32 KiB blocks: 1161.7 IO/s, 36.3 MiB/s (304.5 Mbit/s) 64 KiB blocks: 1068.0 IO/s, 66.7 MiB/s (559.9 Mbit/s) 128 KiB blocks: 685.9 IO/s, 85.7 MiB/s (719.2 Mbit/s) 256 KiB blocks: 623.5 IO/s, 155.9 MiB/s ( 1.3 Gbit/s) 512 KiB blocks: 578.6 IO/s, 289.3 MiB/s ( 2.4 Gbit/s) 1 MiB blocks: 432.2 IO/s, 432.2 MiB/s ( 3.6 Gbit/s) 2 MiB blocks: 201.3 IO/s, 402.6 MiB/s ( 3.4 Gbit/s) 4 MiB blocks: 141.8 IO/s, 567.3 MiB/s ( 4.8 Gbit/s) 8 MiB blocks: 72.8 IO/s, 582.2 MiB/s ( 4.9 Gbit/s) 16 MiB blocks: 39.9 IO/s, 638.4 MiB/s ( 5.4 Gbit/s) 32 MiB blocks: 18.0 IO/s, 575.7 MiB/s ( 4.8 Gbit/s) 64 MiB blocks: 9.2 IO/s, 588.2 MiB/s ( 4.9 Gbit/s) 128 MiB blocks: 4.6 IO/s, 584.2 MiB/s ( 4.9 Gbit/s) 256 MiB blocks: 2.1 IO/s, 547.7 MiB/s ( 4.6 Gbit/s) 512 MiB blocks: 1.1 IO/s, 558.4 MiB/s ( 4.7 Gbit/s) 1 GiB blocks: 0.5 IO/s, 523.8 MiB/s ( 4.4 Gbit/s)
