Differences

This shows you the differences between two versions of the page.

--- linux:benchmark [2008/05/05 18:04]
a
+++ linux:benchmark [2010/08/18 14:57] (current)
a
@@ Line 1: / Line 1: @@
-   apt-get install contest dbench stress tiobench
+====== Benchmarking process / IO / latency ======
+links:
+   * [[http://www.howtoforge.com/measuring-linux-latency-with-latencytop-on-ubuntu-8.10-and-debian-lenny|LatencyTop on Ubuntu and Debian]]
+   apt-get install contest dbench stress tiobench latencytop
 FIXME
@@ Line 15: / Line 21: @@
 Please download and run it on your dedicated or VPS. Then post your result here for discussion. Thanks!
+===== IO test =====
+   * [[http://benjamin-schweizer.de/files/iotest/|script available here]], mirrored copy is down below.
+<code>
+/--(a@tbox)-(Tue Oct 21 15:28:24 CEST 2008)-(0)--
+\--(#!:~/src)-- python iotest-2008-10-15 /dev/sda3
+/dev/sda3, 23 GB, 512B blocks: 66.1 IOs/s, 33 kB/s
+/dev/sda3, 23 GB, 1024B blocks: 65.7 IOs/s, 65 kB/s
+/dev/sda3, 23 GB, 2 kB blocks: 64.9 IOs/s, 129 kB/s
+...
+</code>
+<code python|f iotest-2008-10-15.py>
+#!/usr/bin/env python
+#
+# Copyright (c) 2008 Benjamin Schweizer. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in the
+#     documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Benjamin Schweizer nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
+# IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+#
+#
+# Abstract
+# ~~~~~~~~
+# Benchmark disk IOs
+#
+# Authors
+# ~~~~~~~
+# Benjamin Schweizer, http://benjamin-schweizer.de/contact
+#
+# Changes
+# ~~~~~~~
+# 2008-10-16, benjamin: initial release
+#
+# Todo
+# ~~~~
+# - we'll see
+#
+import sys
+import random
+import time
+def usage():
+    print """Copyright (c) 2008 Benjamin Schweizer. All rights reserved.
+usage:
+    iostat <device> [time]
+    device  := some block device
+    time    := time in seconds
+example:
+    iostat /dev/sda
+"""
+def greek(size):
+    """Return a string representing the greek/metric suffix of a size"""
+    # Copyright (c) 1999 Martin Pohl, copied from
+    # http://mail.python.org/pipermail/python-list/1999-December/018519.html
+    _abbrevs = [
+        (1<<50L, ' P'),
+        (1<<40L, ' T'),
+        (1<<30L, ' G'),
+        (1<<20L, ' M'),
+        (1<<10L, ' k'),
+        (1, '')
+    ]
+    for factor, suffix in _abbrevs:
+        if size > factor:
+            break
+    return `int(size/factor)` + suffix
+def iotest(dev, blocksize=512, t=10):
+    """io test"""
+    fh = open(dev, 'r')
+    bof = 0
+    fh.seek(0,2)
+    eof = fh.tell()
+    io_num = 0
+    start_ts = time.time()
+    while time.time() < start_ts+t:
+        io_num += 1
+        pos = random.randint(bof,eof)
+        fh.seek(pos)
+        blockdata = fh.read(blocksize)
+    end_ts = time.time()
+    total_ts = end_ts - start_ts
+    print "%s, %sB, %sB blocks: %.1f IOs/s, %sB/s" % (dev, greek(eof), greek(blocksize), io_num/total_ts, greek(int(blocksize*io_num/total_ts)))
+    return io_num/total_ts
+if __name__ == '__main__':
+    if len(sys.argv) < 2:
+        usage()
+        raise SystemExit
+    device = sys.argv[1]
+    t = 10
+    if len(sys.argv) == 3:
+        t = int(sys.argv[2])
+    blocksize = 512
+    try:
+        iops = 2
+        while iops > 1:
+            iops = iotest(device, blocksize, t)
+            blocksize *= 2
+    except IOError, (err_no, err_str):
+        raise SystemExit(err_str)
+    except KeyboardInterrupt:
+        print "caught ctrl-c, bye."
+# eof.
+</code>
+==== Measuring Disk IO Performance (alternative) ====
+see: http://benjamin-schweizer.de/measuring-disk-io-performance.html \\
+files: http://benjamin-schweizer.de/files/iops/ \\
+\\
+**Performance on AWS**
+<code>
+EBS blocks ..
+root@api:~# ./iops-2010-08-12 --num_threads 1 --time 2 /dev/sdf
+/dev/sdf, 107.37 GB, 1 threads:
+   B blocks:   24.9 IO/s,  12.5 KiB/s (102.2 kbit/s)
+KiB blocks:   84.3 IO/s,  84.3 KiB/s (690.9 kbit/s)
+KiB blocks:  107.6 IO/s, 215.1 KiB/s (  1.8 Mbit/s)
+KiB blocks:   96.9 IO/s, 387.6 KiB/s (  3.2 Mbit/s)
+KiB blocks:   97.0 IO/s, 776.3 KiB/s (  6.4 Mbit/s)
+KiB blocks:   55.5 IO/s, 888.5 KiB/s (  7.3 Mbit/s)
+KiB blocks:    9.0 IO/s, 287.7 KiB/s (  2.4 Mbit/s)
+KiB blocks:   44.4 IO/s,   2.8 MiB/s ( 23.3 Mbit/s)
+KiB blocks:   53.3 IO/s,   6.7 MiB/s ( 55.9 Mbit/s)
+KiB blocks:   47.4 IO/s,  11.9 MiB/s ( 99.5 Mbit/s)
+KiB blocks:   29.0 IO/s,  14.5 MiB/s (121.5 Mbit/s)
+MiB blocks:   24.0 IO/s,  24.0 MiB/s (201.3 Mbit/s)
+MiB blocks:   20.9 IO/s,  41.7 MiB/s (350.0 Mbit/s)
+MiB blocks:   12.2 IO/s,  48.7 MiB/s (408.7 Mbit/s)
+MiB blocks:    8.4 IO/s,  67.6 MiB/s (566.8 Mbit/s)
+MiB blocks:    2.1 IO/s,  33.9 MiB/s (284.3 Mbit/s)
+MiB blocks:    0.8 IO/s,  25.4 MiB/s (213.4 Mbit/s)
+MNT default device
+[root@process3 ~]# ./iops-2010-08-12 --num_threads 1 --time 2 /dev/sdb
+/dev/sdb, 440.20 GB, 1 threads:
+   B blocks: 1259.6 IO/s, 629.8 KiB/s (  5.2 Mbit/s)
+KiB blocks: 1216.9 IO/s,   1.2 MiB/s ( 10.0 Mbit/s)
+KiB blocks: 1441.9 IO/s,   2.8 MiB/s ( 23.6 Mbit/s)
+KiB blocks:  927.2 IO/s,   3.6 MiB/s ( 30.4 Mbit/s)
+KiB blocks: 1384.7 IO/s,  10.8 MiB/s ( 90.7 Mbit/s)
+KiB blocks: 1109.9 IO/s,  17.3 MiB/s (145.5 Mbit/s)
+KiB blocks: 1161.7 IO/s,  36.3 MiB/s (304.5 Mbit/s)
+KiB blocks: 1068.0 IO/s,  66.7 MiB/s (559.9 Mbit/s)
+KiB blocks:  685.9 IO/s,  85.7 MiB/s (719.2 Mbit/s)
+KiB blocks:  623.5 IO/s, 155.9 MiB/s (  1.3 Gbit/s)
+KiB blocks:  578.6 IO/s, 289.3 MiB/s (  2.4 Gbit/s)
+MiB blocks:  432.2 IO/s, 432.2 MiB/s (  3.6 Gbit/s)
+MiB blocks:  201.3 IO/s, 402.6 MiB/s (  3.4 Gbit/s)
+MiB blocks:  141.8 IO/s, 567.3 MiB/s (  4.8 Gbit/s)
+MiB blocks:   72.8 IO/s, 582.2 MiB/s (  4.9 Gbit/s)
+MiB blocks:   39.9 IO/s, 638.4 MiB/s (  5.4 Gbit/s)
+MiB blocks:   18.0 IO/s, 575.7 MiB/s (  4.8 Gbit/s)
+MiB blocks:    9.2 IO/s, 588.2 MiB/s (  4.9 Gbit/s)
+MiB blocks:    4.6 IO/s, 584.2 MiB/s (  4.9 Gbit/s)
+MiB blocks:    2.1 IO/s, 547.7 MiB/s (  4.6 Gbit/s)
+MiB blocks:    1.1 IO/s, 558.4 MiB/s (  4.7 Gbit/s)
+GiB blocks:    0.5 IO/s, 523.8 MiB/s (  4.4 Gbit/s)
+</code>