Linux related stuff

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Benchmarking process / IO / latency ======

links:
   * [[http://www.howtoforge.com/measuring-linux-latency-with-latencytop-on-ubuntu-8.10-and-debian-lenny|LatencyTop on Ubuntu and Debian]]


   apt-get install contest dbench stress tiobench latencytop

FIXME


http://lbs.sourceforge.net/

===== Unix Benchmarking =====
see: http://www.webhostingtalk.com/showthread.php?s=&threadid=308055 \\
Download: [[http://members.dslextreme.com/users/andylee/unixbench-4.1.0-wht.tar.gz|UnixBench v4.1.0 - WHT Variant]]
   # gunzip -dvc unixbench-4.1.0-wht.tar.gz | tar xvf -
   # cd unixbench-4.1.0-wht
   # make
   # ./Run

Please download and run it on your dedicated or VPS. Then post your result here for discussion. Thanks!

===== IO test =====

   * [[http://benjamin-schweizer.de/files/iotest/|script available here]], mirrored copy is down below.

<code>
/--(a@tbox)-(Tue Oct 21 15:28:24 CEST 2008)-(0)--
\--(#!:~/src)-- python iotest-2008-10-15 /dev/sda3
/dev/sda3, 23 GB, 512B blocks: 66.1 IOs/s, 33 kB/s
/dev/sda3, 23 GB, 1024B blocks: 65.7 IOs/s, 65 kB/s
/dev/sda3, 23 GB, 2 kB blocks: 64.9 IOs/s, 129 kB/s
...
</code>

<code python|f iotest-2008-10-15.py>
#!/usr/bin/env python
#
# Copyright (c) 2008 Benjamin Schweizer. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
#   * Redistributions of source code must retain the above copyright
#     notice, this list of conditions and the following disclaimer.
#   * Redistributions in binary form must reproduce the above copyright
#     notice, this list of conditions and the following disclaimer in the
#     documentation and/or other materials provided with the
#     distribution.
#   * Neither the name of Benjamin Schweizer nor the names of its
#     contributors may be used to endorse or promote products derived
#     from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
# IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
#
# Abstract
# ~~~~~~~~
# Benchmark disk IOs
#
# Authors
# ~~~~~~~
# Benjamin Schweizer, http://benjamin-schweizer.de/contact
#
# Changes
# ~~~~~~~
# 2008-10-16, benjamin: initial release
#
# Todo
# ~~~~
# - we'll see
#

import sys
import random
import time

def usage():
    print """Copyright (c) 2008 Benjamin Schweizer. All rights reserved.

usage:

    iostat <device> [time]

    device  := some block device
    time    := time in seconds

example:

    iostat /dev/sda

"""


def greek(size):
    """Return a string representing the greek/metric suffix of a size"""
    # Copyright (c) 1999 Martin Pohl, copied from
    # http://mail.python.org/pipermail/python-list/1999-December/018519.html
    _abbrevs = [
        (1<<50L, ' P'),
        (1<<40L, ' T'), 
        (1<<30L, ' G'), 
        (1<<20L, ' M'), 
        (1<<10L, ' k'),
        (1, '')
    ]

    for factor, suffix in _abbrevs:
        if size > factor:
            break

    return `int(size/factor)` + suffix


def iotest(dev, blocksize=512, t=10):
    """io test"""
    fh = open(dev, 'r')
    bof = 0
    fh.seek(0,2)
    eof = fh.tell()

    io_num = 0
    start_ts = time.time()
    while time.time() < start_ts+t:
        io_num += 1
        pos = random.randint(bof,eof)
        fh.seek(pos)
        blockdata = fh.read(blocksize)
    end_ts = time.time()

    total_ts = end_ts - start_ts

    print "%s, %sB, %sB blocks: %.1f IOs/s, %sB/s" % (dev, greek(eof), greek(blocksize), io_num/total_ts, greek(int(blocksize*io_num/total_ts)))

    return io_num/total_ts

if __name__ == '__main__':
    if len(sys.argv) < 2:
        usage()
        raise SystemExit

    device = sys.argv[1]
    t = 10
    if len(sys.argv) == 3:
        t = int(sys.argv[2])

    blocksize = 512
    try:
        iops = 2
        while iops > 1:
            iops = iotest(device, blocksize, t)
            blocksize *= 2
    except IOError, (err_no, err_str):
        raise SystemExit(err_str)
    except KeyboardInterrupt:
        print "caught ctrl-c, bye."

# eof.
</code>


==== Measuring Disk IO Performance (alternative) ====

see: http://benjamin-schweizer.de/measuring-disk-io-performance.html \\

files: http://benjamin-schweizer.de/files/iops/ \\
\\
**Performance on AWS** 
<code>
EBS blocks ..

root@api:~# ./iops-2010-08-12 --num_threads 1 --time 2 /dev/sdf
/dev/sdf, 107.37 GB, 1 threads:
 512   B blocks:   24.9 IO/s,  12.5 KiB/s (102.2 kbit/s)
   1 KiB blocks:   84.3 IO/s,  84.3 KiB/s (690.9 kbit/s)
   2 KiB blocks:  107.6 IO/s, 215.1 KiB/s (  1.8 Mbit/s)
   4 KiB blocks:   96.9 IO/s, 387.6 KiB/s (  3.2 Mbit/s)
   8 KiB blocks:   97.0 IO/s, 776.3 KiB/s (  6.4 Mbit/s)
  16 KiB blocks:   55.5 IO/s, 888.5 KiB/s (  7.3 Mbit/s)
  32 KiB blocks:    9.0 IO/s, 287.7 KiB/s (  2.4 Mbit/s)
  64 KiB blocks:   44.4 IO/s,   2.8 MiB/s ( 23.3 Mbit/s)
 128 KiB blocks:   53.3 IO/s,   6.7 MiB/s ( 55.9 Mbit/s)
 256 KiB blocks:   47.4 IO/s,  11.9 MiB/s ( 99.5 Mbit/s)
 512 KiB blocks:   29.0 IO/s,  14.5 MiB/s (121.5 Mbit/s)
   1 MiB blocks:   24.0 IO/s,  24.0 MiB/s (201.3 Mbit/s)
   2 MiB blocks:   20.9 IO/s,  41.7 MiB/s (350.0 Mbit/s)
   4 MiB blocks:   12.2 IO/s,  48.7 MiB/s (408.7 Mbit/s)
   8 MiB blocks:    8.4 IO/s,  67.6 MiB/s (566.8 Mbit/s)
  16 MiB blocks:    2.1 IO/s,  33.9 MiB/s (284.3 Mbit/s)
  32 MiB blocks:    0.8 IO/s,  25.4 MiB/s (213.4 Mbit/s)

MNT default device

[root@process3 ~]# ./iops-2010-08-12 --num_threads 1 --time 2 /dev/sdb
/dev/sdb, 440.20 GB, 1 threads:
 512   B blocks: 1259.6 IO/s, 629.8 KiB/s (  5.2 Mbit/s)
   1 KiB blocks: 1216.9 IO/s,   1.2 MiB/s ( 10.0 Mbit/s)
   2 KiB blocks: 1441.9 IO/s,   2.8 MiB/s ( 23.6 Mbit/s)
   4 KiB blocks:  927.2 IO/s,   3.6 MiB/s ( 30.4 Mbit/s)
   8 KiB blocks: 1384.7 IO/s,  10.8 MiB/s ( 90.7 Mbit/s)
  16 KiB blocks: 1109.9 IO/s,  17.3 MiB/s (145.5 Mbit/s)
  32 KiB blocks: 1161.7 IO/s,  36.3 MiB/s (304.5 Mbit/s)
  64 KiB blocks: 1068.0 IO/s,  66.7 MiB/s (559.9 Mbit/s)
 128 KiB blocks:  685.9 IO/s,  85.7 MiB/s (719.2 Mbit/s)
 256 KiB blocks:  623.5 IO/s, 155.9 MiB/s (  1.3 Gbit/s)
 512 KiB blocks:  578.6 IO/s, 289.3 MiB/s (  2.4 Gbit/s)
   1 MiB blocks:  432.2 IO/s, 432.2 MiB/s (  3.6 Gbit/s)
   2 MiB blocks:  201.3 IO/s, 402.6 MiB/s (  3.4 Gbit/s)
   4 MiB blocks:  141.8 IO/s, 567.3 MiB/s (  4.8 Gbit/s)
   8 MiB blocks:   72.8 IO/s, 582.2 MiB/s (  4.9 Gbit/s)
  16 MiB blocks:   39.9 IO/s, 638.4 MiB/s (  5.4 Gbit/s)
  32 MiB blocks:   18.0 IO/s, 575.7 MiB/s (  4.8 Gbit/s)
  64 MiB blocks:    9.2 IO/s, 588.2 MiB/s (  4.9 Gbit/s)
 128 MiB blocks:    4.6 IO/s, 584.2 MiB/s (  4.9 Gbit/s)
 256 MiB blocks:    2.1 IO/s, 547.7 MiB/s (  4.6 Gbit/s)
 512 MiB blocks:    1.1 IO/s, 558.4 MiB/s (  4.7 Gbit/s)
   1 GiB blocks:    0.5 IO/s, 523.8 MiB/s (  4.4 Gbit/s)
</code>