Table of Contents

General linux tips and tricks

Limit the CPU usage of an application (process) - cpulimit

Installation:

Download last stable version of cpulimit Then extract the source and compile with make:

  tar zxf cpulimit-xxx.tar.gz
  cd cpulimit-xxx
  make

Executable file name is cpulimit. You may want to copy it in /usr/bin.

Usage: Limit the process 'bigloop' by executable name to 40% CPU:

  cpulimit --exe bigloop --limit 40
  cpulimit --exe /usr/local/bin/bigloop --limit 40

Limit a process by PID to 55% CPU:

  cpulimit --pid 2960 --limit 55 
If your machine has one processor you can limit the percentage from 0% to 100%, which means that if you set for example 50%, your process cannot use more than 500 ms of cpu time for each second. But if your machine has four processors, percentage may vary from 0% to 400%, so setting the limit to 200% means to use no more than half of the available power. In any case, the percentage is the same of what you see when you run top.

Move a running process into a screen session

http://isteve.bofh.cz/~isteve/knowledgebase/articles/screenify.html:

#!/bin/sh
# Copyright Timo Lindfors 2004
usage () {
    echo usage: $0 pid
    exit 1
}
TCGETS=0x5401
TCSETS=0x5402
SIZEOF_STRUCT_TERMIOS=60
O_RDWR=2
((FLAGS=O_RDWR))

PID=$1
if [ x`which gdb` == x ]; then
    echo gdb not found in PATH. Please apt-get install gdb
    exit
fi
if [ x$PID == x ]; then
    usage;
fi
if [ x$2 != x ]; then
    usage;
fi
MYPID=$$
MYFD0=`readlink /proc/$MYPID/fd/0`
MYFD1=`readlink /proc/$MYPID/fd/1`
MYFD2=`readlink /proc/$MYPID/fd/2`
EXE=`readlink /proc/$PID/exe`
if [ x$EXE == x ]; then
    echo $0: $PID: no such pid
    exit 1
fi

BATCHFILE=`mktemp /tmp/gdb.XXXXXXXXXXXX`
cat >$BATCHFILE <<EOF
file $EXE
attach $PID
call malloc($SIZEOF_STRUCT_TERMIOS)
call malloc($SIZEOF_STRUCT_TERMIOS)
call malloc($SIZEOF_STRUCT_TERMIOS)
call ioctl(0, $TCGETS, \$1)
call ioctl(1, $TCGETS, \$2)
call ioctl(2, $TCGETS, \$3)
call close(0)
call close(1)
call close(2)
call open("$MYFD0", $FLAGS)
call open("$MYFD1", $FLAGS)
call open("$MYFD2", $FLAGS)
call ioctl(0, $TCSETS, \$1)
call ioctl(1, $TCSETS, \$2)
call ioctl(2, $TCSETS, \$3)
call free(\$1)
call free(\$2)
call free(\$3)
detach
EOF
gdb -batch -x $BATCHFILE >/dev/null 2>&1 </dev/null
rm $BATCHFILE

cat <<EOF
Process $PID should now be talking to this pty. Refresh the screen
(e.g. CTRL+L) and have fun!
EOF

exec tail -f --pid=$PID /proc/$PID/stat

Force a clean reboot when the system freezes

If you manage to freeze your system in such a way that even Ctrl+Alt+Del doesn’t work anymore, the mouse is stuck on the screen any none of the key combinations will work, don’t think of the reset button just yet.

Hold down the Right Alt and SysRq keys and press this sequence:

R E I S U B

R: Switch the keyboard from raw mode to XLATE mode
E: Send the SIGTERM signal to all processes except init
I: Send the SIGKILL signal to all processes except init
S: Sync all mounted filesystems
U: Remount all mounted filesystems in read-only mode
B: Immediately reboot the system, without unmounting partitions or syncing

This will cleanly unmount drives, terminate processes and nicely reboot your machine.

If you suspect memory corruption killed your kernel, be careful with Sync option. It may flush corrupted disk buffers over your filesystem and cause more headache after reboot.

also for more detailed info see: http://en.wikipedia.org/wiki/Magic_SysRq_key

suid, sgid, sticky bit, permissions

Four Tweaks for Using Linux with Solid State Drives

SSDs (solid state drives) are great. They’re shock resistant, consume less power, produce less heat, and have very fast seek times. If you have a computer with an SSD, such as an Eee PC, there are some tweaks you can make to increase performance and extend the life of the disk.

 sudo gedit /etc/fstab

Ubuntu uses the relatime option by default. For your SSD partitions (formatted as ext3), replace relatime with noatime in fstab. Reboot for the changes to take effect.

 sudo gedit /etc/fstab

Add this line to fstab to mount /tmp (temporary files) as tmpfs (temporary file system):

 tmpfs /tmp tmpfs defaults,noatime,mode=1777 0 0

Reboot for the changes to take effect. Running df, you should see a new line with /tmp mounted on tmpfs:

 tmpfs 513472 30320 483152 6% /tmp

The default I/O scheduler in Linux is cfq, completely fair queuing. cfq is works well on hard disks, but I’ve found it to cause problems on my Eee PC’s SSD. While writing a large file to disk, any other application which tries to write hang until the other write finishes. The I/O scheduler can be changed on a per-drive basis without rebooting. Run this command to get the current scheduler for a disk and the alternative options:

 cat /sys/block/sda/queue/scheduler

You’ll probably have four options, the one in brackets is currently being used by the disk specified in the previous command:

 noop anticipatory deadline [cfq]

Two of these are better suited to SSD drives: noop and deadline. Using one of these in the same situation, the application will still hang but only for a few seconds instead of until the disk is free again. Not great, but much better than cfq.

Here’s how to change the I/O scheduler of a disk to deadline:

 echo deadline > /sys/block/sda/queue/scheduler
The above command needs to be run as root. If you use sudo, make sure you escape the > with a backslash (\), otherwise output of sudo command instead of echo will go to /sys/block/sda/queue/scheduler, and of course it won't be as the root user.

You can replace sda with the disk you want to change, and deadline with any of the available schedulers. This change is temporary and will be reset when you reboot.

If you’re using the deadline scheduler, there’s another option you can change for the SSD. This command is also temporary and also is a per-disk option:

 echo 1 > /sys/block/sda/queue/iosched/fifo_batch

You can apply the scheduler you want to all your drives by adding a boot parameter in GRUB. The menu.lst file is regenerated whenever the kernel is updated, which would wipe out your change. Instead of this way, I added commands to rc.local to do the same thing.

Open rc.local:

 sudo gedit /etc/rc.local

Put any lines you add before the exit 0. I added six lines for my Eee PC, three to change sda (small SSD), sdb (large SSD), and sdc (SD card) to deadline, and three to get the fifo_batch option on each:

 echo deadline > /sys/block/sda/queue/scheduler
 echo deadline > /sys/block/sdb/queue/scheduler
 echo deadline > /sys/block/sdc/queue/scheduler
 echo 1 > /sys/block/sda/queue/iosched/fifo_batch
 echo 1 > /sys/block/sdb/queue/iosched/fifo_batch
 echo 1 > /sys/block/sdc/queue/iosched/fifo_batch

Reboot to run the new rc.local file.

[update] Commenter dondad has pointed out that it’s possible to add boot parameters to menu.lst that won’t be wiped out by an upgrade. Open menu.lst (Remember to make a backup of this file before you edit it):

    sudo gedit /boot/grub/menu.lst
    The kopt line gives the default parameters to boot Linux with. Mine looks like this:
    # kopt=root=UUID=6722605f-677c-4d22-b9ea-e1fb0c7470ee ro
    Don’t uncomment this line. Just add any extra parameters you would like. To change the I/O scheduler, use the elevator option:
    elevator=deadline
    Append that to the end of the kopt line. Save and close menu.lst. Then you need to run update-grub to apply your change to the whole menu:
    sudo update-grub[end update] 

Want to know how fast your SSD or other storage device is? Using hdparm you can test the read performance of your disk: sudo hdparm -t /dev/sda

The 4 GB SSD on my Eee PC 901 gets about 33 MB/s. My desktop PC’s hard drive gets about 78 MB/s. (What hdparm doesn’t show is that the seek time for an SSD is much, much lower than a hard disk.)

Have any other suggestions for SSDs, or disagree with any of these? Leave a comment to let me know.

Reducing the frequency of disk-writes

I found that my Linux-system was way to noisy for my bedroom: almost every 5 seconds it was writing to disk. I found 2 solutions:

  1. use noflushd, but I didn't like that: it also spins down the harddisk which reduces the lifetime of your harddisk
  2. fiddle around with mount-options and kernel-parameters

For the last option, I did 2 things: in /etc/fstab I added 'noatime' to the options of the mounted partition:

 /dev/hda2       /       ext3    defaults,noatime  1 1

and I set some parameters which make disk-reads occur as soon as (reasonably) possible and disk-writes only when all buffers are at their limits:

 echo "90 500 0 0 600000 600000 95 20 0" > /proc/sys/vm/bdflush
 /sbin/elvtune /dev/hda -r 2048 -w 131072
Warning: don't use these settings if your PC is not reliable and crashes often or when power-outtages occur frequently!

SpamAssassin 'sa_learn'/updatedb/backups make system unresponsive

Not only can you renice the cpu priority, one can also renice the I/O priority. For this to work you need to use the 'cfq' I/O scheduler and the ionice utility. Enabling the cfq I/O scheduler:

 echo cfq > /sys/block/hda/queue/scheduler

Then create the following script (/usr/local/bin/LP):

f /usr/local/bin/LP

#!/bin/sh
/bin/nice -n 40 /usr/bin/ionice -c 3 $*

Now when you want to run some disk-intensive process while not suffering too much, but 'LP ' in front of the command. E.g. 'LP updatedb'.

How to clear (and keep empty) the pagecache, dentries and inodes

As this is a non-destructive operation, and dirty objects are not freeable, the user should run “sync” first in order to make sure all cached objects are freed. If you are writing data at the time you run these commands, you’ll actually be dumping the data out of the filesystem cache before it reaches the disk, which could lead to very bad things.

This tunable was added in 2.6.16.

 echo 3 > /proc/sys/vm/drop_caches

Afterwards, echo '0' to this file.

Detailed explanation

echo # > /proc/sys/vm/drop_caches
# description
1 To free pagecache
2 To free dentries and inodes
3 To free pagecache, dentries and inodes

Linux memory management

Memory overcommit can be configured via two parameters:

 loreto:/tmp # sysctl -a|grep overcommit
 vm.overcommit_ratio = 50
 vm.overcommit_memory = 0

From Red Hat manual:

Note This setting is only recommended for systems with swap areas larger than physical memory.

/proc/sys/vm/vfs_cache_pressure

swappiness (/proc/sys/vm/swappiness)

Example:

Getting rid of that obsolete delay after password authentication

In Debian, add nodelay to the line with pam_unix.so in /etc/pam.d/common-auth.

Reducing disk-access (and thus saving power)

Mount filesystems with parameters:

Enable laptop-mode with this command:

 echo 5 > /proc/sys/vm/laptop_mode

Expire dirty buffers only after a day (same warning applies):

 echo 864000 > /proc/sys/vm/dirty_expire_centisecs

See also: Tips & Tricks - Saving power with linux

Reduce disk I/O for small reads using memory

from Racker Hacker by major

Many applications that are used on a standard server perform quite a few of small writes to the disk (like MySQL or Apache). These writes can pile up and limit the performance of your applications. If you have kernel 2.6.9 or later, you can adjust how these small writes are handled to allow for better performance.

There’s two main kernel variables to know:

vm.dirty_ratio - The highest % of your memory that can be used to hold dirty data. If you set this to a low value, the kernel will flush small writes to the disk more often. Higher values allow the small writes to stack up in memory. They’ll go to the disk in bigger chunks.

vm.dirty_background_ratio - The lowest % of your memory where pdflush is told to stop when it is writing dirty data. You’ll want to keep this set as low as possible.

These might confuse you. In short, when your memory begins filling with little pieces of data that needs to be written to the disk, it will keep filling until it reaches the dirty_ratio. At that point, pdflush will start up, and it will write data until it reduces the dirty data to the value set by dirty_background_ratio.

Stock 2.6.9 kernels have a dirty_background_ratio of 10% and a dirty_ratio of 40%. Some distributions tweak these defaults to something different, so you may want to review the settings on your system. On a system with heavy disk I/O, you can increase the dirty_ratio and reduce the dirty_background_ratio. A little experimentation may be necessary to find the perfect setting for your server.

If you want to play with the variables, just use your standard echo:

 echo 5 > /proc/sys/vm/dirty_background_ratio
 echo 60 > /proc/sys/vm/dirty_ratio

Once you’ve found the right setting, you can set it permanently by adding lines to your /etc/sysctl.conf:

 vm.dirty_background_ratio = 5
 vm.dirty_ratio = 60

If you have a reliable server with a good RAID card and power supply, you could set the dirty_ratio to 100 and the dirty_background_ratio to 1. This was recommended by a buddy of mine who runs quite a few servers running virtual machines.

convert files from ISO to UTF-8

 
for x in `find . -name '*.htm'` ; do iconv -f ISO-8859-2 -t UTF-8 $x > "$x.utf8"; rm $x; mv "$x.utf8" $x; done

How do I force users to change their passwords upon first login?

1.) Firstly, lock the account to prevent the user from using the login until the change has been made:

 # usermod -L <username>

2.) Change the password expiration date to 0 to ensure the user changes the password during the next login:

 # chage -d 0 <username>

3.) To unlock the account after the change do the following:

 # usermod -U <username>

space report

for i in `ls -1 /home`;do du -sh /home/$i|mail $i -a “From: \”Admininistrator\“ root@aufbix.org” -s “Poraba prostora `date +%d.%m.%y`”;done