Showing posts with label bash. Show all posts
Showing posts with label bash. Show all posts

Wednesday, November 17, 2010

Using readahead to speed up disk loading times of any application

Here's a way to get a list of files read by any application, so you can use readahead to preload those files optimally from disk (conventional spinning hard drives, this probably doesn't do so much for SSDs):


CMD=firefox
strace -fe open $CMD 2>&1 | grep open | sed 's/.*open("\(.*\)".*/\1/' > $CMD.preload

# you can sift through that $CMD.preload file to look for things that don't belong

readahead $CMD.preload # preloads all those files into cache

time $CMD # should now start quite a bit faster, without much disk activity

## to clear disk cache as root (useful for testing / benchmarking)
echo 3 > /proc/sys/vm/drop_caches

If it works, you might want to append the contents of the .preload files for your commonly-used apps to /etc/readahead.d/default.later , so they are automatically loaded on startup (RAM size permitting)


Running commands on several remote hosts using ssh and xargs

There are a few different ways to run commands on groups or clusters of remote nodes, depending upon how complex the command.

Assuming your machines are named "node01" - "node22" :


# Run a command in parallel on all remote nodes
# results come back in random order as they are received.
pdsh -w "node[01-22]" df


# pdsh allows some more complex listings of hosts
pdsh -w "node04,node[06-09]" reboot


# Run a command sequentially on all remote nodes
# slow, but results come back in order
seq -w 1 22 | xargs -I '{}' ssh node'{}' df


# Run a command in parallel on all remote nodes without pdsh
seq -w 1 22 | xargs -P 22 -I '{}' ssh node'{}' df


# Run a command in parallel needing pipes on the remote host
# Otherwise, pipes are processed locally
seq -w 1 22 | xargs -P 22 -I '{}' ssh node'{}' \
"ps afx > \`hostname\`.txt"


# Run a command in parallel needing root
# sudo requires a tty, hence we pop up xterm windows
seq -w 1 22 | xargs -P 22 -I '{}' xterm -e \
"ssh -t node'{}' sudo gdm-restart"


# Run a command in parallel needing root and pipes on the remote host
seq -w 1 22 | xargs -P 22 -I '{}' xterm -e \
"ssh -t node'{}' sudo bash -c \"echo 3 > /proc/sys/vm/drop_caches\""