Here's a way to get a list of files read by any application, so you can use readahead to preload those files optimally from disk (conventional spinning hard drives, this probably doesn't do so much for SSDs):
CMD=firefox
strace -fe open $CMD 2>&1 | grep open | sed 's/.*open("\(.*\)".*/\1/' > $CMD.preload
# you can sift through that $CMD.preload file to look for things that don't belong
readahead $CMD.preload # preloads all those files into cache
time $CMD # should now start quite a bit faster, without much disk activity
## to clear disk cache as root (useful for testing / benchmarking)
echo 3 > /proc/sys/vm/drop_caches
If it works, you might want to append the contents of the .preload files for your commonly-used apps to /etc/readahead.d/default.later , so they are automatically loaded on startup (RAM size permitting)
Showing posts with label bash. Show all posts
Showing posts with label bash. Show all posts
Wednesday, November 17, 2010
Running commands on several remote hosts using ssh and xargs
There are a few different ways to run commands on groups or clusters of remote nodes, depending upon how complex the command.
Assuming your machines are named "node01" - "node22" :
# Run a command in parallel on all remote nodes
# results come back in random order as they are received.
pdsh -w "node[01-22]" df
# pdsh allows some more complex listings of hosts
pdsh -w "node04,node[06-09]" reboot
# Run a command sequentially on all remote nodes
# slow, but results come back in order
seq -w 1 22 | xargs -I '{}' ssh node'{}' df
# Run a command in parallel on all remote nodes without pdsh
seq -w 1 22 | xargs -P 22 -I '{}' ssh node'{}' df
# Run a command in parallel needing pipes on the remote host
# Otherwise, pipes are processed locally
seq -w 1 22 | xargs -P 22 -I '{}' ssh node'{}' \
"ps afx > \`hostname\`.txt"
# Run a command in parallel needing root
# sudo requires a tty, hence we pop up xterm windows
seq -w 1 22 | xargs -P 22 -I '{}' xterm -e \
"ssh -t node'{}' sudo gdm-restart"
# Run a command in parallel needing root and pipes on the remote host
seq -w 1 22 | xargs -P 22 -I '{}' xterm -e \
"ssh -t node'{}' sudo bash -c \"echo 3 > /proc/sys/vm/drop_caches\""
Assuming your machines are named "node01" - "node22" :
# Run a command in parallel on all remote nodes
# results come back in random order as they are received.
pdsh -w "node[01-22]" df
# pdsh allows some more complex listings of hosts
pdsh -w "node04,node[06-09]" reboot
# Run a command sequentially on all remote nodes
# slow, but results come back in order
seq -w 1 22 | xargs -I '{}' ssh node'{}' df
# Run a command in parallel on all remote nodes without pdsh
seq -w 1 22 | xargs -P 22 -I '{}' ssh node'{}' df
# Run a command in parallel needing pipes on the remote host
# Otherwise, pipes are processed locally
seq -w 1 22 | xargs -P 22 -I '{}' ssh node'{}' \
"ps afx > \`hostname\`.txt"
# Run a command in parallel needing root
# sudo requires a tty, hence we pop up xterm windows
seq -w 1 22 | xargs -P 22 -I '{}' xterm -e \
"ssh -t node'{}' sudo gdm-restart"
# Run a command in parallel needing root and pipes on the remote host
seq -w 1 22 | xargs -P 22 -I '{}' xterm -e \
"ssh -t node'{}' sudo bash -c \"echo 3 > /proc/sys/vm/drop_caches\""
Subscribe to:
Posts (Atom)