Friday, November 26, 2010

Migrating to Android for Palm / Linux users

As a longtime Palm TX / Linux user, this is the quickstart guide I wish I had when I finally decided to stop waiting for a clear successor to the PDA and buy a smartphone. Note: I still haven't migrated completely off of my Palm TX, so I still carry it around to consult for a few things. Fortunately, the battery lasts for weeks per charge.

Step 1: Choose your hardware:
 Something you can run CyanogenMOD on. T-Mobile has been saying they'll come out with OTA updates to the latest Android version "just next month" every month since 2/2010. Lies and marketing. Samsung's policy is a bit more transparent... they will never bother to provide updates for their phones. Unless you really think you can stay happy with stock firmware, (and every update so far has contained very useful missing functionality), CyanogenMOD will help you keep your phone current and relevant.

I picked up an HTC myTouch Slide from Craigslist, since there are plenty of people dumping them for better phones, presumably because they're too lazy to figure out how to update the firmware themselves. The slide roughly costs half as much as the $600+ high-end phones, yet has most of what I'm looking for:

  • Physical keyboard: it's way too easy to fat-finger the virtual keyboard, and I'm not a fan of Swype since I often type words not in its dictionary.
  • HSDPA radio : on T-Mobile's network, this works about as fast as DSL, I routinely get >1Mbps down and >300Kbps up even inside buildings, and it has much lower latency than EDGE/3G.
  • LED Flash: though I more often use this as a flashlight.
  • 3D performance: does about as well as the high-end 1Ghz snapdragon phones on 3D apps, though mostly because it runs at a lower resolution.
So I think the HTC Slide is probably the cheapest Android phone that still gives you 90% of the functionality of the highend ones.  Some drawbacks that might drive you to spending more:

  • 600Mhz ARMv6 cpu won't run Google Earth Mobile, and some high-end games.  I can live without them, though... Google Maps Mobile provides all the practical information I need, and the 3D games have been pretty lame so far compared to what I'm used to.
  • Front-facing camera might be nice for video conferencing apps, but haven't seen any practical ones yet.
I never buy phones with a contract anymore... T-Mobile is nice enough to give you a lower monthly fee if they're not subsidizing your handset, and buying your phone outright usually comes out cheaper in the long run.

Get a nice Class 4 or Class 6 microSD card to use with the phone.  It is essential for most apps.

On the subject of hardware, I'd recommend getting a screen protector and a belt clip holster with magnetic latch as well.  But to each their own.

Step 2: CyanogenMOD ROM flash:  
The instructions for rooting and reflashing your phone are usually pinned to the top of the CyanogenMOD forum post for your phone. You'll likely have to learn all the different names your phone goes by at this point :-P

This basically involves installing the Android-SDK to your PC and using the USB debugger to trick your phone into installing a flashing/rooting package. Then you reboot into the ROMmanager bootloader, and backup / flash the CyanogenMOD ROM image as well as the Google Addon base apps pack.

Step 2a: Repartitioning with CustomMTD
After successfully flashing, but before going too much further, consider applying CustomMTD as well to allocate more space for apps, I routinely run out of space and spend time juggling apps, while there's an extra 100MB or more just sitting there in /system being wasted.  This will repartition your internal memory to provide more space for apps :

Step 3: Sign in and Install some Apps!
You're good to go once you get to this stage!  Connect to your Google / Facebook / Twitter, etc. accounts, and start installing apps.  It's difficult finding the best apps, since the Market app manager is admittedly a mess.... Start with looking for lists of "Android Essentials," such as this one from Lifehacker.  I don't agree with all of them, but it's a start.  Here's my list on AppBrain.

There are two ways you need to know to manage apps that aren't apparent at first.  Tap the icon in mini panel at the bottom center of your screen to bring up a list of all apps.  You can then click and hold to create and arrange a shortcut icon on your ADWLauncher desktop.

Also look at the Application Manager buried in the Launcher prefs, by hitting the "menu button" while on the Launcher / Home screen, then going to "Settings | Applications | Manage Applications".  Select "All", and then "menu button | Sort by size"  You can then go into each application and hit "Move to SD" (except on system apps) to save some internal memory space.  You can also see some options to uninstall or clear data / cache, which is often easier than trying to find your apps in the Market to uninstall them.

As of Android 2.2 / Froyo, "Move to SD" will only migrate maybe half of the data to the SD... the main executables need to stay on the internal memory for security / app signature purposes.  Also, don't use "Move to SD" on any apps that you have or plan to use a desktop widget for, or the widget will disappear and/or fail to load on reboot.

One of the quicker ways to find an app that's not on your launcher is to tap the search / "g"enius button and type in the first few letters of its name, quickbar-style.  The locally installed apps that match will appear at the top of the instant search results list.

Step 4: Migrating data from Palm using JPilot
Unfortunately, there's a lot of old information on how difficult it may be to migrate your Palm PIM data to your Android device.

Under JPilot, simply export your AddressBook in vCard format, and your Calendar and ToDos in iCalendar format.  Then import those files into your GMail Contacts and Google Calendar using a web browser.  They will automatically sync to your device.  That's it!

You're then faced with a somewhat long arduous task of linking / joining duplicate entries from all of your various address books (Facebook, Google, SIM card, etc.) in your Contacts list.  Open the phone icon, hit the Contacts tab, and then "menu button | Edit contact | menu button | Join".  You can then select from the list of suggested dupes, and all the information from the different sources will show up collated together under a single entry.

Haven't found a good way of importing the rest of the information yet.  There's no native "Memos" app, so I ended up exporting all my memos to a text file, and then copying and pasting them into a few EverNote app entries.  At least they're searchable.

For encrypted data / passwords that were stored in Palm "KeyRing", I migrated to "KeePassDroid" by manually copying and pasting on the PC from JPilot's KeyRing plugin to KeePassX.  At least this gave me an opportunity to select icons and properly attach URLs to everything :-/  Lifehacker has a neat tip to place the KeePass kdb into DropBox to keep it automatically synced between your PC and Android device.  But obviously you'd want to place some extra security, perhaps by also using that optional pre-shared key file stored outside of Dropbox.

If you use Palm "CryptoPad," remember to unencrypt those memos on your Palm device so you can see them in JPilot and copy them into something else.

Step 5: Backups!
Once you're happy with your environment (but don't wait until everything is "perfect"), install the "Titanium Backup" app and save everything to SD.  Select the "Schedules" button and then the "RUN" button for "Backup all new apps & newer versions".   After that completes, connect your microSD card to your PC, and copy the entire contents to a backup directory marked with the date.  Hopefully you left the CyanogenMOD ROM backups on the SD as well, so you'll have a full restore-from-a-SD capability.
microSD cards can break (mine did), and since they contain the system backup and half of your apps and data, be mindful of all you will lose :-P

Have fun!  My next article will cover some of the essential apps, my ADWLauncher desktop setup, and some missing pieces.

Wednesday, November 17, 2010

Using readahead to speed up disk loading times of any application

Here's a way to get a list of files read by any application, so you can use readahead to preload those files optimally from disk (conventional spinning hard drives, this probably doesn't do so much for SSDs):

strace -fe open $CMD 2>&1 | grep open | sed 's/.*open("\(.*\)".*/\1/' > $CMD.preload

# you can sift through that $CMD.preload file to look for things that don't belong

readahead $CMD.preload # preloads all those files into cache

time $CMD # should now start quite a bit faster, without much disk activity

## to clear disk cache as root (useful for testing / benchmarking)
echo 3 > /proc/sys/vm/drop_caches

If it works, you might want to append the contents of the .preload files for your commonly-used apps to /etc/readahead.d/default.later , so they are automatically loaded on startup (RAM size permitting)

Running commands on several remote hosts using ssh and xargs

There are a few different ways to run commands on groups or clusters of remote nodes, depending upon how complex the command.

Assuming your machines are named "node01" - "node22" :

# Run a command in parallel on all remote nodes
# results come back in random order as they are received.
pdsh -w "node[01-22]" df

# pdsh allows some more complex listings of hosts
pdsh -w "node04,node[06-09]" reboot

# Run a command sequentially on all remote nodes
# slow, but results come back in order
seq -w 1 22 | xargs -I '{}' ssh node'{}' df

# Run a command in parallel on all remote nodes without pdsh
seq -w 1 22 | xargs -P 22 -I '{}' ssh node'{}' df

# Run a command in parallel needing pipes on the remote host
# Otherwise, pipes are processed locally
seq -w 1 22 | xargs -P 22 -I '{}' ssh node'{}' \
"ps afx > \`hostname\`.txt"

# Run a command in parallel needing root
# sudo requires a tty, hence we pop up xterm windows
seq -w 1 22 | xargs -P 22 -I '{}' xterm -e \
"ssh -t node'{}' sudo gdm-restart"

# Run a command in parallel needing root and pipes on the remote host
seq -w 1 22 | xargs -P 22 -I '{}' xterm -e \
"ssh -t node'{}' sudo bash -c \"echo 3 > /proc/sys/vm/drop_caches\""

There will be tech

Migrating some of my posts from my Slashdot Journal, which wasn't really conducive to posting and maintaining blocks of code.