Machine-readable Dates

I had some directories named in the format of “Jul 18, 2012″. Thanks, iPhoto export, but no thanks.

/bin/ls | while read dirnam; do 
  echo "$dirnam" | grep -Eq '^\w{3} \d{1,2}, \d{4}$'
  if [ $? -eq 0 ]; then 
    newdir=`gdate -d "$dirnam" +%F`
    mv -i "$dirnam" "$newdir"

Note: gdate is GNU date after doing homebrew install coreutils.
Continue reading

It’s the routing table, stupid!

It only took three years for the frustration to build up sufficiently. :-) Finally I went a little deeper to try to solve the mystery of the 30 second hang time before requests to my Linux guest VM would return. I love doing testing on a Linux guest VM via VirtualBox while working on my Mac, but the occasional slow down (with no load on the VM) drove me to the brink of madness.

I had assumed for a while that it was some complicated (potentially Mac-related) DNS asshattery. No. It’s the routing table, stupid! I finally noticed that this wasn’t right at all.

$ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface UG 100 0 0 eth2 U 0 0 0 eth3 U 1000 0 0 eth3 U 0 0 0 eth2 UG 0 0 0 eth4 U 0 0 0 eth4

So after some fiddling, I added this to my /etc/rc.local:

route -v add -host gw eth4
route -v del -net gw eth4

It is a host-only interface after all. Now the table looks far more sensible.

$ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface UG 100 0 0 eth2 U 0 0 0 eth3 U 1000 0 0 eth3 U 0 0 0 eth2 U 0 0 0 eth4 UGH 0 0 0 eth4

Hope that helps someone in the same boat.

Google Now Needlessly Requires Battery-Draining Location Settings

location based cardsI think Google Now on my Android is pretty cool. I especially like the cards that show how traffic looks for an expected commute. One thing about it that bothers me a lot however, is that it insists that it needs either “high accuracy” or “battery saving” location mode enabled. High Accuracy mode uses GPS, wifi, or mobile networks to determine location. Battery Saving mode uses wifi and mobile networks. Device Only is the third option which uses GPS solely. My question is: Why is the Device Only option not allowed for Google Now to work? Continue reading

I have high hopes for wallabag

I would really like to replace Evernote with a self-hosted solution. Wallabag is one alternative that’s pretty attractive. It’s open source, supports imports and exports, and nice on the eyes.

What’s currently holding me back is the lack of an Evernote importer (I kind of expect that, and am eagerly looking forward to writing an XSLT to make that importable ;-) and the lack of features in the browser extension. Basically, Evernote’s web clipper kicks ass, so I’d really like to keep the selection tools.

Quick Linux ACL

I wanted a directory and everything under it to always get the same owner, group and mode, regardless of who created the files. Access Control Lists to the rescue.

# setfacl -Rm u:myuser:rwX,g:www-data:rwX,d:u:myuser:rwX,d:g:www-data:rwX dir/

I had to apt-get install acl to get the setfacl command. I’m not exactly clear on why I repeat two regular ACLs with the “d:” prefix to make them default ACLs. Why not just use the default syntax exclusively?

Source: SuperUser

Raspberry Pi SSH cipher speed

I was curious to see how quickly I could transfer files to my Pi using SSH rather than FTP. Obviously using FTP is way faster than almost any other method, but still I wanted to see how fast I could transfer data over SSH.

Here’s the time it took to transfer a 50 MB file to my Pi using different SSH ciphers.

aes128-ctr    0m13.496s
3des-cbc      0m26.740s
arcfour128    0m10.221s
arcfour256    0m10.681s
arcfour       0m10.136s
blowfish-cbc  0m12.627s
cast128-cbc   0m12.862s
aes128-cbc    0m12.497s

I later re-tested the aes128-ctr cipher and it took about a second less than what I’d recorded initially. This boils down to:

  • Don’t use triple-DES ever, for both performance and security reasons
  • Most other ciphers give about the same performance, and are generally considered secure
  • arcfour is the fastest class of ciphers, but there is less trust in it from the crypto community. If you’re going to use it, try to avoid the base arcfour cipher and instead use the 128 or 256 version, which tosses out some of the initial bits as a precaution

Enable X11 Forwarding on Raspberry Pi

The usual suspects failed me last night when the $DISPLAY environment variable wasn’t being set after I logged in via SSH to my Pi. The usual suspects being to make sure that the X11 forwarding options were turned on in /etc/ssh/sshd_config on the server and in ssh_config on the client, or to use the command line options -X or -Y.

So I tried logging in again with the debug level turned up (-vvv) and saw the message, X11 forwarding request failed on channel 0. I had remembered from when this happened to me before that you also need a particular package on the server side to allow X11 authentication, whatever package contains the xauth binary. However, it was there and seemed to be working properly.

The Googles turned up this link, which showed that a new option may need to be in your sshd_config on a newer version of OpenSSH:

X11UseLocalhost no

I then did a sudo service ssh restart, which thankfully is smart enough not to kill your existing SSH session, and logged in again. Finally, I saw

$ echo $DISPLAY

and once again, all was well with the world.

Upgrade Java for the CLI on Mac OS X

I was screwing around for too long with various installers on my Mac, trying to get the latest available Java runtime working from my terminal.

Finally, some kind soul on StackExchange I think mentioned that if you want to get Java on the command line working, you have to install the JDK package, even if all you need is a JRE. Thanks, Obama!

Anyway, after installing the package from Oracle, now all is well.

$ java -version
java version "1.7.0_51"
Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)

Multi-page PDF to multiple images

convert multiple_pages.pdf +adjoin broken_up_%d.jpg

Prefix that with gm if you’re running GraphicsMagick rather than ImageMagick.

The %d part means that a number should be placed there to indicate the page number from the original that the image file contains. So a three-page PDF will produce


You can get a better default sort order on the files by telling *magick to zero-prefix the number with a slight change to the format string. So changing %d to %03d will starting numbering at 000.

You should be able to generate any image file type that *magick supports by just specifying the destination extension (jpg, png, gif, etc.).

join: the command

From the manual:

     join -- relational database operator

     join [-a file_number | -v file_number] [-e string] [-o list] [-t char]
          [-1 field] [-2 field] file1 file2

I had two CSVs, baz01.csv and baz02.csv. They shared the same first column, which was a list of database table names. The second column contained the number of rows from each table. The row numbers between the two files were different, and I wanted to compare them. The join command to the rescue!

join -t , -1 1 -2 1 baz01.csv baz02.csv \
  | awk -F, '{print $1","$2","$3", "($3 - $2)}'

gave me exactly what I wanted: the output contains the first identical column from both files, followed by column 2 of the baz01.csv, followed by the second column of baz02.csv, followed by the third column minus the second.

Of course, this will only work on the simplest CSV files, meaning no escaped or quoted commas allowed.

New green method of making concrete

This post on a product designer’s foray into making a much more Earth-friendly concrete gives me hope for humanity. It’s estimated that about 5% of greenhouse gasses are released due to the existing process of making the cement in traditional concrete. The new process, dubbed “Dupe,” uses approximately none (although I’m sure a fair amount does go into the mass production of urea, which is a major component of fertilizer).

Turns out, all you need is some sand, bacteria, calcium chloride and a decent amount of urea.

The reason I think Dupe is so much better is because it only requires “biological temperatures” (I suppose that means 80-100°F), as opposed to the thousands of degrees needed to make calcium carbonate and clays react to form cement. The only real downside I see at this point is that structurally, Dupe concrete is only about 2/3 as strong as traditional concrete. But come on, I know some smart materials science people must want to get in on this and try to make it even better.