Woxidu

It's too dangerous to go alone. Take this.

Archive for the "Nerdery" Category


Ghetto copy/paste

A lot of times, I SSH from one machine into another and end up in a shell maybe three or four ssh “links” away from my current machine. Then, innocently enough, I’d like to copy a file between that machine and my current machine. Invariably, there’s the pang of laziness as I contemplate how to hook up the necessary SSH pipeline so that I can perform the scp. And, since we all know that laziness is the father of invention, I was inspired to write some simple shell scripts.

ghettocopy:

#!/usr/bin/env bash

file=$1
if [ ! -f "$file" ]; then
    echo "usage: $0 <file>" 1>&2
    exit 1
fi

cat "$file" | bzip2 - | openssl enc -a

ghettopaste:

#!/usr/bin/env bash

out=${1:-$(mktemp ghetto.XXXXXX)}
openssl enc -d -a | bunzip2 - > $out

These scripts allow me to use the clipboard to copy/paste most reasonably-sized files. It won’t work on huge files (or big files that don’t compress very well) but it will simplify moving annoyingly large binary files. Enjoy!

Burn a VIDEO_TS folder to a DVD in Mac OS X

I had a surprisingly hard time finding this tonight, so for everyone’s benefit, here’s the “out of the box” way to get a VIDEO_TS folder onto a DVD. Sadly, it’s not really layperson-friendly.

  1. VIDEO_TS -> ISO
    hdiutil makehybrid \
    	-udf \
    	-udf-volume-name DVD_NAME \
    	-o ~/Desktop/movie.iso \
    	/path/to/VIDEO_TS/..
    
  2. Now just open the iso in Disk Utility and burn it to a DVD

Polling x.org’s keyboard state

Recently I needed to write a script that changed its behavior when the shift-key was held down. After some research to find a pre-made (and command-line-friendly) solution, I gave up and decided to rip open xkbwatch and see how it does it. It turned out not to be so scary and so I extracted the meaningful bits of code from it and condensed it into a more general-purpose tool. Let me know if I overlooked a really easy or simple way of doing this.

/*
** xkbdump.c
** Dumps the modifier keys onto the command line in a space-separated list.
*/
#include <stdio.h>
#include <X11/Xlib.h>
#include <X11/XKBlib.h>

int main()
{
    Display *dpy = XOpenDisplay(0);
    XkbStateRec state; // I don't fully understand this structure. 
                       // I only know that it reports my modifier keys.
    XkbGetState(dpy, XkbUseCoreKbd, &state);

    int i;
    for (i = 0; i < (sizeof(state.mods) * 8); ++i)
        if ((1 << i) & state.mods)
            printf("%d ", (1 << i) & state.mods);
    printf("\n");
    return 0;
}

The use-case I had imagined was something like this.

#!/usr/bin/env bash

SHIFT_KEYCODE=1
CTRL_KEYCODE=4
# ^ Inferred above just by running xkbdump with ctrl/shift held down

both_held=$(xkbdump | grep "\<$SHIFT_KEYCODE\>" | grep "\<$CTRL_KEYCODE\>");
shift_held=$(xkbdump | grep "\<$SHIFT_KEYCODE\>");
ctrl_held=$(xkbdump | grep "\<$CTRL_KEYCODE\>");

if [ "$both_held" ]; then echo "Both shift and control were held down";
elif [ "$shift_held" ]; then echo "Only shift was held down";
elif [ "$ctrl_held" ]; then echo "Only control was held down";
else echo "Neither shift nor control were held down";
fi

One last note. In order to build this, you need to tell gcc where to link against X11. Build with a gcc command like this.

gcc xkbdump.c -o xkbdump -lX11

Bash Array Tutorial

Over the past two years at my job at OkCupid, I’ve collected a few useful nuggets of bash know-how. These days, my coworkers come to me frequently and ask for help with solving various problems with this tool and so I wanted to share some of the pieces here. This is going to show off how to make use of bash arrays to make managing lists easier in bash.

Our project is going to be a program that reads numbers from standard input and then writes the trailing average of those numbers onto standard output. It should take one integer parameter for the number of trail points it should average together. If it hasn’t read enough to average all of the trail points, then it should just average everything. That should be enough to get started.

The basic structure of the program will be the main input loop, so let’s get that going:

#!/usr/bin/env bash

# How many numbers to average at a time
traillength=$1;

# Main input loop
while read number; do
    # TODO: Put useful code here.
done

We’ll also need the trail length parameter, as well as a queue of numbers that we want to average. We’ll use a bash array for that part.

# How many numbers to average at a time
traillength=$1;

# A bash array
queue=();

The parentheses tell bash to let us access $queue with some special array syntax.

Next we’re going to want to append $number onto $queue at the start of ever loop iteration.

    # Append the next number onto the array
    queue=(${queue[*]} $number);

The same parentheses notation is telling bash to treat the contents of the parentheses as members of an array. ${queue[*]} tells bash to give us the entire contents of the array separated by spaces (or whatever is in your $IFS variable — which we’ll make use of in just a second).

Anyway, the next step is to add up all of the things in our list. Since bash can’t do floating-point arithmetic, we’ll need to make use of bc, the arbitrary precision calculator for unix. We’re going to accomplish this by echoing the expression we want evaluated into bc and then saving its answer. The general form for such a maneuver looks like this:

answer=$(echo "1+1" | bc);

So that’s simple enough. But we want something a little fancier. In order to tell bc that we want to do floating-point arithmetic, we need to set its scale so that it knows how many decimal points to use for output.

answer=$(echo "scale=2; 12.5 + 3.14" | bc);

Finally, we need to actually get the numbers in our queue in there. This is where it gets tricky because we need to get the + signs in between all of our numbers. This is where the $IFS variable comes in. $IFS (Internal Field Separator) tells bash what value to use to separate lines and words within data. The result is:

    # Add up all of the numbers in the array
    _IFS="$IFS"; # Save old IFS value
    IFS="+";
    total=$(echo "scale=4; ${queue[*]}" | bc);
    IFS="$_IFS"; # Restore old IFS value

NOTE: The @ special index can be used in many of the places where the * index can be used. THIS IS NOT ONE OF THEM! In fact, @ and * are only different in how they behave inside of quotes and how they respect the $IFS variable.

So that will sum up all of the things in our queue. Now we have half of the formula. The other half is the total number of things in the queue. In bash, we can get this value with the following syntax:

    size=${#queue[*]};

Now we can use bc to get the average.

    echo "scale=4; $total/$size" | bc;

The last step is to actually limit the queue size. On each loop iteration, we’re going to check the queue size. If it’s bigger than $traillength then we’re going to chop off the first element. That step looks like this:

    # If we've gone over our specified array size...
    if [ "$size" -gt "$traillength" ]; then
        # Lop off the first (oldest) element of the array
        queue=(${queue[*]:1});
    fi

We’ve made use of one final piece of tricky bash-array syntax here. The colon parameter expansion operator lets us specify an index offset for the array. The part inside the parentheses, ${queue[*]:1} expands to the entire array, except starting at index 1. The parentheses behave just like they have above.

And Voila! We’re done! The full script looks something like this:

#!/usr/bin/env bash

# Watch a stream of numbers from standard input and print out the N-point
# trailing average of those numbers

# How many numbers to average at a time
traillength=$1;

# A bash array
queue=();

while read number;
do
    # Append the next number onto the array
    queue=(${queue[*]} $number);

    # Count up all the things in the array
    size=${#queue[*]};
    
    # If we've gone over our specified array size...
    if [ "$size" -gt "$traillength" ]; then
        # Lop off the first (oldest) element of the array
        queue=(${queue[*]:1});
    fi

    # Add up all of the numbers in the array
    _IFS="$IFS";
    IFS="+";
    total=$(echo "scale=4; ${queue[*]}" | bc);
    IFS="$_IFS";

    # Count them up again
    size=${#queue[*]};

    echo "scale=4; $total/$size" | bc;
done

Final Thoughts

Most people who know me well know that I hate perl for two main reasons. The first is that it’s capable of some VERY scary looking syntax. The second and arguably more important reason is that, in my experience, the most widely-held best practices for writing perl involve using these exact same unintelligible pieces of syntax. I don’t write Perl because I have a choice and have chosen not to.

When it comes to bash, my choice is much harder to make. As you can plainly see above, bash is capable of some seriously busted syntax. I don’t wish these hateful strings of punctuation on any more than is necessary, but there’s just so much less to chose from in the world of shells. At the end of the day, your options are sh, bash, csh, tsh, ksh, and zsh. Many of those aren’t going to be installed on any/all machines that you need to use in your geeky existence. One of my coworkers uses zsh and constantly needs to bug our sysadmin to make sure it’s installed everywhere.

If you want a portable command-line experience, you’re relegated to the feature-sparse sh, or bash. This is why I think it’s worth making friends with bash and its otherwise unsightly syntax. Knowing some bash-fu moves can save you from doing lots of tedious text processing by hand, and can make you feel at-home and much more comfortable with such a ubiquitous tool.

PyMacDialog

Download it

PyMacDialog is a small python script that reads a list of options from stdin, gives you a gui picker, and writes your selection to stdout. It serves a similar purpose to applications like dialog or CocoaDialog. The latter almost did exactly what I wanted except it only allows dropdown menus, not full lists (Using the NSTableView class).

After discovering PyObjC and seeing how easy it could be to write code that was both command-line friendly and Cocoa friendly, I decided to write PyMacDialog. The trickiest part was rooting through message boards and other open-sourced code to find out how to construct the UI purely within python without making use of any NIB files. I hope to put together some tutorials to illustrate the different parts.

If I have some command that generates some kind of list:

[eliman@dian-nao]-[scratch] $ ps auxww | grep -v grep |  grep Applications |\
awk '{print $11}' | awk -F'/' '{print $NF}'
iPulse
Terminal
Safari
Mail
Dropbox
iTunes
Twitterrific
iCal
Colloquy
Adium
HardwareGrowler
Caffeine
iTunesHelper
Vim

I can pipe this command through PyMacDialog.py and get this:

PyMacDialog

And then I can pick one, hit enter, and get this:

[eliman@dian-nao]-[scratch] $ ps auxww | grep -v grep |  grep Applications | \
awk '{print $11}' | awk -F'/' '{print $NF}' | ./PyMacDialog.py 
Dropbox
[eliman@dian-nao]-[scratch] $ 

Vim in OS X

When I’m working in Terminal and want to edit something quickly, I’m most likely to use vim. But if I’m working on a big project and writing lots of code, vim tends to be pretty cumbersome. Terminal keeps vim from being able to use any mouse input, which starts to get in the way after a while. MacVim provides a OS X GUI for vim, but suffers from bad scrolling latency problems as well as other issues. Recently, another contender has stepped onto the scene. vim-cocoa aims to be a simpler, faster alternative. My only qualm so far has been that the distributed binaries don’t have cscope enabled by default. Here are some simple instructions for anyone wanting to use cscope with vim-cocoa.


> git clone git://repo.or.cz/vim-cocoa.git
> cd vim-cocoa/src
> ./bootstrap.sh
> ./configure --enable-multibyte --enable-gui=cocoa --enable-cscope
> make
> sudo make install

Readability

I am anal about readable code. I think about it a lot. How code is spaced, how different variables and functions are named, how the abstraction I’m using is reflected in the way the code actually appears. These are all very important to me.

Most people who know me also know that I am a believer that Perl is a great language for writing supremely unreadable code. It always sort of seemed like the use of confusing syntax was the accepted “best practice” form of doing anything in the language. It’s now clear to me that perl was designed to promote confusion as the norm, and to offer readability as an afterthought. (more…)

Fuck You Zibri

I strongly recommend against using Zibri’s ZiPhone in order to jailbreak your iPhone. His “Fix Wi-Fi Issue” feature does you the favor of changing your MAC address after setting the following environment variable:

setenv wifiaddr “00:5a:49:42:52:49″

Awesome. Thanks so much. I always wanted to be MAC-address twins with all my best iPhone buds. Now we get to share the one IP that our office router is content to give the two of us.

Fixing it

Fortunately, I found this link that worked to get things back in working order: http://www.hackint0sh.org/forum/showthread.php?t=36078. It’s not pretty. I don’t see why I needed to fuck with my baseband in order to undo this crap.

Conclusions

I hate you, Zibri.

Tactlessly Loud

Vocals

I have a really big voice. It’s been that way since high school. While most wouldn’t say it’s harsh or abrasive, I never have any trouble being heard. In a crowd, on a busy street, or from the back of the lecture hall, I stand out. Clearly this has its advantages. My thoughts and opinions generally will be heard and acknowledged. I’m hard to ignore if I’m set on expressing myself. And like most things, there’s disadvantages. Since I’m never not heard, there’s no room for me to hold a “private” conversation with anyone in a public place unless I’m whispering conspicuously. I think that a consequence of growing up like this is that I tend to assume that everyone can always hear everything I say and so I should only say things that are okay to say around everybody. Sometimes I slip up and then everyone hears something awkward or inappropriate.

Work

I’m working on the new-school testing engine for OkCupid. We’re seeing lots of traffic lately and an overhaul of our test system is overdue. I can’t give too much away, but the thing that we have in the works is very slick. Much respect to Chris Coyne’s work in laying the groundwork for it.

Storage/Media

I think the solution to my storage/media problem is easier than I thought. It turns out that a software hack exists for the Apple TV “Take 2″ which involves making a bootable Patch Stick. The instructions look scary, but lots of people seem to be having luck with it, so I think it’s worth the risk. I’ve already bought a 1 Terabyte MyBook World drive with Ethernet and USB connectivity. It should arrive on Monday. Once I get the drive working like I want it, I’ll add an Apple TV to the mix and try to bring it all together. Updates to come.

Misc

  • I just bought CoverSutra and it rocks. I was baited by the free sample I got via MacHeist and the search menu was enough to sell me on the upgrade.
  • I’m working on an OS X Chinese – English dictionary called Cidian. It’s based on CEDICT, the same source used by MandarinTools.com. MarndarinTools.com is great, but its UI is pretty tiresome.
  • Much respect to my old co-worker Aston Motes and his friends at GetDropBox.com. They’re making headlines in some pretty visible places these days. I’ve seen them on the Digg homepage, TechCrunch, and even the venerable DaringFireball.net. Good to see startups with talented people get the attention they deserve.

Apple TV 2.0

I have a scratch that I need itched. My roommate and I have a bit over 100 GB of music, I have a bit over 300 GB of Movies and TV Shows (in various formats), and I have tons of photos and images (National Geographic, NASA, Salvador Dali, etc) that I’d like to have available for enjoying in my living room. On top of that, I’d like to be able to review menus of local restaurants and have a file server available to everyone on our wireless network. Since Apple announced the Apple TV 2.0, I’ve dreamt that it’s the piece of technology to scratch that itch, but reviews I’ve read recently leave me unconvinced that it will do everything I want it to do. My concerns boil down to the following:

  • Re-encoding all of my movies/TV shows
  • Maxing out the Apple TV’s built-in storage and not being able to expand it easily
  • Resorting to physical hacks in order to get network services on it (SSH, Samba, AFP, etc)

I could spend a few hundred more dollars and get a Mac Mini, but then I don’t think I’ll be able to use the remote to rent movies very easily. At $299, it’s probably worth it for what it will bring to the table. If it can’t be easily hacked to take a larger hard drive, it might be worth it for me to get a beefier NAS (drobo, anyone?) device to live someplace where I don’t care about noise (my closet) and then sync stuff from that onto the Apple TV using my computer, though I’d rather avoid needing an intermediate computer for this. By far, the most worrisome part of the Apple TV is that it simply can’t (and won’t) be able to fit all of my media. For anyone else who’s been thinking along these lines also, here are some links to Apple TV 2.0 reviews that I’ve collected:

UPDATE [03/12/08]

Ars Technica has released their ‘Mano a Mano‘ shootout between the XBox 360 and the Apple TV. It’s a great review, but sadly doesn’t really leave me with a solution that I’m happy with. I’m still leaning towards the Apple TV, and some software hacks for it would be very welcomed. I’ll research more and post later about it.