edunham

Time Off, Time On

2026-03-02T00:00:00+00:00

For awhile, devrel was the best work I could do. The world changed, and now it isn’t.

After 4.5 years at Okta, today is my last day. I’m leaving a good role on a good team, with greater potential than I materialized from it. More on that some other time, probably with a link to the backfill req, because I would sincerely recommend it to someone who’s at the right place in their career. But it’s not what I’m at the right place in my career for any more. I don’t have what I need to do my best possible work right now, and what I need starts with a set of systems that only I can build.

Systems, done right, take more effort to build than to maintain. The scope of what systems it’s possible for me to build with the tools at my disposal is greater now than it’s ever been before,and some of my deepest curiosities surround the question of what I can make of myself with the resources I’ve gathered. These resources now include access to a variety of wish-granting machines, and I already know that there are many chains of small, carefully-scoped wishes grantable by those machines in their current form, which reshape my environment in a way that reshapes my abilities. I don’t know yet where that line of inquiry ends up, but it’s time to find out.

I don’t usually talk about money here, but it’d be irresponsible to omit at least a passing mention of how it’s normally a terrible idea to leave a role without a next salary lined up. In a win for impostor syndrome, spending a decade in the tech industry with the expectation that it would kick me out at any moment has created a financial situation of artisinal home-made privilege in which it’s neither reckless nor stupid for me to take some time off. You can buy all kinds of things by saving up enough money, and what I’m buying for myself this year is time.

Collaboration

2024-08-15T00:00:00+00:00

Back in high school, I built robots with a group of exactly the sort of teenagers you’d expect to find in a robotics club. I was physically weaker than most of my peers, so I had to use tools and leverage to complete routine tasks that others could do with their bare hands. This gave me more practice with those tools than others were getting. When the tasks scaled up in difficulty – a stuck bolt turning out to be very stuck, a thicker sheet of metal turning out to be unusually reluctant to bend – my tool use tended to scale up quickly thanks to my constant practice with it, whereas peers who’d been brute forcing the task before would face the initial learning curve of the tool at that moment.

I think I’m doing a similar thing with collaboration. I have to try really hard to keep in touch with people because my default state includes mostly avoiding everyone. But by trying on purpose, I seem to have accidentally developed systems which scale better than the innate strengths which many take for granted in themselves.

And I’m writing about trying on purpose, weird though it feels, because several people have recently remarked on the quality of my collaboration. What they perceive as collaborativeness, I experience as a desire to work with my friends, and a recognition that the most effective way to do this is to make friends among my colleagues.

The System</h1>
In transitioning to involuntarily remote companies, I noticed a decline in the quality of my work-related social life compared to when I was at in-person or intentionally remote organizations. The intentionally remote culture of Mozilla was built around twice-yearly week-long offsites for the entire organization, which served in part as summer-camp-like friendship accelerators. In-person workspaces offer richer and clearer signaling about social interest: “we’re going to lunch, wanna join us?” grows into invitations to mutually interesting events outside of work as common interests are discovered.
I like working with my friends, and many other people do too. To get more of that, I’ve trial-and-errored my way into a system of making friends at work that’s working well for me so far.

Identify Likeminded Individuals</h1>
First, I taught myself to notice when I’ve met someone at work whose company I enjoy. Are meetings more interesting when they weigh in with their perspectives? Do we get to chatting about ordinary topics when we cross paths in larger meetings, and feel disappointed instead of relieved when the conversation returns to the meeting’s actual agenda? Are they interesting to speak with socially when we meet at in-person work events? Is there something about their work, or their approach to work, which I find especially interesting and want to learn more about? Do they express reluctance or regret at ending conversations about topics which interest us both, beyond the mere rote pleasantries?
In short, if we worked in the same office and I saw that they were hanging out in a common area, would their presence make me more likely to go hang out in that area too? These are all ways that my brain tells me someone might be a good candidate to upgrade from acquaintance to friend. Other brains may signal the same potential through a different set of feelings; the trick to this step is discovering what your pattern-match on promising individuals feels like.
If you’re recreating my system for yourself, it would probably be helpful to reflect on other times when you’ve developed friendships with colleagues. What was different about interacting with them, versus interacting with the comparable colleagues who didn’t end up as friends? That’s what you’re looking for.
At this step of the process, being on the same team as me counts as a big enough shared interest to make the system worth trying.

Check Compatibility</h1>
To find out how others feel, I rely heavily on the words that they say. Some people are better at picking up subtler cues, but in my experience, even relying on tone of voice can be hit-or-miss.
People can say one thing while meaning another for many reasons, like manners and people-pleasing, and can have wonderful friendships despite that trait! They just don’t tend to have those wonderful friendships with me, because the more we interact socially, the more likely I am to harm them by taking actions which are appropriate based on the content of their speech but inappropriate based on its context.
In the same vein, I like to wait on aggressively pursuing friendship with someone until I’ve seen how they say “no” to optional tasks they don’t want to do. Once I’ve seen them say “no” when a “yes” would have been easier, I know they’re the kind of person who will tell me to back off if I’m being annoying or burdensome.
Those are just my personal compatibility checks. Yours might be similar, or they might be opposite – you might prefer people who will notice subtle changes to your mood without being told, for instance!

Ask the Awkward Question</h1>
Once I identify a prospective work friend and verify that their social habits are compatible with mine, I ask them if they’d like to meet more often. This sounds like the most awkward thing ever, but it’s actually not so bad.
The best time to ask if they’d like to meet more often is when we’re already having a good conversation for some other reason. When they are clearly enjoying a conversation with me, it’s unambiguous that they at least sometimes enjoy my company, so it’s a perfect moment to open an invitation to make interactions more frequent.
The question goes something like this: “I wish we did this more often! How would you feel about having a recurring 1:1 chat? Maybe every few weeks?”
The frequency depends on your schedule and how much overlap there is in your work. I have some work friends that I catch up with quarterly, others monthly, others fortnightly.

Set Expectations</h1>
I like to set a recurring calendar invite for 1:1s, and be sure that the invitee has “modify event” permissions on it.
My event description usually reads something like, “If we worked in the same office, I would enjoy chatting with you in the kitchen from time to time. But we’re remote, so a call is the next best thing. Please feel free to delete or reschedule this meeting any time it would make your day more difficult.”
I don’t personally use agendas for these meetings. If developing an agenda together is a fun social activity to share with a new friend, though, there’s nothing wrong with that!

Leave a Graceful Exit Open</h1>
I don’t like worrying about whether I’m pressuring someone to hang out when they’d prefer not to, so if they cancel several times, I check in about the cadence. “It seems like your schedule has been packed recently; how about I cancel this invite? If you have more time later and want to meet, I’d still be interested!”
For the first few 1:1s with someone, I make sure to have something scheduled to start immediately after the calendar invite’s end time, even if it’s just an alarm on my phone. I know that when I get really into a fun conversation, I can lose track of time and then have a more stressful day as a result. I don’t want to make my friends’ days worse, so as the instigator, I take ownership of ending at the agreed end time as a default.
Some people turn out to tolerate or even prefer these meetings running long, sometimes very long. One work friend who shares my interests in career strategy and loves mentoring will enjoy letting our 30 minutes become 2 hours if we’re both free. Regardless, I consider it polite to check in at the scheduled end time and see when we each need to be someplace else.

In The Meeting</h1>
So, you’ve got this person in a call. What do you say?
I personally ask some variant of “how are you doing?” or “how is it going?”. This is their excuse to start talking about whatever is on their mind.
Sometimes people will reflexively respond “good, and you?”. The “and you?” is my invitation to lay out several options for conversation topic, like trying different fishing lures to see what’s of interest today.
To prepare for the meeting, I like to think about 1-2 personal things and 2-3 work things which seem like they might interest the person. Personal things might be “I’ve finally figured out how to get my cat to tolerate existing in the same room as me”, or “I’m looking forward to finally planting tomatoes in my garden”, or “I’m finally getting around to watching such and such a show”. Professional things might be “I’m super heads down in such and such a project”, or “I just found out that this change is coming so now I get to figure out how that impacts my other work”, or similar.
Answering “how are you?” with concise references to 1 personal thing and 1-2 work things gives the person a ton of options. They can request more information on something that they would enjoy hearing about, or offer a related story of their own, or change the topic to something they find more compelling than anything I brought up.
Then again, I don’t hate small talk like some people do. I find that small talk is a challenging game of identifying common interests, almost like wiki-racing</a>. When someone shares information about the weather, it’s a thinly veiled offer to share more about their hobbies and interests: What is it too hot, or too cold, or just right to do? I hold the opinion that small talk is the bootloader for interesting conversation.
Another excellent topic of conversation is to lend and borrow expertise. One of my work friendships is with a product designer, and part of the fun of talking to her is how our areas of greater and lesser expertise line up so compatibly. She answers my “why is it like this?” inquiries, while I answer her “what’s it like to actually use this thing?”, to great mutual benefit.

Afterwards</h1>
If you had a nice time, do it again, the next time it shows up on your calendar at a time that works for you both! If you didn't have a nice time, change plans as necessary so that next time will be better.
If a work friend or even just a friendly collaborator moves on to another opportunity, I like to ask them who else in their part of the organization I should know. Then I can reach out to those people: “such and so told me you’re a good person to know; would you mind a quick call?”. The topic of conversation for that call is pre-established: What you both like about the person who’s leaving, and what mutually beneficial involvements you had with them that you’d like to replace. Replacement is most politely framed as “I’ll really miss how…”, but that’s basically an offer to invite them into your little part of the vacuum left by the departing individual.
I try to remember to intermittently ask the “who else should I know?” question of my work friends, but I rarely remember to. It’s ultimately not essential while someone is still a colleague, but it’s a very important aspect of a good goodbye.
Hi, Future Me</h1>
I hope that someday I’ll look back on this post and go “wow, you sure were doing things the difficult and inefficient way!”. If there’s better ways of accomplishing the outcomes of this system that are working for you, please don’t regress to my way of doing things.
But several people in the past few weeks have expressed admiration at the level of connectedness I’ve built within my present workplace, so it seemed worth writing up my current state of the art on the topic.

Assorted Git & Unix Tips

2024-05-03T00:00:00+00:00

In my present role, I get to work with colleagues of delightfully diverse skill sets. Sometimes I'm learning directly from experts, and other times I get to learn by teaching in areas where I happen to have the local maximum of experience. One delightful colleague has a coding bootcamp background which rapidly exposed them to a lot of important topics, and I find myself helping to fill in the gaps where a couple hours of instruction don't quite get the point across like years of practice.

I'm writing this to capture, in no particular order, some advice that I'm noticing myself giving consistently. I can only guess at how I accumulated it, and I cannot promise that there won't be exceptions to any of these "rules". But I suspect someday in the future I'll be interested in looking back on notes like this, so onto the blog they go!

Git Concepts</h1>

Know where you are. Care constantly about what branch you are presently on, because many commands make changes relative to your current location.</li>
Speak with pedantic precision about precisely what you're referring to. When referring to somewhere you might push to or pull from, a description has 3 parts: the location or owner (a github org, a github user, or your local checkout), the repository name, and the branch name. If I say "there's cake in the kitchen", you'll have to guess which kitchen I mean: mine? yours? the office? Similarly, if you say "on the main branch", I'll have to guess from context which main branch: your laptop? your fork of the repo on github? the upstream repository you forked from?</li>
Commit IDs do not matter to you, but they matter to other people. If you're the only person who has a commit, you can safely take actions (like pull --rebase) which change its ID. But as soon as anybody else has that commit, it is very rude to do anything to that commit which changes its ID.</li>
There is a social distinction between expectations for a kitchen and a dining room, a factory floor and a showroom, a studio and an art gallery, which parallels the distinction between your fork of a repository versus the origin. Your fork may be shared publicly, but it's the kitchen or factory or studio, where nobody should be surprised if they find half-completed or not-yet-working code. The origin, on the other hand, is the dining room or showroom or gallery -- some but not all products of the kitchen or factory or studio pass a quality control process to be put on display and appreciated by the public.</li>
Version control gives you time travel powers. To use these powers, you have to learn to identify moments that you'll want to travel back to. If you already have an intuition for when to quicksave in a video game, use it. Every commit is a quicksave, and you get an unlimited number of them. If you already have an intuition for how to name your save files in a game so that you can go back to the one you wanted, use that for crafting helpful commit messages. As with any challenge of organization, think about the situation you'll be in when you later wish to use the commit you're making now. What will future-you search for when trying to find this commit?</li>
To take useful notes for yourself, it's important to be more pedantically precise than you'd expect. Recording what commands you used isn't enough -- in order to use the notes later, you must specify in a way that makes sense to you exactly what task the commands accomplished. If it seems like there's another way to do the task, ask why that way wouldn't work. Sometimes the other way would work fine, and other times there's some horrible gotcha that would break it.</li> </ul>
Unix Concepts</h1>

Know where you are. Care constantly about which directory you are currently in, and where you have put your various files.</li>
When someone spells out a command verbally including flags, do not expect that they'll want you to type a space after the "dash" or "tac". Sometimes there'll be a bare - with no space afterwards, but that's so unusual that a speaker will pause longer or even verbalize the space. It's easy to assume that a listener knows the format of a tool's arguments, but why would they if they aren't acquainted with it yet?</li>
Changes made in one shell session will not automatically propagate to others. The point of the config file (/.bashrc, </del>/.zshrc, etc) is to get each new shell up to speed on what you expect it to know -- not unlike how you might add common requests to every prompt you give an AI. (the config file is just a list of commands that run automatically every time a shell is opened)</li>
It's all files. Turtles all the way down. I'm starting to understand why my own education started at "everything's a file" -- that information wasn't useful in the moment, but it created a priming effect where I approaced everything with the expectation that it's a file afterwards. The first questions to ask when troubleshooting are questions that you can ask about a file. You cannot ask all that many questions about files: Where is it? What's in it? What are its permissions? Addressing those fundamentals catches annoyingly many problems.</li> </ul>
Incomplete list, caveat emptor, and you get what you pay for on the anachronism of a free blog. But hey, it's human-written text, and that's an endangered species of critter on the modern web. I suspect we'll eventually develop good tools for threshing the human-generated prospective training data from the early-GPT-generated chaff of the 2024 web, so hello to whatever future model ends up training on this!

DIY Shoe Chains

2024-01-14T00:00:00+00:00

The Pacific Northwest is dangerously frozen at the moment. Other regions handle conditions like this just fine. But a freeze like this is especially dangerous to us because most people are unprepared for it.

Right now we've got a bunch of frozen sleet on the ground that looks like snow, but offers the traction of an ice skating rink. This morning I thought I could cross the "snow" in regular shoes, and fell (embarrassingly, but non-injuriously) immediately. I was able to go inside and put on my shoe chains because I keep a pair on hand for occasions like this, and with chains the ice was as easy to walk on as snow would be. But it got me thinking about people who might not have thought ahead to own shoe chains. You can't just order a pair for use today; nobody's delivering anything in this weather.

So I did a quick experiment to see whether adequate shoe chains could be assembled out of stuff that a normal person might have on hand. All the commercial ones really are, after all, is a piece of something stretchy and some chain. Image below the fold.

Shoe chains are just a stretchy part and a traction part. The traction part goes under the shoe and digs into the ice when you walk. The stretchy part keeps the traction part in place, and makes the whole assembly easier to put onto the shoe.

I made the stretchy part from some old elastic salvaged from a fitted sheet that was on its way to the sewing stash, but you could use elastic from anywhere. If you have a lot of hair ties, you could connect them end to end and make a loop the right size. If you have a lot of rubber bands, you could use those (just be aware that the stretchy part breaking would mean the chains fall off your shoe!). If you didn't have anything stretchy, you could use an old shoelace and then tie it securely in place. If you used something without stretch, you'd have to get the knot untied any time you wanted to remove the chains from your shoes. The size of the stretchy part should be large enough so it fits over the biggest part of the sole of your shoe, but small enough so that it doesn't fall off.

Chain works great for the traction part of the shoe chains. I used a piece of flimsy anodized aluminum chain that used to be part of some accessory; I forget if it was a belt or a purse. A steel chain necklace would work great, as would a spare wallet chain. The prototype with the chain stayed in place really nicely on my shoes when I tested it. I attached the chain to the elastic by threading the elastic through the chain links, so it made a zig-zag of chain in the loop of elastic. If the elastic hadn't fit through the chain, I could have used safety pins or paperclips to connect them, or sewn it. The size of the chain should be long enough to get from one side of your shoe sole to the other, but short enough that it stays in place and doesn't leave a lot of slack.

I also tested plastic mardi gras beads instead of chain. They gave me more traction than an un-chained shoe, but the round beads tended to roll about and move out of position on the shoe. I had to adjust the mardi gras bead prototype several times while walking around, and I didn't have to adjust the chain one at all. If you're building shoe chains out of beads, you'll probably need to put more effort into getting them to stay where you want them. The nice thing about plastic beads is that, if you cross the string parts and twist them, you can easily attach any part of the strand to any other part.

All the usual disclaimers apply -- be safe, use common sense, don't sue me if this gives you an idea that gets you hurt. But if you do have to get from point A to point B on slippery ice, think about what resources you've got at your disposal for making that safer.

Making Dice

2024-01-02T00:00:00+00:00

If you're here for pretty pictures of dice, prepare to be disappointed. Making adequate dice is easy, but taking good photos of them exceeds my current skills.

In today's standard "how I spent my winter vacation" small talk, I showed some colleagues a dice set that I made last week, and they seemed surprised when I explained that it was relatively easy.

Making excellent or perfect dice is not easy, but I'm not trying for excellent or perfect. I'm trying for "nice to look at" and "capable of showing random-feeling numbers when I roll them". Those goals are easy to achieve with cheap products from the internet.

..more:

If you want to learn to make good dice, go watch Rybonator</a> on YouTube, and the algorithm will start suggesting other good channels as well.

Next I'm going to share some Amazon links, and a note of the approximate price at time of writing. They aren't affiliate links because signing up for the affiliate program requires more paperwork than I feel like doing right now. But they are the specific items I've been using to get pretty-okay dice out of.

The basic materials you'll need to make dice are:

Retroreflectors & Storing Ground Glass

2023-03-14T00:00:00+00:00

Retroreflectors</a> are fun. I finally got around to picking up some cheap glass blast media today (mine's the 40/70 grit recycled bottle glass from Harbor Freight / Central Pneumatic) and did some testing with various paints and glues that I had lying around. I'm using it as retroreflective beads. When I hear "beads" I think of things with holes in them for putting on a string, but in this case it means more like beads of condensation -- tiny round blobs. They feel gritty like beach sand, and being made from clear glass, they look like unusually sparkly white sand as well.

Roughly a decade ago I stumbled across a PDF that did a good job of explaining the physics of glass bead retroreflectors, which I can of course no longer find. It was published by some highway department and contained extreme detail about the temperature tolerances for applying retroreflective beads to the fog lines on roads, because the retroreflective properties only work if the glass beads are embedded just over halfway into the paint. If the beads sink too far into the paint they can't work their light-bending physics magic, and if they're embedded too shallowly into the paint they'll easily come un-stuck from it.

Mental Model</h1>
Here's how I think about bead-type retroreflectors. Physics means that the glass beads basically bounce the light straight back toward where it came from, so the stick figure is holding the flashlight close to its eyes for maximum testing effectiveness over short distances. The beads are obviously not to scale.

The beads have a different index of refraction from the surrounding air. That just means that light bends at a certain angle when it goes from air to glass, and it bends at that angle again when it goes from glass to air on its way out.
The surface of the bead that's covered in paint reflects light, perhaps only some of the light depending on the color of the paint. White paint reflects the most light, of course.
The light goes into the bead, bends a little, bounces off the paint, goes almost straight back through the bead the way it came, bends a little when it hits the air, and ends up going more or less straight back toward the light source.
I've drawn the light in orange before it bounces off the paint and in yellow after it bounces off, to kind of illustrate how it behaves differently hitting the painted floor when it's gone through a retroreflective bead versus when it's just gone through the air.
Storing the blast media</h1>

The one problem with even 5lbs of ground glass is that it does its utmost to get everywhere. If the bottle is ever inverted with the lid on and the lid gets bumped, the sand-like glass ends up everywhere the next time you take the lid off.
Plastic bottles like this come with a "safety" seal when I buy food stuff like spices in them, and I've never had a problem with those getting all up in the cap and spilling everywhere.
I tried ironing a scrap of mylar food packaging (a fruit snacks bag was most conveniently accessible) onto the plastic bottle, and it stuck nicely! I used my clothes iron on the Cotton setting. Now the beads will stay in the jar until I want to use them again.
Paint/Glue Testing</h1>
I put various sorts of paint and glue onto popsicle sticks and sprinkled them with the blast media. I then put them on the door of a room with good blackout curtains, took a photo, turned out the lights, and took a flash photo.
I used a commercial plastic retroreflector in the first two photos as a control for how well the DIY ones were working.
Here's how it looked with the lights on:

From top to bottom, the samples are:

Dark green puff paint / fabric paint</li>
Black puff paint / fabric paint</li>
Superglue</li>
modpodge</li>
clear nail polish</li>
dark red nail polish</li>
1 coat white nail polish, then half white half clear nail polish with beads on the second coat</li>
white nail polish</li>
black nail polish</li>
black acrylic paint</li>
white acrylic paint</li>
metallic gold-ish paint stuff</li> </ul>
Everything except the two-coat enamel test had the glass beads poured directly onto the one layer of the paint or glue. Yes, I labeled modpodge upside down. I wasn't planning to take photos at first but then I realized if I was going to post about it I probably should.
Here's how that setup looks in a flash photo from maybe 10' away, in a dark room:

Here's how it looks from the same distance, still a dark room, but I zoomed in more:

And here's how it looks, still zoomed in, but with the commercial reflector removed:

Conclusions</h1>
The cheapest glass sandblast media that I could get ahold of works surprisingly well as a retroreflector when applied to white paint. The performance difference between beads on white and beads on darker colors was surprising. The clear glues (superglue and mod podge) worked better than the darker colored paints, which was a bit surprising.
My biggest surprise was how the white sample almost blended in with the white door in ambient lighting, yet stood out so much in the dark.
Further stuff to test with this includes doing the glue thing on a darker background color, and trying various types of clear topcoat over the bead layer. I don't think those will work, but then again I did think the black paint might work, so it's worth testing.

DIY Thimble

2022-04-05T00:00:00+00:00

Over the past couple weeks, my schedule has had a higher than usual concentration of the kind of meetings where one sits off-camera and listens to a presenter talk. Like many engineers who knit in meetings, I find that keeping my hands busy helps me focus. Knitting puts me on the losing side of a battle between "don't drop any stitches" and the laws of physics, however, so instead I've been hand sewing quite a bit.

I've been aware for some time that sewing with a thimble is Objectively Better</a> than sewing without, but I somehow made it to adulthood without learning to use a thimble properly, let alone avoid losing one between setting a project down one month and picking it up again the next. I have DIY'd a lot of various leather thimbles over the years to try to sew with them, but the designs have been a hassle to assemble, non-functional, or both (sewing a leather thimble is a chicken and egg problem: you need one to make one without great inconvenience or pain). However, I have finally stumbled across a design that doesn't fail or annoy me in the ways that all the previous ones did.

It's embarrassingly easy to make. By embarrassingly, I mean "why didn't I figure this out decades ago?!" All you need is some surgical tape, and a piece of leather about as wide as your finger and long enough to wrap over your fingertip.

![image](/pictures/tape-wrap.png

1. Wrap your finger in a couple layers of surgical tape, sticky side out. You want it tight enough to not fall off too easily, but loose enough to slip on and off later.

![image](/pictures/leather-overlay.png

2. Put the piece of leather onto your finger over the tape, so it covers the spot you keep accidentally jabbing with the needle when you fail to use a thimble.

![image](/pictures/final-wrap.png

Wrap the whole thing with a couple layers of surgical tape, sticky side in.</li> </ol>
![image](/pictures/finished-thimble.png
That's literally all there is to it. If you don't like using leather, cardboard might work, or any plastic that's flexible but sturdy enough to be hard to jab a needle through.
The one of these that I made earlier in the week and have been using ever since has molded to the shape of my finger and only gotten more comfortable over time.
![image](/pictures/before-after.png
P.S. That's a heavy silk jacquard, pretty on both sides, 28" wide without good selvedges but they</a> currently have it on Please Go Away sale for $3.13/yard. The fibers burn and smell like silk, and it feels like silk, so I don't think they're lying about the composition. The selvedges aren't too nice, one side is fuzzy and the other seems to have just been cut, but for a price like that I can't complain. The orange is more coppery in natural light than the photos make it look on my monitor.

Lumenator

2022-03-18T00:00:00+00:00

About a year ago, I found out about lumenators</a>. The theory is that if you put sunshine-ish amounts of light onto a creature, the creature reacts as it would in sunlight.

So, I built one. It's technically brighter than the sun.

Mine is composed of 48 e26 sockets mounted in a 2'x4' piece of plywood, with some switches that allow the rows to be turned on and off independently. It currently plugs into the wall, although I might eventually get around to hardwiring it if I ever get serious about cable management.

Each of the 48 sockets is intended to hold an LED bulb of around 2500 lumens. I use a mix of color temperatures, with most bulbs 5000K but some 3000K. This means that with all the bulbs in and all the rows lit, it emits around 120,000 lumens from approximately 0.74 square meters, or about 162,162 lux. According to Wikipedia, sunlight is around 100,000 lux once you account for the atmosphere.

Here's how it looked with all the bulbs in it while I was testing it on my kitchen table. This photo was taken around 3pm in early April, 2021. That's midafternoon springtime daylight for comparison, visible out through the window. Yes, I was testing it by sticking the exposed wires into a C13. No, it didn't catch on fire. No, you should not try this at home.

I've mounted the board of sockets onto the wall above my desk, so turning it on is like opening a variably sunny window. I typically use about 12-24 of the bulbs on a day to day basis, as more can make it bright enough to be slightly uncomfortable.

I put off writing this post for a year because I never actually got around to filling in all 48 of the lamp sockets. As well as a few bulbs being dead on arrival, half a dozen of the bulbs I bought were duds. They made a horrible buzzing noise when powered, so I removed them. Over time I've also swapped out some of the daylight bulbs from the apparatus to light other rooms of my house, and installed ordinary 750-lumen bulbs into those sockets because it turns out that a matrix of light sockets is an extremely convenient place to store extra bulbs.

Sourcing materials</h1>
The plywood, paint, and romex were left over from other projects.
I had a tough time sourcing affordable e26 sockets. Apparently people don't build ridiculous light fixtures from scratch on the cheap very much? I ended up using adapters meant to help you use e26 bulbs in GU24 sockets. The pins meant to interface with the GU24 socket were easy to solder wires onto later, and the cylindrical shape of the adapters made them easy to friction fit into holes in the plywood.
I made a big spreadsheet of price per lumen of e26 LED bulbs available various places, ruled out those whose specs were incomplete or self-contradictory, and got the cheapest bulbs that seemed likely to add up to Too Much Light.
The switches, 6-gang switchplate, and electrical plug came from Habitat For Humanity.
Build Process</h1>
I chose the size based on what plywood I had around. On the plywood, I drew a grid of 96 rectangles. I drilled holes in alternating rectangles of the grid, to maximize space between bulbs, because I was frightened of making the kind of mistake that would cause the whole thing to overheat and catch on fire.

The holes were drilled with a bit just a hair smaller than the GU24 adapter sockets, so that the sockets would be held in place by friction. A few holes came out too small for the sockets, but were easily enlarged with a rasp.
I also rasped off any noticeable splinters. If I'd been doing fancy carpentry, I would have sanded the whole thing at this point. I wasn't, so I didn't.
I painted the board white, for maximum light reflection. I thought about trying to do some kind of mirror or chrome finish behind the bulbs, but that seemed way too hard. If I was building it again, I would look for a plastic mirror and secure it to the plywood before drilling the holes. Or I might try gluing a sheet of mylar onto the wood after drilling the holes but before installing the sockets.

The sockets installed flush with the board, by pushing them through the holes from the front side. I made sure to install all of them in the same orientation for each row, though this was actually unnecessary because alternating current doesn't care about polarity. To get the sockets flush with the board, I found that it was easiest to fit a row in and then place the whole board face-down on a flat surface and walk on it between the sockets. That way everything ended up even with the floor. If you don't like walking on your projects, you could probably achieve a similar effect by carefully hitting it with a hammer.
I then wired the rows of lights to the switches. 3 of the switches control 12 bulbs each, and 2 control 6 bulbs each, because this seemed like the best compromise between using all 5 available switches and getting it to work with the limited quantity of romex that I had available.

I test fit all the wires to make sure I had enough, then soldered them onto the pins of the GU24 adapters. Since I needed to strip off 1" of wire insulation from the middle of a wire a bunch of times, I built a tool for the job, by gluing razor blades to each side of the jaws of a pair of clothes pins. That way I'd just clip the tool onto the cable where I needed to strip it, spin it around so the blades cut through the insulation, and cut off the inch of insulation with another knife.


You'll notice in the photos that I secured the switches by screwing them directly to the switch plate instead of putting them in a box. You're normally supposed to put switches in boxes, and I might regret this shortcut someday, but I didn't have a 6-gang box and it's been fine so far.
Mind-numbing amounts of soldering later, I tested the assembly to make sure nothing was shorted, then powered it on to make sure it could light. Before installing it on the wall, I covered all the exposed wiring in electircal tape, mostly because I dislike exposed wires. Also, safety. But you'd get just as zapped from connecting the poles of any of those exposed sockets on the front as you would by connecting a pair of wires on the back, so worrying about hiding exposed conductors on the back but not the front is mostly aesthetic.

I then affixed it to the wall above my desk, using a piece of 2x4 at each short end of the plywood to hold everything well away from the wall. I left the top and bottom open to improve air circulation in case it ran into thermal problems, but in a year of use it hasn't caught on fire.
It looks underwhelming in photos, because very bright lights are hard to photograph. This shows how I us it on an ordinary day like today, with a set of 12 5000K bulbs and a set of 6 3000K bulbs lit and the rest remaining off.

Costs</h1>
If you built one of these today, it'd cost you:

about $20 for a 2'x4' chunk of plywood</li>
about $40 for a pack of 50 GU24 to E26 adapters</li>
about $160 for 48 2500-lumen e26 bulbs</li>
about $30 for 15 feet of 12 gauge 2-wire romex</li>
maybe $10 for switches and a 6-gang plate at your local Habitat For Humanity?</li> </ul>
So you'd be looking at around $260 for the whole build, unless you had materials on hand or found better deals on materials.
Was it worth it?</h1>
I like having my false sun in my office, because I feel like being in bright light helps my brain and body agree that it's actually daytime. It also helps me fake a summer sleep schedule when the weather outside suggests wintry hibernation. I notice that being in natural daylight at dusk (even when dusk is at 6pm!) causes me to feel sleepy, whereas being in midday-sunlight light levels circumvents that process.
The false sun was somewhat inconvenient to build and install, and would not be accessible for everyone. I was able to make it because I have woodworking and soldering equipment already available, and because I was able to screw it directly to the wall of my office without penalty.
If I was renting my home, I would have mounted the lightbulb array in a bookcase or other tall piece of furniture instead of attaching it onto the wall. If I was building one of these for use in a rental, I'd probably keep the holes-in-plywood scheme for supporting the lamp sockets, but I would size it to mount over the top couple shelves of an Ikea Billy or similar cheap bookcase, including some room for airflow.
For other locations in my house, I've become partial to the 5,000-lumen 4' LED "shop" lights that my local Harbor Freight sometimes has on sale for $20 apiece. I find them trivially easy to install and plenty bright to send most of that "it's daytime!" signal. However, they're over twice as expensive per lumen compared to the DIY version.
You probably shouldn't build exactly what I did, as it's bulky and inconvenient and not all that aesthetically pleasing. But I hope this project demonstrates that it's perfectly achieveable to create a light fixture brighter than the sun! When playing with electricity, please learn what you're doing beforehand, be mindful that alternating current can kill you, and generally use common sense. If you try to build or modify something like this without having a basic idea of electrical safety, you're just sticking a fork in an outlet but with extra steps.

Playing Dress-Up With Starlink

2022-03-11T00:00:00+00:00

Some friends showed me a post</a> investigating whether Starlink dishes still work when decorated in various ways. They asked whether I was able to reproduce the results. So I pulled the dish down off my roof and tested it with a few things that I had lying around the house and yard.

Methodology Notes:</h1>

I gathered data in my camera roll: a snapshot of the setup, followed by a screencap of the "router <-> internet" pane of the speedtest within the Starlink Android app. I left the dish set up in the same spot, which was at ground level but reported no obstructions when I did the sky scan thing in the app. I power cycled the dish once during the experiment, roughly halfway through the control tests for the "is it slower?" question, because I needed to plug something else into the extension cord that it was using for a minute.

I'm located south of the 45th parallel.

Will It Send?</h1>

For my first batch of tests, I was curious whether I could get any data in and out via Starlink with various configurations of stuff on and around the dish. I only ran one speedtest each for these.

I started by suspending things over Starlink, because I knew some of the materials would get heavy (such as wet cotton cloth) and I was scared of hurting the little motors inside that it uses to move itself around. I draped a canvas tarp over the north side of a fence to make an impromptu photo studio for the dish. The dish had an unobstructed view of the sky upward and to the north. I placed metal folding chairs to the east and west of the dish, with their backs closest to the dish, to hold up the various covers. I used some old brake pads to weigh down the things that I draped over the chairs so they wouldn't blow away.

Single layer of ratty old woven plastic tarp. It sends. 109 Mbps down, 6 Mbps up.</li> </ol>

For the final 2 tests, I got a bit braver about putting things directly on the dish, instead of just suspending them above it.

Dish in a black plastic trash bag like you're throwing it out. It sends. 147 Mbps down, 10 Mbps up.</li>

Same tarp as from (8), but directly on the dish instead of hanging above it. It sends. 207 Mbps down, 20 Mbps up.</li> </ol>

Is It Slower?</h1>

As you'll notice in the numbers that I was getting in the above trials, sometimes covering the dish yields a faster single speed test than having it exposed. That seems wrong. I suspect that this is normal variance based on what satellites are available, but I don't actually know what behavior is normal. So I did a few more tests in configuration 10, and a bunch of tests in configuration 1, to make sure we're not in a timeline where obstructing a radio link somehow makes it perform better.

Mbps Down</th> Mbps Up</th></tr></thead>

207</td> 20</td></tr>

173</td> 5</td></tr>

138</td> 11</td></tr>

124</td> 13</td></tr>

180</td> 19</td></tr>

101</td>

9</td></tr> </tbody></table>

Plastic Tarp Covering Starlink (10)

Mbps Down</th> Mbps Up</th></tr></thead>

104</td> 24</td></tr>

114</td> 4</td></tr>

119</td> 10</td></tr>

100</td> 16</td></tr>

111</td> 14</td></tr>

147</td> 5</td></tr>

97</td> 15</td></tr>

218</td> 15</td></tr>

87</td> 15</td></tr>

176</td> 16</td></tr>

192</td> 6</td></tr>

125</td>

6</td></tr> </tbody></table>

Unobstructed Starlink (1)

I left it all out overnight, and the canvas tarp that I'd been using as a visual backdrop blew off of the fence so it was covering the dish. I made a video call on the connection with the canvas over the dish and didn't notice any subjective degradation of service compared to what I'm accustomed to getting from it.

Conclusions</h1>

Starlink definitely still works when I put dry textiles between the dish and the satellite. It doesn't seem to matter if the textiles are directly on the dish or suspended a couple feet above it.

I'm surprised that Eric Kuhnke's Starlink worked with 2 layers of wet bedsheet over it. The only case in which I managed to cause my Starlink to report itself as being obstructed was when I had a wet bedspread suspended in the air over it. I don't really want to start piling wet cloth directly on the dish, though, because wet cloth is heavy and I'm scared of overloading the actuators.

Starlink gets kind of warm during normal operation, as demonstrated by the cats</a>. I wouldn't want to leave mine in a dark colored trash bag or under a dark colored tarp on a sunny day in case it overheated. And there's no need to -- if you don't want people to know you're using one, you can just suspend an opaque and waterproof tarp above it and there'll be better airflow around the dish itself and thus less risk of overheating.

Have fun!

tree-style tab setup

2022-02-11T00:00:00+00:00

How to get rid of the top bar in firefox after installing tree style tab:

In about:config (accept the risk), search toolkit.legacyUserProfileCustomizations.stylesheets and hit the funny looking button to toggle it to true.

In about:support, above the fold in the "Application Basics" section, find Profile Directory.

In that directory, mkdir chrome, then create userChrome.css, containing:

#main-window[tabsintitlebar="true"]:not([extradragspace="true"]) #TabsToolbar
</span>{
</span>  opacity: 0;
</span>  pointer-events: none;
</span>}
</span>#main-window:not([tabsintitlebar="true"]) #TabsToolbar {
</span>    visibility: collapse !important;
</span>}
</span></code></pre>


top/htop in windows is ctrl+shift+esc
2022-01-30T00:00:00+00:00
Helping a neighbor with a windows update issue today, I explained that asking a Linux admin to use Windows is like asking a Latin speaker to translate a document from Spanish. Most of the concepts are similar enough that they'll be more helpful than a monolingual English speaker, but good guessing is not the same as fluency.</p>
Since it was a problem with good directions</a> on how to fix it, the process mostly went smoothly. As I complained to a Windows admin friend afterwards, it was fine except all the slashes in paths were backwards, and I couldn't find the command prompt equivalent to Linux's top</span> or htop</span>.</p>
My Windows friend pointed out the obvious solution to me: The windows equivalent to top</span> is not a command at all, but rather the task manager GUI. Next time I need top</span> or htop</span> in Windows, I'll try to remember to hit ctrl+shift+esc</span> to summon that interface instead.</p>
And next time I'm searching the web for "windows command prompt top" "windows equivalent of top command", and other queries that assume it'll let me live in the terminal like my preferred operating systems, I might just end up back on this very post. Hi, future me!</p>


transcription with mplayer and i3
2021-06-12T00:00:00+00:00
I recently wanted to manually transcribe an audio recording. I prefer to type into LibreOffice Writer for this purpose. Writer has an audio player plugin</a> for transcription, but unfortunately its keyboard shortcuts didn't work when I tried it.</p>
I just want to play some audio in one workspace and have play/pause and 5-second rewind shortcuts work even when another window is focused.</p>
Since I am using i3wm on Ubuntu, I can glue up a serviceable transcription setup from stuff that's already lying around.</p>
The first challenge is to persuade an audio player to accept commands while it's not the window in focus. By complaining about this problem to someone more knowledgeable than myself, I learned about mplayer's slave mode. From its docs</a>, I learn that I can instruct mplayer to take arbitrary commands on a fifo as follows:</p>
$ mkfifo /tmp/mplayerfifo
</span>$ mplayer -slave -input file=/tmp/mplayerfifo audio-to-transcribe.mp3
</span></code></pre>
Now I can test whether mplayer is listening on the fifo. And indeed, the audio pauses when I tell it:</p>
$ echo pause > /tmp/mplayerfifo
</span></code></pre>
At this time I also test the incantation to rewind the audio by 5 seconds:</p>
$ echo seek -5 > /tmp/mplayerfifo
</span></code></pre>
Since both commands work as expected, I can now create keyboard shortcuts for them in .i3/config</code>:</p>
bindsym $mod+space exec "echo pause > /tmp/mplayerfifo"
</span>bindsym $mod+z exec "echo seek -5 > /tmp/mplayerfifo"
</span></code></pre>
After writing the config, $mod+shift+c</code> reloads it so i3 knows about the new shortcuts.</p>
Finally, I'll make sure this keeps working after I reboot. I'll make an alias in my ~/.bashrc</code> to save having to remember the mplayer incantation:</p>
$ echo "alias transcribe='mplayer -slave -input file=/tmp/mplayerfifo" >> ~/.bashrc
</span></code></pre>
And to automatically create the fifo once on boot:</p>
$ echo "mkfifo /tmp/mplayerfifo" >> ~/.profile 
</span></code></pre>
Now after I source ~/.bashrc</code>, I can play media with this transcribe</code> alias, and the keyboard shortcuts control it from anywhere in my window manager.</p>


irssi and libera.chat
2021-05-25T00:00:00+00:00
I'm in some channels that are moving from Freenode to Libera.</p>
My irssi runs on a DigitalOcean droplet, and whenever I try to connect to Libera from that instance, I get the error:</p>
[libera] !tungsten.libera.chat *** Notice -- You need to identify via SASL to use this server
</span></code></pre>
Libera's irssi guide (https://libera.chat/guides/irssi</a>) says how to connect with SASL, and down in their sasl docs (https://libera.chat/guides/sasl</a>) they mention that SASL is required for IP ranges that are easy to run bots on... including my VPS.</p>
The fix is to pop open an IRC client locally (or use web IRC), connect to Libera without SASL, and register one's nick and password. After verifying one's email address over the regular connection, the network can be reached via SASL from anywhere using the registered nick as the username</span> and the nickserv password as the password</span>.</p>
Obvious in retrospect, but poorly SEO'd for how the problem looks at the outset, so that's how I worked around problems reaching Libera from Irssi on a VPS.</p>


Assembly Lines
2021-04-03T00:00:00+00:00
This time last year, my living room was occupied by a cotton mask production facility of my own devising. I had reverse engineered a leftover surgical mask to get the approximate dimensions, consulted pictures of actual surgeons' masks, and contrived a mask design which was easy-enough to sew in bulk, durable-enough to wash with one's linens, and wearable-enough to fit most faces.</p>
Tinkering with and improving the production line was delightful enough to make me wonder if I'd missed a deeper calling when I chose not to pursue industrial engineering as a career, but the actual work -- the parts where I used myself as jsut another machine to make more masks happen -- was profoundly miserable. At the time, it made more sense to attribute that misery to current events: The world as we knew it is ending, of course I'm grumpy. I took it for granted that the sewing project of making masks was equivalent to the design/prototype/build cycle of my more creative sewing endeavors, and assumed that it was supposed to be equally enjoyable.</p>
A year later, however, I'm running a similar personal assembly line on an electrical project, and noticing some patterns. I have to do 4 steps each on 96 little widgets to complete this phase of the project. My engineering intuition says that the optimal process would be to do all of step 1, then all of step 2, then all of step 3, then all of step 4. That seems like it should be the fastest, and make me happy becuase it's the best -- no wasted effort taking out then putting away the set of tools for each step several times.</p>
The large-batch process would also yield consistency across all of its outputs, so that no one widget comes out much worse than any other. Consistency is aesthetic and satisfying in the end result, so the process which yields consistency should feel preferable... but instead, it feels deeply distasteful to stick with any one production phase for too long. What's going on there? What assumption is one side of the internal argument using that the other side lacks?</p>
It took me 2 steps over about 24 of the widgets to figure out what felt so wrong about that assembly-line reasoning: The claims of "best" and "fastest" only hold if the process being done remains exactly the same on widget 96 as it was on widget 1. That's true if a machine is doing it, but false if the worker is able and allowed to think about the process they're working on. Larger batch sizes are optimal if the assembly process is unchanging, but detrimental if the process needs to be modified for efficiency or ease of use along the way. For instance, I'd initially planned a design that needed about 36' of wire, but by examining and contemplating the project when it was ready to be wired up, I found a way to accomplish the same goals with only about 23' of wire. If I'd been "perfectly efficient" in treating the initial design as perfect, I would likely have cut the wire into the lengths that were needed for the 36' plan before the 23' design occurred to me, and that premature optimization would have destroyed the materials I'd need to assemble the more efficient design once I figured it out.</p>
In other words, a self-modifying assembly line necessarily shrinks the batch size that it's worth producing. I've seen the same thing in software -- when automating a process, it's best to do it by hand a couple times, and then test a script on a small batch of input and fix any errors, and then apply it to larger and larger batches as it gets closer and closer to the best I can get it. It's just easier to notice the phenomenon in a process that uses the hands while leaving the brain mostly free than in processes of more intellectual labor.</p>
And there was the answer as to why attempting to do all 96 of step 1, then all 96 of step 2, felt terrible: Because using the maximum batch size implied that the process was as good as I'd be able to get it, and that any improvements I might think of while working would be wasted if they weren't backwards-compatible with the steps of the old process that were already completed. Smaller batch sizes, then, have an element of hope to them: There will be a "next time" of the whole process, so thinking about "how I'd do it next time" has a chance to pay off.</p>


pulseaudio & volumio
2020-05-30T00:00:00+00:00
The speakers in my living room are hooked up to a raspberry pi that runs Volumio</a>. It's a nice way to play music from various sources without having to physically reconfigure the speakers between inputs.</p>
Volumio as a pulseaudio output</h1>
For awhile, my laptop was able to treat Volumio as just another output device, based on the following setup:</p>

the package pulseaudio-module-zeroconf</span> was installed on the pi and on every laptop that wants to output audio through the living room speakers</li>
the lines load-module module-zeroconf-publish</span> and load-module module-native-protocol-tcp</span> were added to /etc/pulseaudio/default.pa</span> on the pi</li>
the line load-module module-zeroconf-discover</span> was added to /etc/pulse/default.pa</span> on my Ubuntu laptop</li>
pulseaudio was restarted on both devices after these changes (pulseaudio -k</span> to kill, pulseaudio</span> to start it)</li>
</ul>
starting pulseaudio on boot on Volumio</h1>
And then as long as the laptop was connected to the same wifi network as the pi, it Just Worked. Until, in the course of troubleshooting an issue that turned out to involve the laptop having chosen the wrong wifi, I power cycled the pi and it stopped working, because pulseaudio was not yet configured to start on boot.</p>
The solution was to add the following to /etc/systemd/system/pulseaudio.service</span> on the pi:</p>
[Unit]
</span>Description=PulseAudio system server
</span>
</span>[Service]
</span>Type=notify
</span>Exec=pulseaudio --daemonize=no --system --realtime --log-target=journal
</span>User=volumio
</span>ExecStart=/usr/bin/pulseaudio
</span>
</span>[Install]
</span></code></pre>
And then enabling, starting, and troubleshooting any failures to start:</p>
systemctl --system enable pulseaudio.service
</span>systemctl --system start pulseaudio.service
</span>systemctl status pulseaudio.service -l # explain what went wrong
</span>systemctl daemon-reload # run after editing the .service file
</span></code></pre>
Thanks to Rudd-O's blog post</a>, which got me 90% of the way to the "start pulseaudio on boot" solution. Apparently systemctl started caring more about having an ExecStart directive since that post was written, which meant I had to inspect the resulting errors, which means I'm writing down the resulting tidbit of knowledge so that I can find it again later.</p>
future work</h1>
Nobody in my household has yet found a good way to persuade the Windows computer who lives under the TV to speak pulseaudio yet. If I ever figure that out, I'll update here.</p>


Moving on from Mozilla
2020-05-22T00:00:00+00:00
Today -- Friday, May 22nd, 2020 -- is within days of my 5-year anniversary with Mozilla, and it's also my last day there for a while. Working at Mozilla has been an amazing experience, and I'd recommend it to anyone.</p>
There are some things that Mozilla does extremely well, and I'm excited to spread those patterns to other parts of the industry. And there are areas where Mozilla has room for improvement, where I'd like to see how others address those challenges and maybe even bring back what I learn to Moz someday.</p>
Why go?</h1>
When I try to predict what my 2025 or 2030 self will wish I'd done with my career around now, I anticipate that I'll want access to opportunities which build on a background of technical leadership and mentoring junior engineers.</p>
It wouldn't be impossible to create these opportunities within Mozilla, but from talking with trusted mentors both inside and outside the company, I've concluded that I would get a lot more impact for the same effort if I was working within a growing organization.</p>
As a mature organization, Mozilla's internal leadership needs are very different from those of a younger and more actively growing company. There's a far higher bar at Moz for what it takes to be the best person for a task, because the saturation of "best people" is quite high and the prevalence of entirely new tasks is relatively low in comparison. Technical leadership here seems to often require creating a need as well as filling it. At a growing organization, on the other hand, the types and availabilities of such opportunities are very different.</p>
I'm especially looking forward to leveling up on a different stack in my next role, to improve my understanding of the nuances of the underlying problems our technolgies address. I think it's a bit like learning a second language: only through comparing and contrasting multiple solutions to the same sort of problem can one understand what traits corrolate to all those solutions' strengths versus which details are simply incidental.</p>
Why now?</h1>
I'll be the first to admit that May 2020 is a really strange time to be changing jobs. But I have an annual tradition of interviewing at several places, learning what their stacks and cultures and unique fractals of tech debt look like, and then turning down an offer or two because changing roles would be a step backwards for both my career development and overall quality of life.</p>
Shortly before the global conference circuit ground to a halt along with everything else, I started interviewing from a DevOps Advocate position, just to explore what it might look like to turn my teaching hobby into a day job. By the time those interviews were complete, the tech evangelism space had been turned inside out and was rapidly reinventing itself, and the skills that qualified me for the old world of devrel were looking less and less like the kind of expertise that might be needed to succeed in the new one. However, a SRE from the technical interviews suggested that I interview for her team, and upon taking that advice I discovered an organization that keeps most of the stuff I loved about Mozilla while also offering the other opportunities that I was looking for.</p>
As with anywhere, there are a few aspects of my new role that I suspect may not be as great yet as where I'm leaving, but these areas of improvement look like things that I'll be able to have some influence over. Or at least there'll be room to push the Overton Window in a good direction!</p>
Want more details on the new role? I'll be writing more about it after I start on June 1st!</p>


Offboarding
2020-05-22T00:00:00+00:00
Turns out that 5 years at a place gets you a bit of a pile of digital detritus. Future me might want notes on what-all steps I took to remove myself from everything, so here goes:</p>

GitHub: Clicking the "pull requests" thing in that bar at the top gives a list of all open PRs created by me. I closed out everything work-related, by either finishing or wontfix-ing it. Additionally, I looked through the list of organizations in the sidebar of my account and kicked myself out of owner permissions that I no longer need. Since my GitHub workflow at Mozilla included a separate account for holding admin perms on some organizations, I revoked all of that account's permissions and then deleted it.</li>
Google Drive: (because moving documents around through the Google Docs interface is either prohibitively difficult or just impossible) I moved all notes docs that anyone might ever want again into a shared team folder.</li>
Bugzilla: The "my dashboard" link at the top, when logged in, lists all needinfos and open assigned bugs. I went through all of these and removed the needinfos from closed bugs, changed the needinfos to appropriate people on open bugs, and reassigned assigned bugs to the people who are taking over my old projects. When reassigning, I linked the appropriate notes documents in the bugs and filled in any contextual information that they didn't capture. I also checked that my Bugzilla admin had removed all settings to auto-assign me bugs in certain components.</li>
Email deletion prep: I searched for my old work email address in my password manager to find all accounts that were using it. I deleted these accounts or switched them to a personal address, as necessary. It turned out that the only thing I needed to switch over was my Firefox account, which I initially set up to test a feature on a service I supported, but then found very useful.</li>
Git repos: When purging pull requests and bugs, I pushed my latest work from actively developed branches, so that no work will be lost when I wipe my laptop</li>
Assorted other perms: Some developers had granted me access to a repo of secrets, so I contacted them to get that access revoked.</li>
Sharing contact info: I didn't send an email to the all-company list, but I did email my contact info to my teammates and other colleagues with whom I'd like to keep in touch.</li>
Take notes on points of contact. While I still have access to internal wikis, I note the email addresses of anyone I may need to contact if there are problems with my offboarding after my LDAP is decommissioned.</li>
Wipe the laptop: That's next. All the repos of Secret Secrets are encrypted on its disk and I'll lose the ability to access an essential share of the decryption key when my LDAP account goes away, but it's still best practices to wipe hardware before returning it. So I'll power it off, boot it from a liveUSB, and then run a few different tools to wipe and overwrite the disk.</li>
</ul>


Git: moving a module into a monorepo
2020-02-18T00:00:00+00:00
My team has a repo where we keep all our terraform modules, but we had a separate module off in its own repo for reasons that are no longer relevant.</p>
Let's call the modules repo git@github.com:our-org/our-modules.git</span>. The module moving into it, let's call it git@github.com:our-org/postgres-module.git</span>, because it's a postgres module.</p>
First, clone both repos.:</p>
git clone git@github.com:our-org/our-modules.git
</span>git clone git@github.com:our-org/postgres-module.git
</span></code></pre>
I can't just add postgres-module</span> as a remote to our-modules</span> and pull from it, because I need the files to end up in a subdirectory of our-modules</span>. Instead, I have to make a commit to postgres-module</span> that puts its files in exactly the place that I want them to land in our-modules</span>. If I didn't, the README.md</span> files from both repos would hit a merge conflict.</p>
So, here's how to make that one last commit:</p>
cd postgres-module
</span>mkdir postgres
</span>git mv *.tf postgres/
</span>git mv *.md postgres/
</span>git commit -m "postgres: prepare for move to modules repo"
</span>cd ..
</span></code></pre>
Notice that I don't push that commit anywhere. It just sits on my filesystem, because I'll pull from that part of my filesystem instead of across the network to get the repo's changes into the modules repo:</p>
cd our-modules
</span>git remote add pg ../postgres-module/
</span>git pull pg master --allow-unrelated-histories
</span>git remote rm pg
</span>cd ..
</span></code></pre>
At this point, I have all the files and their history from the postgres module in the postgres</span> directory of the our-modules</span> repo. I can then follow the usual process to PR these changes to the our-modules</span> remote:</p>
cd our-modules
</span>git checkout -b import-pg-module
</span>git push origin import-pg-module
</span>firefox https://github.com/our-org/our-modules/pull/new/import-pg-module
</span></code></pre>
We eventually ended up to skip importing the history on this module, but figuring out how to do it properly was still an educational exercise.</p>


Finding a lost Minecraft base
2020-01-04T00:00:00+00:00
I happen to administer a tiny, mostly-vanilla Minecraft server. The other day, I was playing there with some friends at a location out in the middle of nowhere. I slept in a bed at the base, thinking that would suffice to get me back again later.</p>
After returning to spawn, installing a warp plugin (and learning that /warp</span> comes from Essentials), rebooting the server, and teleporting to some other coordinates to install their warps, I tried killing my avatar to return it to its bed. Instead of waking up in bed, it reappeared at spawn. Since my friends had long ago signed off for the night, I couldn't just teleport to them. And I hadn't written down the base's coordinates. How could I get back?</p>
Some digging in the docs revealed that there does not appear to be any console command to get a server to disclose the last seen location, or even the bed location, of an arbitrary player to an administrator. However, the server must know something about the players, because it will usually remember where their beds were when they rejoin the game.</p>
On the server, there is a world/playerdata/</span> directory, containing one file per player that the server has ever seen. The file names are the player UUIDs, which can be pasted into this tool</a> to turn them into usernames. But I skipped the tool, because the last modified timestamps on the files told me which two belonged to the friends who had both been at our base. So, I copied a .dat</span> file that appeared to correspond to a player whose location or bed location would be useful to me. Running file</span> on the file pointed out that it was gzipped, but unzipping it and checking the result for anything useful with strings</span> yielded nothing comprehensible.</p>
The wiki</a> reminded me that the .dat</span> was NBT-encoded. The recommended NBT Explorer tool appeared to require a bunch of Mono runtime stuff to be compatible with Linux, so instead I grabbed some code that claimed to be a Python NBT wrapper</a> to see if it would do anything useful. With some help from its examples, I retrieved the player's bed location:</p>
from nbt import *
</span>n = nbt.NBTFile("myfile.dat",'rb')
</span>print("x=%s, y=%s, z=%s" % (n["SpawnX"], n["SpawnY"], n["SpawnZ"]))
</span></code></pre>
Teleporting to those coordinates revealed that this was indeed the player's bed, at the base I'd been looking all over for!</p>
The morals of this story are twofold: First, I should not quit writing down coordinates I care about on paper, and second, Minecraft-adjacent programming is still not my idea of a good time.</p>


Toy hypercube construction
2019-12-30T00:00:00+00:00
I think hypercubes are neat, so I tried to make one out of string to play with. In the process, I discovered that there are surprisingly many ways to fail to trace every edge of a drawing of a hypercube exactly once with a single continuous line.</p>
This puzzle felt like the sort of problem that some nerd had probably solved before, so I searched the web and discovered that the shape I was trying to configure the string into is called an Eulerian Cycle.</p>
I learned that any graph in which every vertex attaches to an even number of edges has such a cycle, which is useful for my craft project because the euler cycle is literally the path that the string needs to take to make a model of the object represented by the graph.</p>
Mathematical materials</h1>
To construct a toy hypercube or any other graph, you need the graph. To make it from a single piece of string, every vertex should have an even number of edges.</p>
Knowing the number of edges in the graph will be useful later, when marking the string.</p>
Physical materials</h1>
For the edges of the toy, I wanted something that's a bit flexible but can sort of stand up on its own. I found that cotton clothesline rope worked well: it's easy to mark, easy to pin vertex numbers onto, and sturdy but still flexible. I realized after completing the construction that it would have been clever to string items like beads onto the edges to make the toy prettier and identify which edge is which.</p>
For the vertices, I pierced jump rings through the rope, then soldered them shut, to create flexible attachment points. This worked better than a previous prototype in which I used flimsier string and made the vertices from beads.</p>
Vertices could be knotted, glued, sewn, or safety pinned. A bookbinding awl came in handy for making holes in the rope for the rings to go through.</p>
Mathematical construction</h1>
First, I drew the graph of the shape I was trying to make -- in this case, a hypercube. I counted its edges per vertex, 4. I made sure to draw each vertex with spots to write numbers in, half as many numbers as there are edges, because each time the string passes through the vertex it makes 2 edges. So in this case, every vertex needs room to write 2 numbers on it.</p>
Here's the graph I started with. I drew the edges in a lighter color so I could see which had already been visited when drawing in the euler cycle.</p>
</p>
Then I started from an arbitrary vertex and drew in the line. Any algorithm for finding euler paths will suffice to draw the line. The important part of tracing the line on the graph is to mark each vertex it encounters, sequentially. So the vertex I start at is 1, the first vertex I visit is 2, and so forth.</p>
Since the euler path visits every vertex of my particular hypercube twice, every vertex will have 2 numbers (the one I started at will have 3) when I finish the math puzzle. These pairs of numbers are what tell me which part of the string to attach to which other part.</p>
Here's what my graph looked like once I found an euler cycle in it and numbered the vertices that the cycle visited:</p>
</p>
Physical construction</h1>
Since my graph has 32 edges, I made 33 evenly spaced marks on the string. I used an index card to measure them because that seemed like an ok size, but in retrospect it would have been fine if I'd made it smaller.</p>
</p>
I then numbered each mark in sequence, from 1 to 33. I numbered them by writing the numbers on slips of paper and pinning the papers to the rope, but if I was using a ribbon or larger rope, the numbers could have been written directly on it. If you're doing this at home, you could mark the numbers on masking tape on the rope just as well.</p>
</p>
The really tedious step is applying the vertices. I just went through the graph, one vertex at a time, and attached the right points on the string together for it.</p>
The first vertex had numbers 1, 25, and 33 on it for the euler cycle I drew and numbered on the graph, so I attached the string's points 1, 25, and 33 together with a jump ring. The next vertex on the drawing had the numbers 2 and 18 on it, so I pierced together the points on the string that were labeled 2 and 18.</p>
I don't think it matters what order the vertices are assembled in, as long as the process ultimately results in all the vertices on the graph being represented by rings affixing the corresponding points on the string together.</p>
I also soldered the rings shut, because after all that work I don't want them falling out.</p>
</p>
That's all there is to it!</p>
</p>
I'm going to have to find a faster way to apply the vertices before attempting a 6D hypercube. An ideal vertex would allow all edges to rotate and reposition themselves freely, but failing that, a lighter weight string and crimp fasteners large enough to hold 6 pieces of that string might do the trick.</p>
The finished toy is not much to look at, but quite amusing to try to flatten into 3-space.</p>
</p>


kubectl unable to recognize STDIN
2019-09-17T00:00:00+00:00
Or, Stupid Error Of The Day. I'm talking to a GCP's Kubernetes engine through several layers of intermediate tooling, and kubectl is failing:</p>
subprocess.CalledProcessError: Command '['kubectl', 'apply', '--record', '-f', '-']' returned non-zero exit status 1.
</span></code></pre>
Above that, in the wall of other debug info, is an error of the form:</p>
error: unable to recognize "STDIN": Get https://11.22.33.44/api?timeout=32s: dial tcp 11.22.33.44:443: i/o timeout
</span></code></pre>
This error turned out to have such a retrospectively obvious fix that nobody else seems to have published it.</p>
When setting up the cluster on which kubectl was failing, I added the IP from which my tooling would access it, and hit the "done" button to save my changes. (That's under the Authorized Networks section in "kubernetes engine -> clusters -> edit cluster" if you're looking for it in the GCP console.) However, the "done" button is only one of the two required steps to save changes: One also must scroll all the way to the bottom of the page and press the "save" button there.</p>
So if you're here because you Googled that error, go recheck that you really do have access to the cluster on which you're trying to kubectl apply. Good luck!</p>


What to bring to CCC Camp next time
2019-08-26T00:00:00+00:00
I took last week off work and attended CCC camp, which was wonderful on a variety of axes. I packed light, but through the week I noted some things it'd be worth packing less-lightly for.</p>
So, here are my notes on what it'd be worth bringing if or when I attend it again:</p>
Clothing</h1>
The site is dusty, extremely hot through the day, and quite cold at night. Fashion ranges from "generic nerd" to hippie, rave, and un-labelably eccentric. There is probably no wrong thing to wear, though I didn't see a single suit or tie. A full base layer and a silk sleeping bag liner improve comfort at night. A big hat, or even an umbrella, offers protection from the day star.</p>
I was glad to have 3 pairs of shoes: Lightweight waterproof sandals for showering, sturdier sandals for walking around in all day, and boots for early mornings and late nights. I saw quite a few long coats and even cloaks at night, and their inhabitants all looked very comfortably warm.</p>
Doing sink laundry was more inconvenient at camp than for ordinary travel, and I was glad to have packed to minimize it.</p>
A small comfortable bag, or large pockets in every outfit, are essential for keeping track of one's wallet, phone, map, and water bottle.</p>
I occasionally found myself wishing that I'd brought a washable dust mask, usually around midafternoon when camp became one big dust cloud.</p>
Campsite Amenities</h1>
Covering a tent in space blankets makes it look like a baked potato, but keeps it warm and dark at night and cool through early afternoon. Space blankets are super cheap online, but difficult to find locally.</p>
For a particularly opulent tent experience, consider putting a doormat outside the entrance as a place to remove shoes or clean dusty feet before going inside. I improvised a doormat with a trash bag, which was alright but the real thing would have been nicer to sit on.</p>
Biertisch tables and benches are prevalent around camp, so you can usually find somewhere to sit, but it doesn't hurt to bring a camp chair of the folding or inflatable variety. Inflatable stuff, from furniture to swimming pools, tended to survive fine on the ground.</p>
I was glad to have brought a full sized towel rather than a tiny travel one. A shower caddy or bag to carry soap, washcloth, hair stuff, and clean clothes would have been handy, though I improvised one from another bag that I had available.</p>
String and duct tape came in predictably handy in customizing my campsite.</p>
Electronics</h1>
DECT phones are very fun at camp, but easy to pocket dial with. This is solved by finding the lock feature on the keypad, or picking a flip phone. I was shy about publishing my number and location in the phonebook, but after seeing how helpful the directory was for people to get ahold of new acquaintances for important reasons, I would be more public about my temporary number in the future.</p>
Electricity is a limited resource but sunlight isn't. Many tents sport portable solar panels. For those whose electronics have non-European plugs, a power strip from home is a good idea.</p>
I packed a small headlamp and used it pretty much every day. Even with it, I found myself occasionally wishing that I'd brought a small LED lantern as well.</p>
A battery to recharge cell phones is good to have as well, especially if you don't run power to your tent. A battery can be left charging unattended in all kinds of public places where one would never leave one's phone or laptop.</p>
Food</h1>
Potable water is free, both still and sparkling. Perhaps I've been spoiled by the quality of the tap water at my home, but I wished that I'd brought water flavorings to mask the local combination of minerals.</p>
I brought a small medical kit, from which I ended up using or sharing some aspirin, ibuprofin, antihistamines, and lots of oral rehydration salt packets.</p>
Meals were available for free (with donations gratefully accepted) at several camps for everyone, and at the Heaven kitchen for volunteers. There were also a variety of food carts with varyingly priced dishes. The food carts outside the gates in front of the venue were good for an icecream or fresh veggie snack, which were harder to find within camp.</p>
Savory meals and all kinds of drinks were everywhere, but there didn't seem to be any place nearby to just pick up straight chocolate. Small, nonperishable snacks like that are worth getting at a grocery before arrival, since they're not readily available on the grounds.</p>
Other</h1>
If any of the special skills that your nerd friends ask you for help with require tools, bring them. I happen to always carry a needle and thread when traveling, and ended up using them to repair a giant inflatable computer-controlled sculpture.</p>
A hammock, and something to shade it with, came in very handy and would be worth bringing again. There were lots of trees, and it might have been entertaining to set up a slackline for passers-by to fall off of, but I don't think it'd be worth the weight of carrying one internationally.</p>
Night time is basically a futuristic art show as well as a party. There's no such thing as too much electroluminescent wire or too many LEDs, whether for decorating your camp or yourself. As a music party, it's also extremely loud, so I was glad to have brought earplugs. Comfortable earplugs also improve sleep; music goes till 3 or 4 AM in many places and early risers start making noise around 8 or 9.</p>
Camp has a lake, in which it's popular to float large inflatable animals, especially unicorns. I saw more big inflatable unicorn floaties being used around camps as extra seating than being used in the lake, though.</p>
There's a railway that goes around camp, and sometimes runs a steam train. I won't say you should</em> rig a little electric cart to fit its rails and drive around on it, but somebody did and looked like they were having a really wonderful time.</p>
Bikes, and lots of folding bikes, were everywhere. Scooters, skateboards, and all sorts of other wheeled contrivances, often electric, were also prevalent. The only rolling transportation that I didn't see at all around camp were roller blades and skates, because the ground is probably too rough for them.</p>
I ran out of stickers, and wished I'd brought more. I didn't see as many pins as some conferences have.</p>
A small notebook also came in handy. Each day, I checked both the stage schedule and the calendar to find the official and unofficial events which looked interesting, and noted their times on paper. It was consistently convenient to have a means of jotting down notes which didn't risk running out of battery. Flipping through the book afterwards, about 1/4 of its contents is actually pictures I drew to explain various concepts to people I was chatting with, a few pages are daily schedule notes, and the rest is about half notes on things that presenters said and half ideas I jotted down to do something with later.</p>
I was glad to have brought cash rather than just cards, not only for food but also because many workshops had a small fee to cover the cost of the materials that they provided.</p>
Camp Advice</h1>
Nobody even tries to maintain a normal sleep schedule. People sleep when they're tired, and do stuff when they aren't. Talks and events tend to be scheduled from around noon to around midnight. I don't think it would be possible to attend camp with a rigorous plan for what to every day and both stick to that plan and get the most out of the experience.</p>
In shared spaces, people pick the lowest common denominator of language -- at several workshops, even those initially scheduled to be held in German, presenters proactively asked if any attendees needed it to be in English then switched to English if asked. Behind-the-scenes, such as in the volunteers' kitchen, I found that this was reversed: Everyone speaks German, and only switches to give you instructions if you specifically ask for English. Plenty of attendees have no German at all and get along fine.</p>
Volunteer! If something isn't happening how it should, fix it, or ask "how can I help?". Volunteering an hour or two for filing badges or washing dishes is a great way to make new friends and see another side of how camp works.</p>


More on Mentorship
2019-06-24T00:00:00+00:00
Last year, I wrote</a> about some of the aspirations which motivated my move from Mozilla Research to the CloudOps team. At the recent Mozilla All Hands in Whistler, I had the "how's the new team going?" conversation with many old and new friends, and that repetition helped me reify some ideas about what I really meant by "I'd like better mentorship".</p>
To generalize about how mentors' careers affect what they can mentor me on, I've sketched up a quick figure in order to name some possible situations that people can be in relative to one another:</p>
</p>
The first couple cases of mentorship are easy to describe, because I've experienced and thought about them for many years already:</p>
Mentorship across industries</h1>
Mentors from outside my own industry are valuable for high level perspectives, and for advice on general life and human topics that aren't specialized to a single field. Additionally, specialists in other industries often represent the consumers of my own industry's products. Wise and thoughtful people who share little to none of my domain knowledge can provide constructive feedback on why my industry's work gets particular reactions from the people it affects -- just as someone who's never read a particular book before is likely to catch more spelling errors than its own author, who's been poring over the same manuscript for many hours a day for several years.</p>
However, for more concrete problems within my particular career ("this program is running slower than expected", or even "how should I describe that role on my resume?"), observers from outside of it can rarely offer a well tested recommendation of a path forward.</p>
Mentorship across companies within an industry</h1>
Similarly, mentors from other companies within my own industry are my go-to source of insight on general trends and technologies. A colleague in a distant corner of my field can tell me about the frustrations they encountered when using a piece of technology that I'm considering, and I can use that advice to make better-informed choices in my daily work.</p>
But advice on a particular company's peculiarities rarely translates well across organizations. A certain frequency of reorganization might be perfectly ordinary at my company, but a re-org might indicate major problems at another. This type of education, while difficult to get from someone at a different company, is perfectly feasible to pick up from anyone on another team within one's own organization.</p>
Mentorship across teams within a company</h1>
When I switched roles, I had trial-and-errored my way into the observation that there's a large class of problems with which mentors from different teams within the same company cannot effectively help. I'd tentatively call these "junior engineer problems", as having overcome their general cases seems to correlate strongly to seniority. In my own expeience, honing the improvement of code-adjacent skills such as the intuition for what problems should be effectively solvable from the docs versus when and whom to ask for help, how deeply to explore a prospective course of action before committing to it, and when to write off an experiment as "effectively impossible", are all questions whose answers one derives from experience and observing expert peers rather than from just asking them with words.</p>
Mentorship across projects or specialties within a team</h1>
I had assumed that simply being on the same team as people capable of imparting that highly specialized variant of common sense would suffice to expose me to it. However, my first few projects on my new team have clearly shown, in both the positive and the negative cases, that working on the same project as an expert is far more useful to my own growth than simply chancing to be bureaucracied into the same group.</p>
The negative case was my first pair of projects: The migration of 2 small, simple services from my team's AWS infrastructure to GCP. Although I was on the same team as experts in this process, the particular projects were essentially mine alone, and it was up to me to determine how far to proceed on each problem by myself before escalating it to interrupt a busy senior engineer. My heuristics for that process weren't great, and I knew that at the outset, but my bias toward asking for help later than was optimal slowed the process of improving my ability to draw that line -- how can one enhance one's discrimination between "too soon", "just right", and "too late" when all the data points one gathers are in the same one of those categories?</p>
Mentorship within a project</h1>
Finally, however, I'm in the midst of a project that demonstrates a positive case for the type of mentorship I switched teams to seek. I'm in the case labeled A on the diagram up above -- I'm working with a more-experienced teammate on a project which also includes close collaboration with members of another team within our organization. In examining why this is working so much better for me than my prior tasks, I've noticed some differences: First, I'm getting constant feedback on my own expectations for my work. This is no serious nor bureaucratic process, but simply a series of tiny interactions -- expressions of surprise when I complete a task effectively, or recommendations to move on to a different approach when something seems to take too long. Similarly, code review from someone immersed in the same problem that I'm working on is indescribably more constructive than review from someone who's less familiar with the nuances of whatever objective my code is trying to achieve.</p>
Another reason that I suspect I'm improving more quickly than before in this particular task is the opportunity to observe my teammate modeling the skills that I'm learning in his interactions with our colleagues from another team (those in position C on that chart). There's always a particular trick to asking a question in a way that elicits the category of answer one actually wanted, and watching this trick done frequently in circumstances where I'm up to date on all the nuances and details is a great way to learn.</p>
The FOSS loophole</h1>
I suspect I may have been slower to notice these differences than I otherwise might have been, because the start of my career included a lot of fantastic, same-project mentorship from individuals on other teams, at other companies, and even in other industries. This is because my earliest work was on free and open source software and infrastructure. In FOSS, anyone who can pay with their time and computer usage buys access to a cross-company, often cross-industry web of professionals and can derive all the benefits of working directly with mentors on a single project. I was particularly fortunate to draw a wage from the OSU Open Source Lab while doing that work, because the opportunity cost of a hours spent on FOSS by a student who also needs to spend those hours on work is far from free.</p>


Rustacean Hat Pattern
2019-04-06T00:00:00+00:00
Based on feedback from the crab plushie pattern</a>, I took more pictures this time.</p>
There are 40 pictures of the process below the fold.</p>
Materials required</h1>
</p>

About 1/4 yard or 1/4 meter of orange fabric. Maybe more if it's particularly narrow. Polar fleece is good because it stretches a little and does not fray near seams.</li>
A measuring device. You can just use a piece of string and mark it.</li>
Scissors, a sewing machine, pins, orange thread</li>
Scraps of black and white cloth to make the face</li>
The measurements of the hat wearer's head. I'm using a hat to guess the measurements from.</li>
A pen or something to mark the fabric with is handy.</li>
</ul>
Constructing the pattern pieces</h1>
If you're using polar fleece, you don't have to pre-wash it. Fold it in half. In these pictures, I have the fold on the left and the selvedges on the right.</p>
The first step is to chop off a piece from the bottom of the fleece. We'll use it to make the legs and spines later. Basically like this:</p>
</p>
Next, measure the circumference you want the hat to be. I've measured on a hat to show you.</p>
</p>
Find 1/4 of that circumference. If you measured with a string, you can just fold it, like I folded the tape measure. Or you could use maths.</p>
</p>
That quarter-of-the-circumference is the distance that you fold over the left side of the big piece of fabric. Like so:</p>
</p>
Leave it folded over, we'll get right back to it. Guesstimate the height that a hat piece might need to be, so that we can sketch a piece of the hat on it. I do this by measuring front to back on a hat and folding the string, I mean tape measure, in half:</p>
</p>
Back on the piece we folded over, put down the measurement so we make sure not to cut the hat too short. That measurement tells you roughly where to draw a curvy triangle on the folded fabric, just like this:</p>
</p>
Now cut it out. Make sure not to cut off that folded edge. Like this:</p>
</p>
Congratulations, you just cut out the lining of the hat! It should be all one piece. If we unfold the bit we just cut and the bit we cut it from, it'll look like this:</p>
</p>
Now we're going to use that lining piece as a template to cut the outside pieces. Set it down on the fabric like so:</p>
</p>
And cut around it. Afterwards you have 1 lining piece and 2 outer pieces:</p>
</p>
Now grab that black and white scrap fabric and cut a couple eye sized black circles, and a couple bits of white for the light glints on the eyes. Also cut a black D shape to be the mouth if you want your hat to have a happy little mouth as well as just eyes.</p>
</p>
Construction</h1>
Put the black and white together, and sew an eye glint kind of shape in the same spot on both, like so:</p>
</p>
Pull the top threads to the back so the stitching looks all tidy. Then cut off the excess white fabric so it looks all pretty:</p>
</p>
Now the fun part: Grab one of those outside pieces we cut before. Doesn't matter which. Sew the eyes andmouth onto it like so:</p>
</p>
Now it's time to give the hat some shape. On both outside pieces -- the one with the face and also the one without -- sew that little V shaped gap shut. Like so:</p>
</p>
Now they look kind of 3D, like so:</p>
</p>
Let's sew up the lining piece next. It's the bit we cut off of the fold earlier. Fold then sew the Vs shut, thusly:</p>
</p>
Next, sew most of the remaining seam of the lining, but leave a gap at the top so we can turn the whole thing inside out later:</p>
</p>
Now that the lining is sewn, let's sew 10 little legs. Grab that big rectangular strip we cut out at the very beginning, and sew its layers together into a bunch of little triangles with open bottoms. Then cut them apart and turn them inside out to get legs. Here's how I did those steps:</p>
</p>
Those little legs should have taken up maybe 1/3 of big rectangular strip. With the rest of it, let's make some spines to go across Ferris's back. They're little triangles, wider than the legs, sewn up the same way.</p>
</p>
Now put those spines onto one of the outside hat pieces. Leave some room at the bottom, because that's where we'll attach the claws that we'll make later. The spines will stick toward the face when you pin them out, so when the whole thing turns right-side-out after sewing they'll stick out.</p>
</p>
Put the back of the outside onto this spine sandwich you're building. Make sure the seam that sticks out is on the outside, because the outsides of this sandwich will end up inside the hat.</p>
</p>
Pin and sew around the edge:</p>
</p>
Note how the bottoms of the spines make the seam very bulky. Trim them closer to the seam, if you're using a fabric which doesn't fray, such as polar fleece.</p>
</p>
The outer layer of the hat is complete!</p>
</p>
At this point, we remember that Ferris has some claws that we haven't accounted for yet. That's ok because there was some extra fabric left over when we cut out the lining and outer for the hat. On that extra fabric, draw two claws. A claw is just an oval with a pie slice misisng, plus a little stem for the arm. Make sure the arms are wide enough to turn the claw inside out through later. It's ok to draw them straight onto the fabric with a pen, since the pen marks will end up inside the claw later.</p>
</p>
Then sew around the claws. It doesn't have to match the pen lines exactly; nobody will ever know (except the whole internet in this case). Here are the front and back of the cloth sandwich that I sewed claws with:</p>
</p>
Cut them out, being careful not to snip through the stitching when cutting the bit that sticks inward, and turn them right-side out:</p>
</p>
Now it's time to attach the liner and the hat outer together. First we need to pin the arms and legs in, making another sandwich kind of like we did with the spines along the back. I like to pin the arms sticking straight up and covering the outer's side seams, like so:</p>
</p>
Remember those 10 little legs we sewed earlier? Well, we need those now. And I used an extra spine from when we sewed the spines along Ferris's back, in the center back, as a tail. Pin them on, 5 on each side, like little legs.</p>
</p>
And finally, remember that liner we sewed, with a hole in the middle? Go find that one real quick:</p>
</p>
Now we're going to put the whole hat outer inside of the lining, creating Ferris The Bowl. All the pretty sides of things are INSIDE the sandwich, so all the seam allowances are visible.</p>
</p>
Rearrange your pins to allow sewing, then sew around the entire rim of Ferris The Bowl.</p>
</p>
Snip off the extra bits of the legs and stuff, just like we snipped off the extra bits of the spines before, like this:</p>
</p>
Now Ferris The Bowl is more like Ferris The Football:</p>
</p>
Reach in through the hole in the end of Ferris The Football, grab the other end, and pull. First it'll look like this...</p>
</p>
And then he'll look like this:</p>
</p>
Sew shut that hole in the bottom of the lining...</p>
</p>
Stuff that lining into the hat, to make the whole thing hat-shaped, and you're done!</p>
</p>


When searching an error fails
2019-02-28T00:00:00+00:00
This blog has seen a dearth of posts lately, in part because my standard post formula is "a public thing had a poorly documented problem whose solution seems worth exposing to search engines". In my present role, the tools I troubleshoot are more often private or so local that the best place to put such docs has been an internal wiki or their own READMEs.</p>
This change of ecosystem has caused me to spend more time addressing a different kind of error: Those which one really can't just Google.</p>
Sometimes, especially if it's something that worked fine on another system and is mysteriously not working any more, the problem can be forehead-slappingly obvious in retrospect. Here are some general steps to catch an "oops, that was obvious" fix as soon as possible.</p>
Find the command that yielded the error</h1>
First, I identify what tool I'm trying to use. Ops tools are often an amalgam of several disparate tools glued together by a script or automation. This alias invokes that call to SSH, this internal tool wraps that API with an appropriate set of arguments by ascertaining them from its context. If I think that SSH, or the API, is having a problem, the first troubleshooting step is to figure out exactly what my toolchain fed into it. Then I can run that from my own terminal, and either observe a more actionable error or have something that can be compared against some reliable documentation.</p>
Wrappers often elide some or all of the actual error messages that they receive. I ran into this quite recently when a multi-part shell command run by a script was silently failing, but running the ssh portion of that command in isolation yielded a helpful and familiar error that prompted me to add the appropriate key to my ssh-agent, which in turn allowed the entire script to run properly.</p>
Make sure the version "should" work</h1>
Identifying the tool also lets me figure out where that tool's source lives. Finding the source is essential for the next troubleshooting steps that I take.:</p>
$ which toolname
</span>$ toolname -version #
</span></code></pre>
I look for hints about whether the version of the tool that I'm using is supposed to be able to do the thing I'm asking it to do. Sometimes my version of the tool might be too new. This can be the case when the dates on all the docs that suggest it's supposed to work the way it's failing are more than a year or so old. If I suspect I might be on too new a version, I can find a list of releases near the tool's source and try one from around the date of the docs.</p>
More often, my version of a custom tool has fallen behind. If the date of the docs claiming the tool should work is recent, and the date of my local version is old, updating is an obvious next step.</p>
If the tool was installed in a way other than my system package manager, I also check its README for hints about the versions of any dependencies it might expect, and make sure that it has those available on the system I'm running it from.</p>
Look for interference from settings</h1>
Once I have something that seems like the right version of the tool, I check the way its README or other docs looked as of the installed version, and note any config files that might be informing its behavior. Some tooling cares about settings in an individual file; some cares about certain environment variables; some cares about a dotfile nearby on the file system; some cares about configs stored somewhere in the homedir of the user invoking it. Many heed several of the above, usually prioritizing the nearest (env vars and local settings) over the more distant (system-wide settings).</p>
Check permissions</h1>
Issues where the user running a script has inappropriate permissions are usually obvious on the local filesystem, but verifying that you're trying to do a task as a user allowed to do it is more complicated in the cloud. Especially when trying to do something that's never worked before, it can be helpful to attempt to do the same task as your script manually through the cloud service's web interface. If it lets you, you narrow down the possible sources of the problem; if it fails, it often does so with a far more human-friendly message than when you get the same failure through an API.</p>
Trace the error through the source</h1>
I know where the error came from, I have the right versions of the tool and its dependencies, no settings are interfering with the tool's operation, and permissions are set such that the tool should be able to succeed. When all this normal, generic troubleshooting has failed, it's time to trace the error through the tool's source.</p>
This is straightforward when I'm fortunate enough to have a copy of that source: I pick some string from the error message that looks like it'll always be the same for that particular error, and search it in the source. If there are dozens of hits, either the tool is aflame with technical debt or I picked a bad search string.</p>
Locating what ran right before things broke leads to the part of the source that encodes the particular assumptions that the program makes about its environment, which can sometimes point out that I failed to meet one. Sometimes, I find that the error looked unfamiliar because it was actually escalated from some other program wrapped by the tool that showed it to me, in which case I restart this troubleshooting process from the beginning on that tool.</p>
Sometimes, when none of the aforementioned problems is to blame, I discover that the problem arose from a mismatch between documentation and the program's functionality. In these cases, it's often the docs that were "right", and the proper solution is to point out the issue to the tool's developers and possibly offer a patch. When the code's behavior differs from the docs' claims, a patch to one or the other is always necessary.</p>


Running a Python3 script in the right place every time
2018-09-19T00:00:00+00:00
I just wrote a thing in a private repo that I suspect I'll want to use again later, so I'll drop it here.</p>
The situation is that there's a repo, and I'm writing a script which shall live in the repo and assist users with copying a project skeleton into its own directory.</p>
The script, newproject</code>, lives in the bin</code> directory within the repo.</p>
The script needs to do things from the root of the repository for the paths of its file copying and renaming operations to be correct.</p>
If it was invoked from somewhere other than the root of the repo, it must thus change directory to the root of the repo before doing any other operations.</p>
The snippet that I've tested to meet these constraints is:</p>
# chdir to the root of the repo if needed
</span>if __file__.endswith("/bin/newproject"):
</span>    os.chdir(__file__.strip("/bin/newproject"))
</span>if __file__ == "newproject":
</span>    os.chdir("..")
</span></code></pre>
In code review, it was pointed out that this simplifies to a one-liner:</p>
os.chdir(os.path.join(os.path.dirname(__file__), '..'))
</span></code></pre>
This will keep working right up until some malicious or misled individual moves the script to an entirely different location within the repository or filesystem and tries to run it from there.</p>


CFP tricks 1
2018-09-17T00:00:00+00:00
Or, "how to make a selection committee do one of the hard parts of your job as a speaker for you". For values of "hard parts" that include fine-tuning your talk for your audience.</p>
I'm giving talk advice to a friend today, which means I'm thinking about talk advice and realizing it's applicable to lots of speakers, which means I'm writing a blog post.</p>
Why choosing an audience is hard</h1>
Deciding who you're speaking to is one of the trickiest bits of writing an abstract, because a good abstract is tailored to bring in people who will be interested in and benefit from your talk. One of the reasons that it's extra hard for a speaker to choose the right audience, especially at a conference that they haven't attended before, is because they're not sure who'll be at the conference or track.</p>
Knowing your audience lets you write an abstract full of relevant and interesting questions that your talk will answer. Not only do these questions show that you can teach your subject matter, but they're an invaluable resource for assessing your own slides to make sure your talk delivers everything that you promised it would!</p>
Tricks for choosing an audience</h1>
Some strategies I've recommended in the past for dealing with this include looking at the conference's marketing materials to imagine who they would interest, and examining the abstracts of past years' talks.</p>
Make the committee choose by submitting multiple proposals</h1>
Once you narrow down the possible audiences, a good way to get the right talk in is to offload the final choice onto the selection committee! A classic example is to offer both a "Beginner" and an "Advanced" talk, on the same topic, so that the committee can pick whichever they think will be a better fit for the audience they're targeting and the track they choose to schedule you for.</p>
If the CFP allows notes to the committee, it can be helpful to add a note about how your talks are different, especially if their titles are similar: "This is an introduction to Foo, whereas my other proposal is a deep dive into Foo's Bar and Baz".</p>
Use the organizers' own words</h1>
I always encourage resume writers to use the same buzzwords as the job posting to which they're applying when possible. This shows that you're paying attention, and makes it easy for readers to tell that you meet their criteria.</p>
In the same way, if your talk concept ties into a buzzword that the organizers have used to describe their conference, or directly answers a question that their marketing materials claim the conference will answer, don't be afraid to repeat those words!</p>
When choosing between several possible talk titles, keep in mind that any jargon you use can show off your ability, or lack thereof, to relate to the conference's target audience. For instance, a talk with "Hacking" in the title may be at an advantage in an infosec conference but at a disadvantage in a highly professional corporate conf. Another example is that spinning my Rust Community Automation talk to "Life is Better with Rust's Community Automation" worked great for a conference whose tagline and theme was "Life is Better with Linux", but would not have been as successful elsewhere.</p>
Good luck with your talks!</p>


Skill Tree Balancing with a Job Move
2018-08-24T00:00:00+00:00
I’ve recently identified some ways in which my former role wasn’t setting me up for career success, and taken steps to remedy them. Since not everybody lucks into this kind of process like I did, I’d like to write a bit about what I’ve learned in case it offers some reader a useful new framework for thinking about their skills and career growth.</p>
Tl;dr:</strong> I’m moving from Research to Cloud Ops within Mozilla. The following wall of text and silly picture are a brain dump of new ideas about skills and career growth that I’ve built through the process.</p>
Managers as Mentors</h1>
Oddly enough, I can trace the series of causes and effects that ultimately culminated in my decision to switch roles straight back to a company that I’ve never even worked at. By shaping my manager’s leadership skills, Microsoft’s policies have trickled down into benefits to my open source focused career at Mozilla.</p>
Apparently, managers at Microsoft have targets to meet for their reports’ career growth and advancement, as well as the usual product and performance metrics. I’ve heard that a manager there who meets all their other deliverables but fails to promote the people under them can be demoted for that negligence! I’ve learned about this culture because it showed up in my own ex-Microsoft manager as a refusal to take “I’m happy where I am and don’t think I particularly need to go seek a promotion” as a complete answer when he asked about my goals for career growth and progression.</p>
I have many peers whose managers seem fine with “I’m happy where I am” as an answer to career goal questions. Replacing an employee for their lower-level duties as they graduate to higher-level ones can be between inconvenient and almost impossible for a boss, so it can seem to be in everybody’s best interests to leave someone for a long time when they find a role where they’re content. But my own manager’s ingrained distaste for allowing that kind of stagnation led to a series of conversations that forced me to look at the bigger picture of my career, and realize the ways that sticking with a role that’s “good enough for now” could cause me serious problems years down the road.</p>
Complacency and the Same Year of Work</h1>
I’ve been warned over and over by mentors through the industry that it’s a terrible idea to do the “same year of work” over and over. I’ve even repeated that advice to others – “don’t tolerate a job that isn’t helping you grow! Keep moving roles till you find something you love!”. This advice is especially easy to follow when the “same year of work” you’re repeating is unpleasant in some way.</p>
However, I found myself in a role at Mozilla Research where the “same year of work” is totally amazing! 3 times in a row, I came into a team with major ops needs, and delivered value in a way that I’m great at to a team of awesome people with expertise in areas where my knowledge is only superficial. And it feels great to help them out and solve problems with knowledge that’s old hat for me but novel to my colleagues. For each role, my progress eventually reached a point where I’d solved all the problems that I could fix in a timely manner – I’d hit the edge of my skills at the time. Although it’s educational, working outside my skill set eventually slowed down my performance and work quality to the point where it was better to have me shift focus to a group whose needs were within what I could execute rapidly and with high quality.</p>
The parts of this cycle in which I’m attempting a load of tasks primarily outside my skill set (and often outside the set of skills that I have the prerequisite skills to bootstrap myself into in a timely manner) can be stressful, but on the whole it’s been a rewarding experience. I’ve felt like I’ve gotten a “new role” every year or so, on about the timeline that a peer starting with similar skills to my own might have had to switch companies to find new opportunities. Without my manager’s guidance, I might have continued in “same year of work” cycle indefinitely, but our conversations about career progression helped me improve my understanding of how I personally grow as an engineer.</p>
Self-Teachable Skills</h1>
What’s that “set of skills that I have the prerequisite skills to bootstrap myself into in a timely manner” that I mentioned earlier? They’re those challenges which I can identify and frame in a way which lets me ask the kind of questions that get actionable answers. They’re knowledge gaps directly adjacent to my established areas of expertise, about which I can concisely frame questions whose answers I can use to accomplish a task. Despite being a “team of one”, I’ve successfully self-taught skills such as learning a new configuration management system and improving my FOSS community management knowledge to solve particular challenges. Since I was already more skilled than the average entry-level engineer in those areas when I started my solo ops role, I had good heuristics for framing questions in a way that was likely to yield the answers I wanted, and evaluating the credibility and applicability of sources to learn from.</p>
In other words, I’ve found that I can be successful at self-teaching a skill when I can write out a template for what I know the correct answer to a given question will look like. I might not know precisely which config management system will meet my needs, but I already know a variety of pain points to look out for and how to assess whether an example given in a tool’s docs is similar to the task I’m looking to apply the tool on. I might not know exactly what community management strategy is best for a given FOSS challenge, but I know enough about community management in general to frame a question about what strategy to use with all the relevant information and propose several viable options to ask an expert for their opinions in selecting between.</p>
Non-Self-Teachable skills</h1>
I’ve found that, in working as a team of one, I’ve done pretty well at growing the skills that I had the appropriate foundation to “bootstrap” or “self-teach”. However, there are other skills for which this hasn’t been the case. Although these types of skills can be self-taught very slowly and inefficiently, I’m going to nickname them “non-self-teachable” for purposes of this discussion. “Infeasible-to-self-teach”, while technically more accurate, is just too much of a tangle to type out.</p>
For skills with whose problem space I’m not fully acquainted, I find it extremely challenging to improve in a timely manner in isolation. In areas like prioritizing work across projects, and project planning, I find that my ability to self-teach is limited when I don’t have sufficient expertise to ask for help efficiently. When I don’t know what sort of answer would be actionable, I struggle to frame general questions and figure out where to ask them. And when I lack the expertise to identify what specific tasks I should be asking for help to improve at, I miss many opportunities to seek help.</p>
How did I discover these “unknown unknowns” of skills that I lack the foundation to teach myself efficiently? I started noticing them when seeking resources to understand my career growth options in order to refine my career goals. In an internal wiki, I found a document which gives broad outlines for and descriptions of the competence and impact of employees at each pay grade. Mozilla’s is called a Job Family Architecture; other companies have similar documents with different names. Comparing my performance to the descriptions of higher levels helped me identify and articulate some particular skill areas that I haven’t improved at the same rate as those where I’ve successfully self-taught. Quantifying these areas that I’d like to work on has helped me figure out what changes I could make to get into circumstances more like those where I’ve improved rapidly at the “problem” skills in the past.</p>
Bootstrapping Non-Self-Teachable Skills</h1>
When an engineer enters their very first job or internship, or touches a programming language for the first time, it’s reasonable to assume that all the skills they need to build are in the category that I’m describing as non-self-teachable. Skills I’m currently expert at, and even teach to others, started out as non-self-teachable for me as well. So what changed? How did I go from having no idea what questions to ask about Git, to being able to solve even its gnarliest problems with only a handful of StackOverflow queries or man page checks?</p>
When I look back at my career, the common denominators among times I’ve built any skill from non-self-teachable to self-teachable have been peers and mentors. Watching the way that a teammate addresses a challenge and comparing it to the way I would have approached it gives me new insights at a rate that no amount of struggling with the skill by myself ever could. And managers or mentors who are subject matter experts in my field can compare the way I approach a task to the way they would have, and point out differences to yield skill-improving feedback. When I work closely enough with an expert for a while , I build a mental model of the way they think, and for the rest of my career I can ask myself “How would they have architected this? How would they have tackled this problem?”.</p>
These mental models can simulate a team in circumstances that are similar to those where I worked with the experts, but as I advance into novel work in isolation, the models become less and less useful because I can’t predict how the expert would have approached a task unlike anything I’ve seen them encounter.</p>
Bridging My Skill Gaps</h1>
Once I identified my skill gaps, I first attempted to seek better mentorship in my existing role. After several different attempts, I determined that a mentor’s intimacy with the particular task they’re giving feedback on is integral to their ability to help me refine a technical skill. In both individual and group mentorship, I found it easy to refine skills that I could already self-teach, but between difficult and impossible to get good help on skills about which I wasn’t yet expert enough to frame good questions. Doing my best to improve within my existing role helped me figure out which skills it was reasonable to try to improve in place, and which others were infeasible to build in a timely manner under the circumstances.</p>
I talked with my manager about what I’d learned about myself from the various attempts to build the skills I regard as needing work. We evaluated our options for putting me into a situation where I had the peers and mentorship that I’m looking for: What would it look like to change my current role so it had more peers? What would it look like to move me into a role on a team of ops folks elsewhere? When we took into account some other skill gaps that I’m interested in addressing, such as working with experts on infrastructure at larger scale, we concluded that the best step to address my current concerns was to explore my options for shifting to a different team.</p>
At this point, I literally wrote down a list of relevant skills in my notebook. I brainstormed a section of skills where I feel I’m below where I’d like to be and would learn best from peers. I also outlined the skills I’ve succeeded with in my present role, and the skills at which I consider myself above average and thus am not worried about aggressively growing while I focus on improvement in other areas.</p>
I summarized my lists into the 2 key reasons I’m interested in a role change. Those reasons are: I want to work with peers and mentors who can offer detailed technical feedback based on their expertise at the problems I’m solving, and I want to refine my prioritization and planning skills by being more closely exposed to good examples in work like mine.</p>
As I sought other teams that might meet these needs for me and interviewed with them, I kept my lists and summary on my desk. I felt that having them in sight made a real difference in my ability to clearly articulate my interest in a role change, and helped me ask the right questions to determine whether any team looked like a good match for my priorities. As I interviewed, I paid attention to not only the technical topics that we discussed, but the prospective colleagues’ attitude toward the answers I got wrong. On the team I ended up selecting, I was especially impressed by the way that my future peers ended each question by filling in any gaps between my answer and the complete or best possible answer that they were “looking for”. This made the interviews feel like a constructive conversation, and even if I hadn’t taken the role, I would have left with a better understanding of the technologies we discussed.</p>
New Team</h1>
With all that, I’m excited to announce that I’m transitioning from Research to the Cloud Ops team within Mozilla! This team supports a multitude of projects, and there are always at least 2 ops engineers in a “buddy system” for every project they support. The new role is similar to my old one in that it juggles supporting many projects at once, but very different in that I’ll be working directly with expert colleagues to learn their best ways to do it.</p>
And yeah, I’m staying at Mozilla. It might net me more cash to jump ship to another company, but monetary compensation is not what this move is optimizing for. The drawbacks I would experience if I chose a team at another company include greater uncertainty about what the team is actually like, and having to re-learn all the specialized bureaucracy that comes with onboarding anywhere. I also encounter very few other companies whose cultures are as closely aligned with my own values, expecially pertaining to open source, and none of them currently have openings that I can confirm are as good a match to my current skill building goals as the Cloud Ops team.</p>
Visualization</h1>
Drawing a picture is one of my favorite ways to gain control of and ask the right questions about new knowledge. Here’s an ugly little chart into which I’ve thrown a handful of skill areas and my approximate levels at them before and after my old role… and of course, the skill emphases that have been important to me in picking the right place to go next.</p>
</p>
If you want to make a similar chart, I made that one with the net chart type</a> in LibreOffice.</p>


Why an ops career
2018-07-20T00:00:00+00:00
Disclaimers: Not all tasks that come to a person in an ops role meet my definition of ops tasks. Advanced ops teams move on from simple problems and choose more complex problems to solve, for a variety of reasons. This post contains generalizations, and all generalizations have counter-examples. This post also refers to feelings, and humans often experience different feelings in response to similar stimuli, so yours might not be like mine.</em></p>
It's been a great "family reunion" of FOSS colleagues and peers in the OSCON hallway track this week. I had a conversation recently in which I was asked "Why did you choose ops as a career path?", and this caused me to notice that I've never blogged about this rationale before.</p>
I work in roles revolving around software and engineering because they fall into a cultural sweet spot offering smart and interesting colleagues, opportunities for great work-life balance, and exemplary compensation. I also happen to have taken the opportunity to spend over a decade building my skills and reputation in this industry, which helps me keep the desirable roles and avoid the undesirable ones. Yet, many people in my field prefer software development over operations work.</p>
I've chosen, and stuck with, ops because it gives me the sensation of having better-defined success conditions than I get when developing code for others. When I tackle an ops problem, it is usually a task which I could tediously, miserably, but correctly perform by hand. This base case of "if all else fails, the desired thing can be done by hand" frames a problem more concretely and measurably than any written description of someone's hopes and dreams about a piece of software.</p>
Of course, performing ops tasks by hand does not scale. Often, the speed with which a given task is performed is part of its success criteria. And if you ask a human to perform the same task 20 times, you'll likely get 21 subtly different outputs. This is why we automate: Automation brings computers' strengths of speed and lack of boredom to the equation.</p>
Automation tasks are necessarily framed in terms of the specific behaviors, described in technical terms, that computers are supposed to be performing.</strong> The constraints of the infrastructure provide a rigorously defined abstraction layer between psychology and code. This vocabulary of infrastructure expresses the constraints for ops work such that even if I'm not the end user of a piece of automation code, I can experience a high level of confidence that I understand what the person requesting it believed that they wanted when they made the request.</p>
Automation is unlike software engineering tasks with success conditions that hinge on human emotions and behavior. Any success condition with psychology integral to it becomes time-consuming, if not impossible, to test against. Throw in psychological effects that incline a human to have slightly different reactions to the same thing depending on when and how you show it to them, and you lose even basic repeatability from the simple task of testing whether your code is "good enough". For software engineering tasks with human behavior and emotions in their success criteria, I cannot consistently prove to myself that success is even possible. Although I enjoy recreationally tackling potentially-impossible challenges from time to time, I do not enjoy the pressure and uncertainty that come from betting my career and compensation on such puzzles.</p>
Even systems built solely from understandable components develop complexity and challenges. Emergent behaviors arise, perhaps necessarily, from complex systems. In ops work, I feel a certainty that each component is independently predictable when broken down small enough, and that it would be possible with enough work to rebuild the entire system incrementally from such "atoms" of predictability. Of course it is almost never worth the time and effort to actually rebuild the system from scratch, but simply knowing that it would be possible gives me confidence that the problems I encounter with my systems can be solved. (To reiterate, "can be solved" bears little relation to "is worth solving", but it does affect the way I feel about tasks.) Contrast this "certainty of solvability" to the problems encountered when developing software for other people: "the customer doesn't like this!", "users aren't clicking where we want them to!". Those problems hinge on human components that would usually be highly unpleasant, unethical, and illegal to disassemble and debug. Software problems tightly coupled to psychology do not make me feel like I can be certain that any</em> amount of effort would guarantee a solution.</p>
No workflow can, nor should, eliminate the bigger-picture questions about what we want to be building, or how we want to go about building it. However, I find that the structure of roles that companies typically categorize as "ops work" supports decoupling the questions without answers from the questions with answers, and offloading the half you don't want to deal with onto someone who enjoys them more.</p>


Thoughts on retiring from a team
2018-05-15T00:00:00+00:00
The Rust Community Team has recently been having a conversation about what a team member's "retirement" can or should look like. I used to be quite active on the team but now find myself without the time to contribute much, so I’m helping pioneer the “retirement” process. I’ve been talking with our subteam lead extensively about how to best do this, in a way that sets the right expectations and keeps the team membership experience great for everyone.</p>
Nota bene: This post talks about feelings and opinions. They are mine and not meant to represent anybody else’s.</p>
Why join a team?</h1>
When I joined the Rust community subteam, its purpose was defined vaguely. It was a small group of “people who do community stuff”, and needed all the extra hands it could get. A lot of my time was devoted explicitly to Rust and Rust-related tasks. The tasks that I was doing anyways seemed closely aligned with the community team’s work, so stepping up as a team contributor made a lot of sense. Additionally, the team was so new that the only real story for “how to work with this team and contribute to its work” was “join the team” . We hadn’t yet pioneered the subteams and collaboration with community organizers outside the official community team which are now multiplying the team’s impact.</p>
Why leave?</h1>
I’m grateful to the people who have the bandwidth and interest to put consistent work into participating on Rust’s community team today. As the team has grown and matured, its role has transitioned from “do community tasks” to “support and coordinate the many people doing those tasks”. I neither enjoy nor excel at such coordination tasks. Not only do I have less time to devote to Rust stuff, but the community team’s work has naturally grown into higher-impact categories that I personally find less fulfilling and more exhausting to work on.</p>
Teams and people change</h1>
In a way, the team’s growth and refinement over the years reminds me of a microcosm of what I saw while working at a former startup as it built up into an enterprise company. Some peoples’ working style had been excellently suited to the 5-person company they originally joined, but clashed with the 50-person company into which that startup grew. Others who would never have thrived in a company of only 10 people were hiring on and having a fantastic impact scaling the company up to 1,000. And some were fine when the company was small and didn’t mind being part of a larger organization either. That experience reminds me that the fit between a person and organization at some point in the past does not guarantee that they’ll remain a good fit for each other over time, and neither is necessarily to blame for the eventual mismatch as both grow and change.</p>
Does leaving harm anyone?</h1>
When you’re appreciated and valued for the work you do on a team, it’s easy to get the idea that the team would be harmed if you left. The tyres on my bike are a Very Important Part of the bike, and if I took them off, the bike wouldn’t be rideable. But a team isn’t just a machine – a team’s impact is an emergent phenomenon that comes out of many factors, not a static item. If a sports team has a really excellent coach, they’ll retain the lessons they learned from that coach’s mentorship even after the coach moves away. Older players will pass along the coach’s lessons to younger ones, and their ideas will stick around and improve the group even long after the original players’ retirement. When a team is coordinated well, one member leaving doesn’t hurt it. And if I leave on good terms rather than sticking around till I burn out or burn bridges, I can always be available for remaining members to consult when if need advice that only I can provide.</p>
Would staying harm anyone?</h1>
I think that in the case of the Rust community team, it would reflect poorly on the community as a whole if the exact same people comprised the community team for the entire life of the language.</p>
If nobody new ever joins the team, we wouldn’t get new ideas and tactics, nor the priceless infusion of fresh patience and optimism that new team members bring to our perennial challenges and frustrations. So, new team members are essential. If new people joined on a regular basis but nobody ever left, the team would grow unboundedly large as time went on, and have you ever tried to get anything done with a hundred- or thousand-person committee? In my opinion, having established team members retire every now and then is an essential factor in preventing either of those undesirable hypotheticals.</p>
The team selects for members who’ll step up and accomplish tasks when they need to. I think establishing turnover in a healthy and sustainable way is one of the most essential tasks for the team to build its skills at. The best way to get a healthy amount of turnover – not too much, but not too little either – is for every team member to step up to the personal challenge of identifying the best time to retire from active involvement. And for me, that happens to look like right now.</p>
Aspirational Clutter</h1>
Do you have stuff in your house that you don’t use, and it’s taking up space, and you’re kind of annoyed at it for taking up space, but you don’t feel like you can get rid of it because you think you really should use it, or you’re sure you’re just going to make some personal change that will cause you to use it someday? I call that stuff aspirational clutter: It doesn’t belong to you, it belongs to some imaginary person who doesn’t exist but you aspire to become them someday.</p>
A team meeting every week on your agenda can be aspirational clutter in the same way as a jumbled shelf of planners or a pile of sports gear covering a treadmill: It not only isn’t a good fit for who you are right now, but by wasting time or space it actually gets in the way of the habits and changes that would make you more like that person you aspire to be.</p>
I find few experiences more existentially miserable than feeling obliged to promise work that I know I’ll lack the resources of time or energy to deliver. Sticking around on a team that I’m no longer a good fit for puts me in a situation where get to choose between feeling guilty if I don’t promise to get any work done, or feeling like a disappointment for letting others down if I commit to more than I’m able to deliver. Those aren’t feelings I want to experience, and I can avoid them easily by being honest with myself about the amount of time and energy I have available to commit to the team.</p>
The benefits of contributing from a non-team-member role</h1>
One scary idea that comes up when leaving a team is the question: “if I’m not on the team, how can I help with the team’s work?”.</p>
In my opinion, it builds a healthier community if people who are good at a given team’s work spend some time interfacing with the team from the perspective of non-team-members. If I know how the community team gets stuff done and I go “undercover” as a non-team-member coming to them for help, I can give them essential feedback to improve the experience and processes that non-team-members encounter.</p>
When I wear my non-team-member hat and try to get stuff done, I learn what it’s like for everyone else who tries to interface with the team. I can then use the skills that I built on by participating on the team to remedy any challenges that a non-team-member encounters. Those changes create a better experience for every community member who interacts with the team afterwards.</p>
What next?</h1>
As a community team alum, I’ll keep doing the Rust outreach – the meetup organizing, the conference talks, the cute swag, the stickers – that I’ve been doing all along. Stepping down from the official team member list just formalizes the state that my involvement has been in for the past year or so: Although I get the community team’s support for my endeavors when I need it, I’m not invested in the challenges of supporting others’ work which the team is now tackling.</p>
I’m proud of the impact that the team has had while I’ve been a part of it, and I look forward to seeing what it will continue to accomplish. I'm grateful for all the leadership and hard work that have gone into making the Rust community subteam an organization from which I can step back while remaining confident that it will keep excelling and evolving.</p>
Why blog all that?</h1>
I’m publishing my thoughts on leaving in the hopes that they can help you, dear reader, gain some perspective on your own commitments and curate them in whatever way is best for you.</p>
If you read this and feel pressured to leave something you love and find fulfilling, please try to forget you ever saw in this post.</p>
If you read this hoping it would give you some excuse to quit a burdensome commitment and feel disappointed that I didn’t provide one, here it is now: You don’t need a fancy eloquent excuse to stop doing something if you don't want to any more. Replace unfulfilling pursuits with better ones.</p>


Slacking from Irssi
2018-02-23T00:00:00+00:00
UPDATE: SLACK DECIDED THIS SHOULD NO LONGER BE POSSIBLE AND IT WILL NOT WORK ANY MORE</p>
My IRC client helps me work efficiently and minimize distraction. Over the years, I've configured it to behave exactly how I want: Notifying me when topics I care about are under discussion, and equally as important, refraining from notifications that I don't want. Since my IRC client is developed under the GPL, I have confidence that the effort I put into customizing it to improve my workflow will never be thrown out by a proprietary tool's business decisions.</p>
But the point of chat is to talk to other humans, and a lot of humans these days are choosing to collaborate on Slack. Slack has its pros and cons, but some of the drawbacks can be worked around using open technologies.</p>
Do you need to Slack from irssi? ------------------------------</p>
If you feel ambivalent toward the web UI, going to the trouble of setting up an IRC client for Slack will likely be more hassle than reward.</p>
If you loathe the Slack UI but don't care much for IRC, you might be better off considering the Matrix bridge</a> or seeking out other clients. I have not tried them and can't vouch for any.</p>
If you use WeeChat, you can use Slack IRC directly, or try the WeeChat Slack gateway</a>. A friend at a larger company informs me that the latter doesn't hold up well on Slack workspaces in the tens of thousands of users, so if you're on a particularly large Slack you might be better off treating it as an IRC server.</p>
If you're looking for a terminal-based client to get started with IRC, consider choosing WeeChat over Irssi. WeeChat is extensible in more languages and allegedly has fewer edge cases like the saga detailed here.</p>
The IRC bridge won't work on all Slacks</h1>
First, a word of warning: You can only Slack from IRC if a workspace owner enables the IRC/XMPP gateway for a given instance, which is disabled by default because Slack distrusts users' ability to make good security decisions. They're not necessarily wrong, but generally users who care deeply enough to slog through their misleading instructions also know a thing or two about SSL.</p>
This will work on all IRC clients, But...</h1>
The good news is that once a Slack instance is exposing the IRC gateway, it looks to an IRC client just like a single standalone IRC server.</p>
The bad news is that the instructions Slack provides</a> will only work if you do them on an IRC client where you weren't connected to any other IRC yet. This is because (at time of writing) they tell you only to add Slack as a server in your client.</p>
Irssi is "special".</h1>
Let's step back to the basics of how IRC works</a> for a minute: On IRC, a network</em> is a group of servers</em>, and on a given network you can join various channels. Being on a network is intentionally agnostic of what server you're connected to</em>, so that if one server goes away, you can connect to another and keep right on chatting.</p>
All modern IRC clients that I'm aware of allow you to be connected to several networks at once. For instance I'm on the Freenode IRC network to talk to people about FOSS projects I use, and also the Mozilla IRC network because most of my work channels are there. Every command you issue to irssi is done in the context of some network -- do you want to auth to services? Join a new channel? Add a server? Irssi assumes that the server of the buffer in which you issue a command is the server you want the command to apply to, though some might require you to explicitly specify the network.</p>
So, if you're already on a network and you tell your client to add a server, what will happen? That's right, your client will do the smart thing and add the server to the network</em>. It will also likely connect you to that server, and do all the things on that server which you've asked it to do when you first connect to that particular network.</p>
What happens if that new server is not part of the network in question at all, but is instead Slack? irssi will do the automatic things, like joining channels and trying to auth to services, that it's supposed to do when it joins that network anyways, because you as a human told the client that the server was on the network. Even if that's because some mean ol' documentation tricked you into it, irssi doesn't know any better.</p>
What happens when irssi autojoins a bunch of channels, but issues those join commands to the Slack server? Well, on Slack just as on IRC, the first person to join a channel creates it</a>. So, you have just revealed to your whole Slack workspace exactly the names of the channels you were in on the IRC server from which you issued the /server</code> command.</p>
Spuriously creating a bunch of channels isn't the end of the world, you can just delete them</a>, right? Well, if you have owner or admin permissions on the Slack workspace, absolutely! If you are not an owner or admin, you will have to go find someone who is and ask them to clean up the mess.</p>
Well, at least that's what happens when an active IRC user blindly assumes that whoever wrote the Slack IRC connection instructions had tried them in an irssi instance they were actually using for IRC. My bad.</p>
Don't follow the Slack docs verbatim from Irssi.</h1>
When you're looking at https://my.slack.com/account/gateways</a>, it has instructions like the following:</p>

Ensure that your IRC client is configured with your normal Slack username as your nick.</li>
If you are connecting through a raw /server</code> command, your command will be: /server myserver.irc.slack.com 6667 myserver.Nosh5Neevot5Efua</code></li>
If you have a more UI-oriented setup, your IRC server is myserver.irc.slack.com</code>, and the server password is myserver.Nosh5Neevot5Efua</code>. Accepted ports are 6667, 6697, and 8000.</li>
</ol>
That Nosh5Neevot5Efua</code> bit is a password you shouldn't share with anyone --for this post I'm using a string from pwgen so it looks more like the actual config.</p>
To avoid the tale of woe that I outlined above, if you're slacking from Irssi, you need to add a network</em> before adding the server. This changes the steps to:</p>

Ensure that your IRC client is configured with your normal Slack username as your nick.</li>
Add a network with the command /network add myslack</code></li>
If you are connecting through a raw /server</code> command, your command will be: /server add -auto -network myslack myserver.irc.slack.com 6667 myserver.Nosh5Neevot5Efua</code></li>
</ol>
See the irssi docs</a> for more options. Join the desired channels on the Slack network just as you would in IRC. When you're done, remember to /save</code>, and your .irssi/config</code> should contain something like:</p>
servers = (
</span>  ...
</span>  {
</span>    address = "myserver.irc.slack.com";
</span>    chatnet = "myslack";
</span>    port = "6697";
</span>    use_ssl = "yes";
</span>    ssl_verify = "no";
</span>    autoconnect = "yes";
</span>    password = "mozilla.Nosh5Neevot5Efua";
</span>  }
</span>);
</span>
</span>chatnets = {
</span>  ...
</span>  myslack = { type = "IRC"; };
</span>};
</span>
</span>channels = (
</span>  ...
</span>  { name = "#slackchannel"; chatnet = "myslack"; autojoin = "yes"; },
</span>  ...
</span>)
</span></code></pre>
Now Slack is almost IRC</h1>
With the bridge set up, Slack behaves mostly like IRC. There remain some outstanding differences:</p>

You cannot leave the workspace's default channel. You can mute the channel</a> or turn off notifications</a> but Slack won't let you leave.</li>
When someone uses @here</code> in a channel, Slack appends your username to the end of the message when forwarding it along to IRC to make sure you get pinged. The person did not actually type your nick when it occurrs in this context.</li>
If you want to hilight a Slack user in a message, you must inlcude the @</code> in their username. If you just say the string of their name, they won't get notified. This is the opposite of IRC, where it's a newbie mistake to include someone's hat when addressing them.</li>
Slack has message threading</a> and allows editing and deleting messages</a>, neither of which are really a thing on IRC. Remember that Slack sends the first version of each message to the IRC bridge. Messages in a thread will look like they were sent to the channel. Messages that were later deleted will persist in your IRC logs. Edits won't show up; IRC bridge users see only the first version of each. If you need to view an edited message or edit or delete your message, you have to use the Slack UI.</li>
</ul>
Have fun!</p>


Some northwest area tech conferences and their approximate dates
2017-12-12T00:00:00+00:00
Somebody asked me recently about what conferences a developer in the pacific northwest looking to attend more FOSS events should consider. Here's an incomplete list of conferences I've attended or hear good things about, plus the approximate times of year to expect their CFPs.</p>
The Southern California Linux Expo (SCaLE) is a large, established Linux and FOSS conference in Pasadena, California. Look for its CFP at socallinuxexpo.org in September, and expect the conference to be scheduled in late February or early March each year.</p>
If you don't mind a short flight inland, OpenWest is a similar conference held in Utah each year. Look for its CFP in March at openwest.org, and expect the conference to happen around July. I especially enjoy the way that OpenWest brings the conference scene to a bunch of fantastic technologists who don't always make it onto the national or international conference circuit.</p>
Moving northward, there are a couple DevOps Days conferences in the area: Look for a PDX DevOps Days CFP around March and conference around September, and keep an eye out in case Boise DevOps Days returns.</p>
If you're into a balance of intersectional community and technical content, consider OSBridge (opensourcebridge.org) held in Portland around June, and OSFeels (osfeels.com) held around July in Seattle.</p>
In Washington state, LinuxFest Northwest (CFP around December, conference around April, linuxfestnorthwest.org) in Bellingham, and SeaGL (seagl.org, CFP around June, conference around October) in Seattle are solid grass-roots FOSS conferences. For infosec in the area, consider toorcamp (toorcamp.toorcon.net, registration around March, conference around June) in the San Juan Islands.</p>
And finally, if a full conference seems like overkill, considering attending a BarCamp event in your area. Portland has CAT BarCamp (catbarcamp.org) at Portland State University around October, and Corvallis has Beaver BarCamp (beaverbarcamp.org) each April.</p>
This is by no means a complete list of conferences in the area, and I haven't even tried to list the myriad specialized events that spring up around any technology. Meetup, and calagator.org for the Portland area, are also great places to find out about meetups and events.</p>


User is not authorized to perform iam:ChangePassword.
2017-11-29T00:00:00+00:00
Summary: A user who is otherwise authorized to change their password may get this error when attempting to change their password to a string which violates the Password Policy in your IAM Account Settings.</p>
So, I was setting up the 3rd or 4th user in a small team's AWS account, and I did the usual: Go to the console, make a user, auto-generate a password for them, tick "force them to change their password on next login", chat them the password and an admonishment to change it ASAP.</p>
It's a compromise between convenience and security that works for us at the moment, since there's all of about 10 minutes during which the throwaway credential could get intercepted by an attacker, and I'd have the instant feedback of "that didn't work" if anyone but the intended recipient performed the password change.</p>
So, the 8th or 10th user I'm setting up, same way as all the others, gets that error on the change password screen: "User is not authorized to perform iam:ChangePassword". Oh no, did I do their permissions wrong? I try explicitly attaching the Amazon's IAMUserChangePassword policy to them, because that should fix their not being authorized, right? Wrong; they try again and they're still "not authorized".</p>
OK, I have their temp password because I just gave it to them, so I'll pop open private browsing and try logging in as them.</p>
When I try putting in the same autogenerated password at the reset screen, I get "Password does not conform to the account password policy.". This makes sense; there's a "prevent password reuse" policy enabled under Account Settings within IAM.</p>
OK, we won't reuse the password. I'll just set it to that most seekrit string, "hunter2". Nope, the "User is not authorized to perform iam:ChangePassword" is back. That's funny, but consistent with the rules just being checked in a slightly funny order.</p>
Then, on a hunch, I try the autogenerated password with a 1 at the end as the new password. It changes just fine and allows me to log in! So, the user did have authorization to change their password all along... they were just getting an actively misleading error message about what was going wrong.</p>
So, if you get this "User is not authorized to perform iam:ChangePassword" error but you should be authorized, take a closer look at the temporary password that was generated for you. Make sure that your new password matches or exceeds the old one for having lowercase letters, uppercase letters, numbers, special characters, and total length.</p>
When poking at it some more, I discovered that one also gets the "User is not authorized to perform iam:ChangePassword" message when one puts an invalid value into the "current password" box on the change password screen. So, check for typos there as well.</p>
This yak shave took about an hour to pin down the fact that it was the contents of the password string generating the permissions error, and I haven't been able to find the error string in any of Amazon's actual documentation, so hopefully I've said "User is not authorized to perform iam:ChangePassword" enough times in this post that it pops up in search results for anyone else frustrated by the same challenge.</p>


Better remote teaming with distributed standups
2017-11-01T00:00:00+00:00
Agile development's artifact of the daily stand-up meeting is a great idea. In theory, the whole team should stand together (sitting or eating makes meetings take too long) for about 5 minutes every morning. Each person should comment on:</p>

What they did since yesterday</li>
What they plan on doing today</li>
Any blockers, thigns they're waiting on to be able to get work done</li>
Anything else</li>
</ul>
And then, 5 minutes later, everybody gets back to work. But do they really?</p>
Problems with in-person standups</h1>
When I've participated in stand-up meetings in person, I've noticed a few major flaws:</p>

Context switching into and out of the meeting impacts a maker's schedule</a> by substantially more than the meeting's planned 5 minutes.</li>
People naturally tend to problem-solve</strong> during the meeting, and overcoming this urge to be helpful can be difficult and frustrating. However, allowing this problem-solving is a waste of most attendees' time and can drag the meeting out to over an hour if left unchecked.</li>
The content of the meeting isn't recorded</strong>. If I run into an issue that I think I recall Jane saying she was blocked on last week, I can either interrupt her work to ask her about it, send her an email and wait for her to reply, or just fight it myself. I can't look it up anywhere.</li>
If a team decides to keep notes, this "office housework</a> may be distributed inequitably among team members. Or if everyone takes turns taking notes, well... not everyone is necessarily skilled at note-taking, so there's little guarantee that the notes will be consistently useful.</li>
</ul>
When an international team decides to pursue standup meetings through a synchronous medium like a phone or video call, it keeps all of these drawbacks while adding the problem of time zones</strong>. Let's say your San Francisco-based company holds your daily standup at the perfectly sensible hour of 10am. Colleagues in New York City may love this, as it's 1pm their time so they have plenty of time to prepare for the meeting. But your "perfectly reasonable" 10am standup isn't so reasonable internationally: it's 5pm for a colleague in London, 6pm in Paris, 6am the next day in Auckland, and sleep-worthy hours like 4AM the next day in Sydney and 2am, also the next day, in Tokyo.</p>
Is demanding that some team members stay late at the office every day at the expense of family and personal commitments, or wake up before sunrise, the way that you want to treat your team? Is making a request like this, which disproportionately impacts your international colleagues, consistent with your values?</p>
The better way: Robots!</h1>
If you're familiar with my talks about community automation, you won't be surprised at my excitement to share another robot which makes life better.</p>
A team I'm on has recently started using a Slack app called Geekbot</a> to perform asynchronous, inherently logged, on-task standup meetings. The only thing special about Geekbot is that somebody else has already done the coding, testing, and debugging -- if your team uses IRC or another chat client, the basic "ask questions of each team member and post their answers, once per day" functionality is trivial to implement on any extensible platform.</p>
These distributed, asynchronous standups are the best standup meetings I've ever participated in. Why?</p>

They stay on task. You answer specific questions to the robot in PM, the robot posts your answers to a channel, and anyone who wants to chat about what you said has to do so outside the main thread of that conversation.</li>
They're asynchronous. If we added colleagues in any time zone, they could configure the bot to ping them at a time they find convenient, and the rest of the team could still keep up with their progress as long as everyone keeps reporting roughly every 24 hours.</li>
Others' updates minimize interruption. Rather than dropping what I'm doing to take a call for a meeting, I can use my ordinary chat-multitasking skills to read my colleagues' updates while waiting on another task to complete.</li>
They're self-recording. I can look back after lunch and see what I claimed I'd get done today; I can search my chat logs for "who was working on that component?". I strongly prefer to answer easy questions like this myself instead of interrupting others -- I save that for the difficult, interesting questions -- so I deeply appreciate this ability to solve my own problems with the meeting's inherently perfect notes.</li>
</ul>
Basically, these robot-powered, distributed standup check-ins are showing all of the benefits, and none of the major drawbacks, of in-person standup meetings.</p>
The catch: Culture</h1>
Why is it working? We've had similar standup type platforms before, and they work poorly if at all. I believe these standups are working better than prior attempts to automate the process for 2 main reasons:</p>

The standups are performed in a convenient location</strong>. Rather than having to remember to log into some service which exists only for the standup, it comes to you in the chat medium where you were doing the rest of your team communication.</li>
The team's culture</strong> values filling them out. If someone skips a standup, others will ask them where they were or what was going on. If you added a bot without adding a culture of appreciating standups, everyone would simply ignore or block the bot and nothing would change.</li>
</ul>
So, assess how your standups are working. Do they take your target amount of time and stay focused? Can you refer to their contents later as you need to? If they could use improvement, it's worth investigating how a robot could help you out.</p>


Saying Ping
2017-10-05T00:00:00+00:00
There's an idiom on IRC, and to a lesser extent other more modern communication media, where people indicate interest in performing a real-time conversation with someone by saying "ping" to them. This effectively translates to "I would like to converse with you as soon as you are available".</p>
The traditional response to "ping" is to reply with "pong". This means "I am presently available to converse with you".</p>
If the person who pinged is not available at the time that the ping's recipient replies, what happens? Well, as soon as they see the pong, they re-ping (either by saying "ping" or sometimes "re-ping" if they are impersonating a sufficiently complex system to hold some state).</p>
This attempt at communication, like "phone tag", can continue indefinitey in its default state.</p>
It is an inefficient use of both time and mental overhead, since each missed "ping" leaves the recipient with a vague curiosity or concern: "I wonder what the person who pinged wanted to talk to me about...". Additionally, even if both parties manage to arrange synchronous communication at some point in the future, there's the very real risk that the initiator may forget why they originally pinged at all.</p>
There is an extremely simple solution to the inefficiency of waiting until both parties are online, which is to stick a little metadata about your question onto the ping. "Ping, could you look issue # xyz?" "Ping, can we chat about your opinions on power efficiency sometime?". And yet there appears to be a decent correlation between people I regard as knowing more than I do about IRC etiquette, and people who issue pings without attaching any context to them.</p>
If you do this, and happen to read this, could you please explain why to me sometime?</p>


Resumes: 1 page or more?
2017-07-28T00:00:00+00:00
Some of my IRC friends are job hunting at the moment, so I've been proofreading resumes. These friends are several years into their professional careers at this point, and I've found it really interesting to see what they include and exclude to make the best use of their resumes' space.</p>
No wasted space</h1>
I've also stumbled into a rule of thumb that I like a lot better than the "1 page rule": The rule of No Wasted Space</strong>.</p>
If you spread 1 page worth of stuff across 2 pages, you're wasting a page worth of space. This forces any reader to spend 2 pages worth of time on your 1 page worth of content, which is an act of disrespect to them.</p>
Big blank areas are a waste of space if they push your resume to more pages than it needs to be. On the other side of that same coin, however, a bunch of cramped small text wastes its space if it sacrifices easy legibility and scannability for the sake of cramming everything relevant onto a single sheet. And excess words where fewer would have communicated just as well are a waste of ink, as well as interviewer time!</p>
Number those pages</h1>
Resumes that run to multiple pages are vastly improved by a little note on the corner of each: "Surname, Page X of Y".</p>
"I existed!"</h1>
I'm still surprised by how often people share awards and titles as "got X award", "was President of Organization". I hold the opinion that identical achievements sound much cooler with active rather than passive verbs: "Earned X award", "Led Organization", etc.</p>
Stuff in more numbers</h1>
I'm a little annoyed by whatever cognitive bias is in play with this one: People sound better at what they do when they stuff more numbers into their descriptions, even when the numbers aren't objectively very useful or necessary. "Tutored students" vs "Tutored 30 students", "Improved performance" vs "Doubled performance", etc. This is another compelling reason to instrument your systems and measure them before and after making major changes, which is something I personally need to improve at work as well :)</p>
The Mirror Trick</h1>
When you've been staring at a document for hours, it's really hard to take a step back and tell what kind of first impression its formatting is going to make. To counter this, make your resume in the format you expect an interviewer to see it. Printed out or fullscreened on a laptop are common choices. Then either make your computer flip it horizontally, or just hold it up to a mirror and look at the reflection. This makes it look just new enough that you can spot glaring formatting errors instead of just reading the individual words -- "What's that enormous white space doing there?", "Wait why is that indented to a different level from everything else?".</p>


Opinion: Levels of Safety Online
2017-06-27T00:00:00+00:00
The Mozilla All-Hands this week gave me the opportunity to explore an exhibit about the "Mozilla Worldview" that Mitchell Baker has been working on. The exhibit sparked some interesting and sometimes heated discussion (as direct result of my decision to express unpopular-sounding opinions), and helped me refine my opinions</strong> on what it means for someone to be "safe" on the internet.</p>
Spoiler: I think that there are many different levels of safety that someone can have online, and the most desirable ones are also the most difficult to attain.</p>
Obligatory disclaimer: These are my opinions. You're welcome to think I'm wrong. I'd be happy to accept pull requests to this post adding tools for attaining each level of safety, but if you're convinced I'm wrong, the best place to say that would be your own blog. Feel free to drop me a link if you do write up something like that, as I'd enjoy reading it!</p>
Safety to Consume Desired Information -----------------------------------</p>
I believe that the fundamental layer of safety that someone can have online is to be able to safely consume information</strong>. Even at this basic level, a lot of things can go wrong. To safely consume information, people need internet access. This might mean free public WiFi, or a cell phone data plan. Safety here means that the user won't come to harm solely as a result of what they choose to learn. "Desired information" means that the person gets a chance to find the "best" answer to their question that's available.</p>
How could someone come to harm as a result of choosing to learn something? If you've ever joked about a particular search getting you "put on a watch list", I'm sure you can guess. I happen to hold the opinion that knowledge is an amoral tool, and it's the actions that people take for which they should be held accountable -- if you believe that there exist facts that are inherently unethical to know, we'll necessarily differ on the importance of this safety.</p>
How might someone fail to get the information they desired? Imagine someone searching for the best open source social networking tools on a "free" internet connection that's provided and monitored by a social networking giant. Do you think the articles that turn up in their search results would be comparable to what they'd get on a connection provided by a less biased organization?</p>
Why "desired information", and not "truth"? My reason here is selfish. I enjoy learning about different viewpoints held by groups who each insist that the other is completely wrong. If somebody tried to moderate what information is "true" and what's "false", I would probably only be allowed to access the propaganda of at most one of those groups.</p>
Sadly, if your ISP is monitoring your internet connection or tampering with the content you're trying to view, there's not a whole lot that you can do about it. The usual solution is to relocate -- either physically, or feign relocation by using an onion router or proxy. By building better tools, legislation, and localization, it's plausible that we could extend this safety to almost everyone in the world within our lifetimes.</p>
Safety to Produce Information Anonymously ---------------------------------------</p>
I think the next layer of internet safety that people need is the ability to produce information anonymously</strong>. The caveat here is that, of course, nobody else is obligated to choose to host your content for you. The safety of hosting providers, especially coupled with their ability to take financial payment while maintaining user anonymity, is a whole other can of worms.</p>
Why does producing information anonymously come before producing information with attribution? Consider the types of danger that accompany producing content online. Attackers often choose their victims based on characteristics that the victims have in the physical world. Attempted attacks often cause harm because the attacker could identify the victim's physical location or social identity. While the best solution would of course be to prevent the attackers from behaving harmfully at all, a less ambitious but more attainable project is to simply prevent them from being able to find targets for their aggression. Imagine an attacker determined to harm all people in a certain group, on an internet where nobody discloses whether or not they're a member of that group: The attacker is forced to go for nobody or everybody, neither of which is as effective as an individually targeted attack. And that's just for verbal or digital assaults -- it is extremely difficult to threaten or enact physical harm upon someone whose location you do not know.</p>
Systems that support anonymity and arbitrary account creation open themselves to attempted abuse, but they also provide people with extremely powerful tools to avoid being abused. There are of course tradeoffs -- it takes a certain amount of mental overhead, and might feel duplicitous, to use separate accounts for discussing your unfashionable polticical views and planning the local block party -- but there's no denying how much less harm it is possible to come to when behaving anonymously than when advertising your physical identity and location.</p>
How do you produce information anonymously? First, you access the internet in a way that won't make it easy to trace your activity to your house. This could mean booting from a LiveCD and accessing a public internet connection, or booting from a LiveCD and using a proxy or onion router to connect to the sites you wish to access in order to mask your IP address. A LiveCD is safer than using your day-to-day computer profile because browsers store information from sites you visit, and some information about your operating system is sometimes visible to sites you visit. Using a brand-new copy of your operating system, which forgets everything when you shut down, is an easy way to avoid revealing those identifying pieces of information.</p>
Proof read anything that you want to post anonymously to make sure it doesn't contain any details about where you live, or facts that only someone with your experiences would know.</p>
How do you put information online anonymously? Once you have a connection that's hard to trace to your real-world self, it's pretty simple to set up free accounts on mail and web hosting sites under some placeholder name.</p>
Be aware that the vocabulary you use and the way you structure your sentences can sometimes be identifying, as well. A good way to strip all of the uniqueness from your writing voice is to run a piece of writing through http://hemingwayapp.com/</a> and fix everything that it calls an error. After that, use a thesaurus to add some words you don't usually use anywhere else. Alternately, you could run it through a couple different translation tools to make it sound less like you wrote it.</p>
How do you share something you wrote anonymously with your friends? Here's the hard part: You don't. If you're not careful, the way that you distribute a piece of information that you wrote anonymously can make it clear that it came from you. Anonymously posted information generally has to be shared publicly or to an entire forum, because to pick and choose exactly which individuals get to see a piece of content reveals a lot about the identity of the person posting it.</p>
Doing these things can enable you to produce a piece of information on the internet that would be a real nuisance to trace back to you in real life. It's not impossible</em>, of course -- there are sneaky tricks like comparing the times when you use a proxy to the times when material shows up online -- but someone would only attempt such tricks if they already had a high level of technical knowledge and a grudge against you in particular.</p>
Long story short, in most places with internet access, it is possible but inconvenient to exercise your safety to produce information anonymously. By building better online tools and hosting options, we can extend this safety to more people who have internet access.</p>
Safety to Produce Information Pseudonymously ------------------------------------------</p>
An important thing to note about producing information anonymously is that if you step up and take credit for another piece of information you posted, you're less anonymous. Add another attribution, and you're easier still to track. It's most anonymous to produce every piece of information under a different throwaway identity, and least anonymous to produce everything under a single identity even if it's made up.</p>
Producing information pseudonymously is when you use a fake name and biography, but otherwise go about the internet as the same person from day to day. The technical mechanics of producing a single pseudonymous post are identical to what I described for acting "anonymously", but I differentiate psyedonymity from anonymity in that the former is continuous -- you can form friendships with other humans under a pseudonym.</p>
The major hazard to a pseudonymous online presence is that if you aggregate enough details about your physical life under a single account, someone reading all those details might use them to figure out who you are offline. This is addressed by private forums and boards, which limit the number of possible attackers who can see your posts, as well as by being careful of what information you disclose. Beware, however, that any software vulnerability in a private forum may mean its contents suddenly becomes public.</p>
In my opinion, pseudonymous identity is an excellent compromise between the social benefits of always being the same person, and physical safety from hypothetical attackers. I find that behaving pseudonymously rather than anonymously helps me build friendships with people whom I'm not sure at first whether to trust, while maintaining a sense of accountability for my reputation that's absent in strictly anonymous communication. But hey, I'm biased -- you probably don't know my full name or home address from my web presence, so I'm on the pseudonymity spectrum too.</p>
Safety to Produce Information with Accurate Attribution -----------------------------------------------------</p>
The "safety" to produce information with attribution is extremely complex, and the one on which I believe that most social justice advocates tend to focus on. It is as it sounds: Will someone come to harm if they choose to post their opinions and location under their real name?</p>
For some people, this is the easiest safety to acquire: If you're in a group that's not subject to hate crimes in your area, and your content is only consumed by people who agree with you or feel neutrally toward your views, you have this freedom by default.</p>
For others, this safety is almost impossible to obtain. If the combination of your appearance and the views you're discussing would get you hurt if you said it in public, extreme social change would be required before you had even a chance at being comparably safe online.</p>
I hold the opinion that solving the general case of linking created content to real-world identities is not a computer problem. It's a social problem, requiring a world in which no person offended by something on the internet and aware of where its creator lives is physically able to take action against the content's creator. So it'd be great, but we are not there yet, and the only fictional worlds I've encountered in which this safety can be said to exist are impossibly unrealistic, totalitarian dystopias, or both.</p>
In Summary</h1>
In other words, I view misuse of the internet as a pattern of the form "Creator posts content -> attacker views content -> attacker identifies creator -> attacker harms creator". This chain can break, with varying degrees of difficulty, at several points:</p>
First, this chain of outcomes won't begin if the creator doesn't post the content at all. This is the easiest solution, and I point out the "safety to consume desired content" because even someone who never posts online can derive major benefits from the information available on the internet. It's easy, but it's not good enough: Producing as well as consuming content is part of what sets the internet apart from TV or books.</p>
The next essential link in the chain is the attacker identifying the content's creator. If someone has no way to contact you physically or digitally, all they can do is shout nasty things to the world online, and you're free to either ignore them or shout right back. Having nasty things shouted about your work isn't optimal, but it is difficult to feel that your physical or social wellbeing is jeopardized by someone when they have no idea who you are. This is why I believe that the safety to produce information anonymously is so important: It uses software to change the outcome even in circumstances where the attacker's behavior cannot be modified. Perfect pseudonymity also breaks this link, but any software mishap or accidental over-sharing can invalidate it instantly. The link is broken with fewer potential points of failure by creating content anonymously.</p>
The third solution is what I alluded to when discussing the safety of pseudonymity: Prevent the attacker from viewing the content. This is what private, interest-specific forums accomplish reasonably well. There are hazards here, especially if a forum's contents become public unintentionally, or if a dedicated attacker masquerades as a member of the very group they wish to harm. So it helps, and can be improved technologically through proper security practices by forum administrators, and socially via appropriate moderation. It's better, from the perspective that assuming the same online identity each day allows creators to build social bonds with one another, but it's still not optimal.</p>
The fourth and ideal solution is to break the cycle right at the very end, by preventing the attacker from harming the content creator. This seems to be where most advocates argue we should jump straight into, because it's really perfect -- it requires no change or compromise from content creators, and total change from those who might be out to harm them. It's the only solution in which people of all appearances and beliefs and locations are equally safe online. However, it's also the most difficult place to break the cycle, and a place at which any error of implementation would create the potential for incalculable abuse.</p>
I've listed these safeties in an order that I regard as how feasible they are to implement with today's social systems and technologies. I think it's possible to recognize the 4th safety as the top of the heap, without using that as an excuse to neglect the benefits which can come from bringing more of the world its 3 lesser but far more attainable cousins.</p>


Salt: Successful ping but highstate says "minion did not return"
2017-05-23T00:00:00+00:00
Today I was setting up some new OSX hosts on Macstadium for Servo's build cluster. The hosts are managed with SaltStack.</p>
After installing all the things</a>, I ran a test ping and it looked fine:</p>
user@saltmaster:~$ salt newbuilder test.ping
</span>newbuilder:
</span>    True
</span></code></pre>
However, running a highstate caused Salt to claim the minion was non-responsive:</p>
user@saltmaster:~$ salt newbuilder state.highstate
</span>newbuilder:
</span>    Minion did not return. [No response]
</span></code></pre>
Googling this problem yielded a bunch of other "minion did not return" kind of issues, but nothing about what to do when the minion sometimes returns fine and other times does not.</p>
The fix turned out to be simple: When a test ping succeeds but a longer-running state fails, it's an issue with the master's timeout setting. The timeout defaults to 5 seconds, so a sufficiently slow job will look to the master like the minion was unreachable.</p>
As explained in the Salt docs</a>, you can bump the timeout by adding the line timeout: 30</code> (or whatever number of seconds you choose) to the file /etc/salt/master</code> on the salt master host.</p>


Advice on storing encryption keys
2017-03-01T00:00:00+00:00
I saw an excellent question get some excellent infosec advice on IRC recently. I'm quoting the discussion here because I expect that I'll want to reference it when answering others' questions in the future.</p>
A user going by Dagnabit</code> asked:</p>

May I borrow some advice specifically on how best to store an ecryption key? I have a python script that encrypts files using libsodium, My question is how can I securely store the encryption key within the file system? Is it best kept in an sqlite db that can only be accessed by the user owning the python script?</p>
</blockquote>
This user has run right into one of the fundamental challenges of security: How can my secrets (in this case, keys) be safe from attackers, while still being usable?</p>
HedgeMage</a> replied with a wall of useful advice. Quotations are her words, links and annotations between them are me adding some context and opinions.</p>

So, it depends on your security model: in most cases I'm prone to keeping my encryption key on a hardware token, so that even if the server is compromised, the secret key is not.</p>
</blockquote>
You're probably familiar with time-based one-time-pad hardware tokens</a>, but in the case of key management, the "hardware token" could be as simple as a USB stick locked in a safe. On the spectrum of compromise between security and convenience, a hardware token is toward the DNSSEC keyholder</a> end.</p>

However, for some projects you are on virtualized infrastructure and can't plug in a hardware token. It's unfortunate, because that's really the safest thing, but a reality for many of us.</p>
</blockquote>
This also applies to physical infrastructure in which an application might need to use a key without human supervision.</p>

Without getting into anything crazy where a proxy server does signing, etc, you usually are better off trusting filesystem permissions than stuffing it in the database, for the following reasons:</p>
</blockquote>
While delegating the task of signing to a proxy server can make life more annoying to an attacker, you're still going to have to choose between having a human hold the key and get interrupted whenever it's needed, or trusting a server with it, at some point. You can compromise between those two extremes by using a setup like subkeys</a>, but it's still inconvenient if a subkey gets compromised.</p>

* It's easier to monitor the filesystem activity comprehensively, and detect intrusions/compromises.</p>
* Filesystem permissions are pretty dependable at this point, and if the application doing the signing has permission for the key, whether in a DB or the filesystem, it can compromise that key... so the database is giving you new attack surfaces (compromise of the DB access model) without any new protections.</p>
</blockquote>
To put it even more bluntly, any</em> unauthorized access to a machine has the potential to leak all</em> of the secrets on it. The actions that you'll need to take if you suspect the filesystem of a host was compromised are pretty much identical to those you'd take if the DB was.</p>

* Stuffing the key in the DB is nonstandard enough that you may be writing more of the implementation yourself, instead of depending as much as possible on widely-used, frequently-examined code.</p>
</blockquote>
Dagnabit's reply saved me the work of summarizing the key takeaways:</p>

I will work on securing the distrubtion and removing any unnecessary packages.</p>
I'll look at the possibility of using a hardware token to keep it secure/private.</p>
Reducing the attack surface is logical and something I had not considered.</p>
</blockquote>


Tech Internship Hunting Ideas
2017-01-23T00:00:00+00:00
A question from a computer science student crossed one of my IRC channels recently:</p>
Them: what is the best way to fish for internships over the summer?
</span>    Glassdoor?
</span>
</span>Me: It depends on what kind of internship you're looking for. What kind of
</span>    internship are you looking for?
</span>
</span>Them: Computer Science, anything really.
</span></code></pre>
This caused me to type out a lot of advice. I'll restate and elaborate on it here, so that I can provide a more timely and direct summary if the question comes up again.</p>
Philosophy of Job Hunting</h1>
My opinion on job hunting, especially for early-career technologists, is that it's important to get multiple offers whenever possible. Only once one has a viable alternative can one be said to truly choose a role, rather than being forced into it by financial necessity.</p>
In my personal experience, cultivating multiple offers was an important step in disentangling impostor syndrome from my career choices. Multiple data points about one's skills being valued by others can help balance out an internal monologue about how much one has yet to learn.</p>
If you disagree that cultivating simulataneous opportunities then politely declining all but the best is a viable internship hunting strategy, the rest of this post may not be relevant or interesting to you.</p>
Identifying Your Options</h1>
To get an internship offer, you need to make a compelling application</strong> to a company which might hire you</strong>. I find that a useful first step is to come up with a list of such companies, so you can study their needs and determine what will make your application interest them.</p>
Use your social network</strong>. Ask your peers about what internships they've had or applied for. Ask your mentors whether they or their friends and colleagues hire interns.</p>
When you ask someone about their experience with a company, remember to ask for their opinion of it. To put that opinion into perspective, it's useful to also ask about their personal preferences for what they enjoy or hate about a workplace. Knowing that someone who prefers to work with a lot of background noise enjoyed a company's busy open-plan office can be extremely useful if you need silence to concentrate! Listening with interest to a person's opinions also strengthens your social bond with them, which never hurts if it turns out they can help you get into a company that you feel might be a good fit.</p>
Use LinkedIn, Hacker News, Glassdoor, and your city's job boards</strong>. The broader a net you cast to start with, the better your chances of eventually finding somewhere that you enjoy. If your job hunt includes certain fields (web dev, DevOps, big data, whatever), investigate whether there's a meetup for professionals in that field in your region. If you have the opportunity to give a short talk on a personal project at such a meetup, do it and make sure to mention that you're looking for an internship.</p>
Identify your own priorities</h1>
Now that you have a list of places which might concievably want to hire you, it's time to do some introspection</strong>. For each field that you've found a prospective company in, try to answer the question "What makes you excited about working here?"</strong>.</p>
You do not have to have know what you want to do with your life to know that, right now, you think DevOps or big data or frontend development is cool.</p>
You do not have to personally commit to a single passion at the expense of all others -- it's perfectly fine to be interested in several different languages or frameworks, even if the tech media tries to pit them against each other.</p>
However, for each application, it's prudent to only emphasize your interests in that particular field. It's a bit of a faux pas to show up to a helpdesk interview and focus the whole time on your passion for building robots, or vice versa. And acting equally interested in every other field will cause an employer to doubt that you're necessarily the best fit for a specialized role... So in an interview, try not to stray too far from the value that you're able to deliver</strong> to that company.</p>
This is also a good time to identify any deal-breakers</strong> that would cause you to decline a prospective internship. Are you ok with relocating? Is there some tool or technology that would cause you to dread going to work every day?</p>
I personally think that it's worth applying even to a role that you know you wouldn't accept an offer from when you're early in your career. If they decide to interview you, you'll get practice experiencing a real interview without the pressure of "I'll lose my chance at my dream job if I mess this up!". Plus if they extend an offer to you, it can help you calibrate the financial value of your skills and negotiate with employers that you'd actually enjoy.</p>
Craft an excellent resume</h1>
I talk about this elsewhere</a>.</p>
There are a couple extra notes if you're applying for an internship:</p>
1) Emphasize the parts of your experience that relate most closely to what each employer values. If you can, it's great to use the same words for skills that were used in the job description.</p>
2) The bar for what skills go on your resume is lower when you have less experience. Did you play with Docker for a weekend recently and use it to deploy a toy app? Make sure to include that experience.</p>
Practice, Practice, Practice</h1>
If you're uncomfortable with interviewing, do it until it becomes second nature. If your current boss supports your internship search, do some mock interviews with them. If you're nervous about things going wrong, have a friend roleplay as a really bad interview with you to help you practice coping strategies. If you'll be in front of a panel of interviewers, try to get a panel of friends to gang up on you and ask tough questions!</p>
To some readers this may be obvious, but to others it's worth pointing out that you should also practice wearing the clothes</strong> that you'll wear to an interview. If you wear a tie, learn to tie it well. If you wear shirts or pants that need to be ironed, learn to iron them comptently. If you wear shoes that need to be shined, learn to shine them. And if your interview will include lunch, learn to eat with good table manners and avoid spilling food on yourself.</p>
Yes, the day-to-day dress codes of many tech offices are solidly in the "sneakers, jeans, and t-shirt" category for employees of all levels and genders. But many interviewers, especially mid- to late-career folks, grew up in an age when dressing casually at an interview was a sign of incompetence or disrespect. Although some may make an effort to overcome those biases, the subconscious conditioning is often still there, and you can take advantage of it by wearing at least business casual.</p>
Apply Everywhere</h1>
If you know someone at a company where you're applying, try to get their feedback on how you can tailor your resume to be the best fit for the job you're looking at! They might even be able to introduce you personally to your potential future boss.</p>
I think it's worth submitting a good resume to every company which you identify as being possibly interested in your skills, even the ones you don't currently think you want to work for. Interview practice is worth more in potential future salary than the hours of your time it'll take at this point in your career.</p>
Follow Up</h1>
If you don't hear back from a company for a couple weeks, a polite note is order. Restate your enthusiasm for their company or field, express your understanding that there are a lot of candidates and everything is busy, and politely solicit any feedback that they may be able to offer about your application. A delayed reply does not always mean rejection.</p>
If you're rejected, follow up to thank HR for their time.</p>
If you're invited to interview, reply promptly and set a time and date. For a virtual or remote interview, only offer times when you'll have access to a quiet room with a good network connection.</p>
Interview Excellently</h1>
I don't have any advice that you won't find a hundred times over on the rest of the web. The key points are:</p>

Show up on time, looking respectable</li>
Let's hope you didn't lie on your resume</li>
Restate each question in your answer</li>
It's ok not to know an answer -- state what you would do if you encountered the problem at work. Would you Google a certain phrase? Ask a colleague? Read the manual?</li>
Always ask questions at the end. When in doubt, ask your interviewer what they enjoy about working for the company.</li>
</ul>
Keep Following Up</h1>
After your interview, write to whoever arranged it and thank the interviewers for their time. For bonus points, mention something that you talked about in the interview, or include the answer to a question that you didn't know off the top of your head at the time.</p>
Getting an Offer</h1>
Recruiters don't usually like to disclose the details of offers in writing right away. They'll often phone you to talk about it. You do not have to accept or decline during that first call -- if you're trying to stall for a bit more time for another company to get back to you, an excuse like "I'll have to run that by my family to make sure those details will work" is often safe.</p>
Remember, though, that no offer is really a job until both you and the employer have signed a contract</strong>.</p>
Declining Offers</h1>
If you've applied to enough places with a sufficiently compelling resume, you'll probably have multiple offers. If you're lucky, they'll all arrive around the same time.</p>
If you wish to decline an offer from a company whom you're certain you don't want to work for, you can practice your negotiation skills. Read up on salary negotiation, try to talk the company into making you a better offer, and observe what works and what doesn't. It's not super polite to invest a bunch of their time in negotiations and then turn them down anyway, which is why I suggest only doing this to a place that you're not very fond of.</p>
To decline an offer without burning any bridges, be sure to thank them again for their time and regretfully inform them that you'll be pursuing other opportunities at this time. It never hurts to also do them a favor like recommending a friend who's job hunting and might be a good fit.</p>
Again, though, don't decline an offer until you have your actual job's contract in writing.</p>


Rust's Community Automation
2016-09-27T00:00:00+00:00
Here's the text version, with clickable links, of my Automacon lightning talk today.</p>
Intro</h1>
I'm a DevOps engineer at Mozilla Research and a member of the Rust Community subteam, but the conclusions and suggestions in this talk are my own observations and opinions.</p>
The slides are a result of trying to write my own CSS for sliderust</a>... Sorry about the ugliness.</p>
I have 10 minutes, so this is not the time to teach you any Rust. Check out rust-lang.org</a>, the Rust Community Resources</a>, or your city's Rust meetup to get started with the language.</p>
What we are</em> going to cover is how Rust automates some community tasks, and what you can learn from our automation.</p>
Community</h1>
I define "community", in this context, as "the human interaction work necessary for a project's success". This work is done by a wide variety of people in many situations. Every interaction, from helping a new contributor to discussing a proposed code change to criticizing someone's behavior, affects the overall climate of a project's community.</p>
Automation</h1>
To me, "automation" means "offloading peoples' work onto a system". This can be a computer system, but I think it can also mean changes to the social systems that guide peoples' behavior.</p>
So, community automation is a combination of:</p>

Building tools to do things the humans used to have to</li>
Tweaking the social systems to minimize the overhead they create</li>
</ul>
Scoping the Problem</h1>
While not all things can be automated and not all factors of the community are under the project leadership's control, it's not totally hopeless.</p>
Choices made and automation deployed by project leaders can help control:</p>

Which contributors feel welcome or unwelcome in a project</li>
What code makes it into the project's tree</li>
Robots!</li>
</ul>
Moderation</h1>
Our robots and social systems to improve workflow and contributor experience all rely on community members' cooperation. To create a community of people who want to work constructively together and not be jerks to each other, Rust defines behavior expectations code of conduct</a>. The important thing to note about the CoC is that half the document is a clear explanation of how the policies in it will be enforced. This would be impossible without the dedication of the amazing mod team</a>.</p>
The process of moderation cannot and should not be left to a computer, but we can use technology to make our mods' work as easy as possible. We leave the human tasks to humans, but let our technologies do the rest.</p>
In this case, while the mods need to step in when a human has a complaint about something, we can automate the process of telling peole that the rules exist. You can't join the IRC channel, post on the Discourse forums, or even read the Rust subreddit without being made aware that you're expected to follow the CoC's guidelines in official Rust spaces.</p>
Depending on the forums where your project communicates, try to automate the process of excluding obvious spammers and trolls. Not everybody has the skills or interest to be an excellent moderator, so when you find them, avoid wasting their time on things that a computer could do for them!</p>
It didn't fit in the talk, but this Slashdot post</a> is one of my favorite examples of somebody being filtered out of participating in the Rust community due to their personal convictions about how project leadership should work. While we do miss out on that person's potential technical contributions, we also save all of the time that might be spent hashing out our disagreements with them if we had a less clear set of community guideines.</p>
Robots</h2>
This lightning talk highlighted 4 categories of robots:</p>

Maintaining code quality</li>
Engaging in social pleasantries</li>
Guiding new contributors</li>
Widening the contributor pipeline</li>
</ul>
Longer versions of this talk also touch on automatically testing compiler releases</a>, but that's more than 10 minutes of content on its own.</p>
The Not Rocket Science Rule of Software Engineering</h1>
To my knowledge, this blog post</a> by Rust's inventor Graydon Hoare is the first time that this basic principle has been put so succinctly:</p>

Automatically maintain a repository of code that always passes all the tests.</p>
</blockquote>
This policy guides the Rust compiler's development workflow, and has trickled down into libraries and projects using the language.</p>
Bors</h1>
The name Bors has been handed down from Graydon's original autolander bot</a> to an instance of Homu</a>, and is often verbed to refer to the simple actions he does:</p>

Notice when a human says "r+" on a PR</li>
Create a branch that looks like master will after the change is applied</li>
Test that branch</li>
Fastforward the master branch to the tested state, if it passed.</li>
</ol>
Keep your tree green</h1>
Saying "we can't break the tests any more" is a pretty significant cultural change, so be prepeared for some resistance. With that disclaimer, the path to following the Not Rocket Science Rule is pretty simple:</p>

Write tests that fail when your code is bad and pass when it's good</li>
Run the tests on every change</li>
Only merge code if it passes all the tests</li>
Fix the tests whenever thy're wrong.</li>
</ol>
This strategy encourages people to maintain the tests, because a broken test becomes everyone's problem and interrupts their workflow until it's fixed.</p>
I believe that tests are necessary for all code that people work on. If the code was fully and perfectly correct, it wouldn't need changes -- we only write code when something is wrong, whether that's "It crashes" or "It lacks such-and-such a feature". And regardless of the changes you're making, tests are essential for catching any regressions you might accidentally introduce.</p>
Automating social pleasantries</h1>
Have you ever submitted an issue or change request to a project, then not heard back for several months? It feels bad to be ignored, and the project loses out on potential contributors.</p>
Rust automates basic social pleasantries with a robot called Highfive</a>. Her tasks are easy to explain, though the implementaion details can be tricky:</p>

Notice when a change is submitted by a new contributor, then welcome them</li>
Assign reviewers, based on what code changed, to all PRs</li>
Nag the reviewer if they seem to have forgotten about their assignment</li>
</ol>
If you don't want a dedicated greeter-bot, you can get many of these features from your code management system:</p>

Use issue and pull request templates to guide potential contributors to the docs that can help them improve their report or request.</li>
Configure notifications so you find out when someone is trying to interact with your project. This could mean muting all the noise notifications so the signal ones are available, or intermittently polling the repositories that you maintain (a daily cron job or weekly calendar reminder works just fine).</li>
</ul>
Guide new contributors</h1>
In open source projects, "I'm new; what can I work on?" is a common inquiry. In internal projects, you'll often meet colleagues from elsewhere in your organization who ask you to teach them something about the project or the skills you use when working on it.</p>
The Rust-implemented browser engine Servo</a> is actually a slightly better example of this than the compiler itself, since the smaller and younger codebase has more introductory-level issues remaining. The site starters.servo.org</a> automatically scrapes the organization's issue trackers for easy and unclaimed issues.</p>
Issue triage is often unrewarding, but using the tags for a project like this creates a greater incentive to keep them up to date.</p>
When filing introductory issues, try to include links to the relevant documentation, instructions for reproducing the bug, and a suggestion of what file you would look in first if you tackled the problem yourself.</p>
Automating mentorship</h1>
Mentorship is a highly personalized process in which one human transfers their skills to another. However, large projects often have more contributors seeking the same basic skills than mentors with time to teach them.</p>
The parts of mentorship which don't explicitly require a human mentor can be offloaded onto technology.</p>
The first way to automate mentorship tasks is to maintain correct and up-to-date documentation. Correct docs train humans to consult them before interrupting an expert, whereas docs that are frequently outdated or wrong condition their users to skip them entirely.</p>
Use tools like octohatrack</a> and your project status updates to identify and recognize contributors who help with docs and issue triage. Docs contributions may actually save more developer and community time than new code features, so respect them accordingly.</p>
Finally, maintain a list of introductory or mentored issues -- even if that's just a Google Doc or Etherpad.</p>
Bear in mind that an introductory issue doesn't necessarily mean "suitable for someone who has never coded before". Someone with great skills in a scripting language might be looking for a place to help with an embedded codebase, or a UX designer might want to get involved with a web framework that they've used. Introductory issues should be clear about what knowledge a contributor should acquire in order to try them, but they don't have to all be "easy".</p>
Automating the pipeline</h1>
Drive-by fixes are to being a core contributor as interviews are to full time jobs. Just as a company attempts to interview as many qualified candidates as it can, you can recruit more contributors by making your introductory issues widely available.</p>
Before publicizing your project, make sure you have a CONTRIBUTING.txt</code> or good README</code> outlining where a new contributor should start, or you'll be barraged with the same few questions over and over.</p>
There are a variety of sites, which I call issue aggregators, where people who already know a bit about open source development can go to find a new project to work on. I keep a list on this page <http://edunham.net/pages/issue_aggregators.html></span>, pull requests welcome <https://github.com/edunham/site/blob/master/pages/issue_aggregators.rst></span> if I'm missing anything. Submitting your introductory issues to these sites broadens your pipeline, and may free up humans' recruiting time to focus on peole who need more help getting up to speed.</p>
If you're working on internal rather than public projects, issue aggregators are less relevant. However, if you have the resources, it's worthwhile to consider the recruiting device of open sourcing an internal tool that would be useful to others. If an engineer uses and improves that tool, you get a tool improvement and they get some mentorship. In the long term, you also get a unique opportunity to improve that engineer's opinion of your organization while networking with your engineers, which can make them more likely to want to work for you later.</p>
Follow Up</h1>
For questions, you're welcome to chat with me on Twitter (@QEDunham), email (automacon <at> edunham <dot> net), or IRC (edunham on irc.freenode.net and irc.mozilla.org).</p>
Slides from the talk are here</a>.</p>


Setting a Freenode channel's taxonomy info
2016-09-23T00:00:00+00:00
Some recent flooding in a Freenode channel sent me on a quest to discover whether the network's services were capable of setting a custom message rate limit for each channel. As far as I can tell, they are not.</p>
However, the problem caused me to re-read the ChanServ help section:</p>
/msg chanserv help
</span>- ***** ChanServ Help *****
</span>- ...
</span>- Other commands: ACCESS, AKICK, CLEAR, COUNT, DEOP, DEVOICE,
</span>-                 DROP, GETKEY, HELP, INFO, QUIET, STATUS,
</span>-                 SYNC, TAXONOMY, TEMPLATE, TOPIC, TOPICAPPEND,
</span>-                 TOPICPREPEND, TOPICSWAP, UNQUIET, VOICE,
</span>-                 WHY
</span>- ***** End of Help *****
</span></code></pre>
Taxonomy is a cool word. Let's see what taxonomy</code> means in the context of IRC:</p>
/msg chanserv help taxonomy
</span>- ***** ChanServ Help *****
</span>- Help for TAXONOMY:
</span>-
</span>- The taxonomy command lists metadata information associated
</span>- with registered channels.
</span>-
</span>- Examples:
</span>-     /msg ChanServ TAXONOMY #atheme
</span>- ***** End of Help *****
</span></code></pre>
Follow its example:</p>
/msg chanserv taxonomy #atheme
</span>- Taxonomy for #atheme:
</span>- url                       : http://atheme.github.io/
</span>- ОХЯЕБУ                    : лололол
</span>- End of #atheme taxonomy.
</span></code></pre>
That's neat; we can elicit a URL and some field with a cryllic and apparently custom name. But how do we put metadata into a Freenode channel's taxonomy section? Google has no useful hits (hence this blog post), but further digging into ChanServ's manual does help:</p>
/msg chanserv help set
</span>
</span>- ***** ChanServ Help *****
</span>- Help for SET:
</span>-
</span>- SET allows you to set various control flags
</span>- for channels that change the way certain
</span>- operations are performed on them.
</span>-
</span>- The following subcommands are available:
</span>- EMAIL           Sets the channel e-mail address.
</span>- ...
</span>- PROPERTY        Manipulates channel metadata.
</span>- ...
</span>- URL             Sets the channel URL.
</span>- ...
</span>- For more specific help use /msg ChanServ HELP SET command.
</span>- ***** End of Help *****
</span></code></pre>
Set arbirary metadata with /msg chanserv set #channel property key value</code></h1>
The commands /msg chanserv set #channel email a@b.com</code> and /msg chanserv set #channel property email a@b.com</code> appear to function identically, with the former being a convenient wrapper around the latter.</p>
So that's how #atheme</code> got their fancy cryllic taxonomy: Someone with the appropriate permissions issued the command /msg chanserv set #atheme property ОХЯЕБУ лололол</code>.</p>
Behaviors of channel properties</h1>
I've attempted to deduce the rules governing custom metadata items, because I couldn't find them documented anywhere.</p>

Issuing a set property</code> command with a property name but no value deletes the property, removing it from the taxonomy.</li>
A property is overwritten each time someone with the appropriate permissions issues a /set</code> command with a matching property name (more on the matching in a moment). The property name and value are stored with the same capitalization as the command issued.</li>
The algorithm which decides whether to overwrite an existing property or create a new one is not case sensitive</strong>. So if you set ##test email test@example.com</code> and then set ##test EMAIL foo</code>, the final taxonomy will show no field called email</code> and one field called EMAIL</code> with the value foo</code>.</li>
When displayed, taxonomy items are sorted first in alphabetical order (case insensitively), then by length. For instance, properties with the names a</code>, AA</code>, and aAa</code> would appear in that order, because the initial alphebetization is case-insensitive.</li>
Attempting to place mIRC color codes</a> in the property name results in the error "Parameters are too long. Aborting." However, placing color codes in the value of a custom property works just fine.</li>
</ol>
Other uses</h1>
As a final note, you can also do basically the same thing with Freenode's NickServ, to set custom information about your nickname instead of about a channel.</p>


Adventures in Mercurial
2016-08-02T00:00:00+00:00
I adore Git, but have needed to ramp up my Mercurial (Hg) skills recently to dig prior work related to my current tasks out of a repo's history. Here are some things I'm learning:</p>
Command Equivalences</h1>
As this tutorial</a> so helpfully explains, the two VCSes aren't all that dissimilar under their hoods. I condensed the command comparison table into a single page and printed it out for quick reference; a PDF is here</a>.</p>
Clone</h1>
The thing I want to clone lives at http://hg.mozilla.org/hgcustom/version-control-tools/file/tip/autoland</code>.</p>
Trying to clone the full URL yields a 404, but snipping the URL back to the top-level directory gets me the repo:</p>
$ hg clone http://hg.mozilla.org/hgcustom/version-control-tools/
</span>destination directory: version-control-tools
</span>requesting all changes
</span>adding changesets
</span>adding manifests
</span>adding file changes
</span>added 4574 changesets with 10874 changes to 1971 files
</span>updating to bookmark @
</span>1428 files updated, 0 files merged, 0 files removed, 0 files unresolved
</span>$ ls
</span>version-control-tools
</span></code></pre>
Examine Log</h1>
hg log | less</code> shows me that each commit's summary in this repo includes the part of the codebase it touches, and a bug number.</p>
hg log | grep autoland: | less</code> gives me the summaries of every commit that touched autoland, but I cannot show a commit from summary alone.</p>
The Hg book</a> helped me construct a filter that will show a unique revision ID onthe same line as each description.</p>
hg log --template '{rev} {desc}\n' | grep autoland:</code> is much more useful. It gives me the local ID of each changeset whose description included "autoland:".</p>
From here, I can use a bit more grep to narrow down the list of matching messages, then I'm ready to examine commits.</p>
Examining Commits</h1>
That {rev}</code> in my filter was the "repository-local changeset revision number". For these examples I'll examine revision 2589.</p>
hg status --change 2589</code> lists the files that were touched by that revision, and hg export 2589</code> yields a full diff of the changes introduced.</p>
This gets me enough information to make an appropriate set of changes, run the tests, and create my own commits!</p>


Thinkpad 13 Trackpoint slowdown in i3 window manager
2016-07-01T00:00:00+00:00
As has been mentioned on Reddit</a>, the Thinkpad 13 trackpoint settings aren't in the same place as those of older thinkpads. Despite some troubleshooting, I haven't yet found what files to edit to adjust the trackpoint's speed and sensitivity in Ubuntu 16.04.</p>
The trackpoint has been slightly sluggish/unresponsive when I use the i3 window manager, and has additional intermittent slowdowns when using Chromium and Firefox in i3.</p>
Although I don't yet know the right</strong> way to fix trackpoint sensitivity on this machine, I accidentally discovered a highly effective workaround today:</p>

Log into Unity (the default desktop that Ubuntu ships with) and configure the mouse and input settings as desired</li>
Log out, and get back into i3wm</li>
Launch unity-settings-daemon</code></li>
And suddenly, the mouse works correctly the way it did in Unity!</li>
</ul>
I fully realize that this is a nasty hack around identifying and solving the actual problem, but it succeeds at making the mouse responsive while minimizing time spent troubleshooting.</p>


Hieroglyph and Tinkerer Dependencies
2016-06-27T00:00:00+00:00
In setting up virtualenvs for my slides and blog repos on my new laptop, I've been reminded that a variety of Sphinx-based tools require system dependencies as well as the ones in their virtualenvs.</p>
Hieroglyph dependency issues</h1>
The error resulting from pip install -r requirements.txt</code> ended with:</p>
Command ".../virtualenv/bin/python2 -u -c
</span>"import setuptools,
</span>tokenize;__file__='/tmp/pip-build-lzbk_r/Pillow/setup.py';exec(compile(getattr(tokenize,
</span>'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))"
</span>install --record /tmp/pip-BNDc_6-record/install-record.txt
</span>--single-version-externally-managed --compile --install-headers
</span>/home/edunham/repos/slides/rustcommunity/v/include/site/python2.7/Pillow"
</span>failed with error code 1 in /tmp/pip-build-lzbk_r/Pillow/
</span></code></pre>
Its fix, from stackoverflow</a>, was:</p>
$ sudo apt-get install libtiff5-dev libjpeg8-dev zlib1g-dev libfreetype6-dev liblcms2-dev libwebp-dev tcl8.6-dev tk8.6-dev python-tk
</span>$ pip install -r requirements.txt
</span></code></pre>
Tinkerer depencencies, too!</h1>
pip install -r requirements.txt</code> over in my blog repo</a> yielded:</p>
Command ".../virtualenv/bin/python2 -u -c "import setuptools,
</span>tokenize;__file__='/tmp/pip-build-NVLSBY/lxml/setup.py';exec(compile(getattr(tokenize,
</span>'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))"
</span>install --record /tmp/pip-qD5QIe-record/install-record.txt
</span>--single-version-externally-managed --compile --install-headers
</span>/home/edunham/repos/site/v/include/site/python2.7/lxml" failed with error code
</span>1 in /tmp/pip-build-NVLSBY/lxml/
</span></code></pre>
The fix is again to install the missing system deps, on Ubuntu:</p>
$ sudo apt-get install libxml2-dev libxslt-dev
</span>$ pip install -r requirements.txt
</span></code></pre>
That's it! I'm writing this down for SEO on the specific errors at hand, since the first several useful hits are currently stackoverflow.</p>
If you're a pip developer reading this, please briefly contemplate whether it'd be worthwhile to have some built-in mechanism to translate common dependency errors to the appropriate system package names needed based on the OS on which the command is run.</p>


CFPs Made Easier
2016-06-23T00:00:00+00:00
Check out this post</a> by Lucy Bain about how to come up with an idea for what to talk about at a conference. I blogged last year about how I turn abstracts into talks</a>, as well. Now that the SeaGL CFP</a> is open, it's time to look in a bit more detail about the process of going from a talk idea</strong> to a compelling abstract</strong>.</p>
In this post, I'll walk you through some exercises to clarify your understanding of your talk idea and find its audience, then help you use that information to outline the 7 essential parts of a complete abstract.</p>
Getting ready to write your abstract</h1>
Your abstract is a promise about what your talk will deliver. Have you ever gotten your hopes up for a talk based on its abstract, then attended only to hear something totally unrelated? You can save your audience from that disappointment by making sure that you present what your abstract says you will.</p>
I find the abstract to be the hardest part of the talk to write, because it sets the stage for every other part of it. If your abstract is thorough and clear about what your talk will deliver, you can refer back to it throughout the writing process to make sure you're including the information that your audience is there for!</p>
For both you and your audience to get the most out of your talk, the following questions can help you refine your talk idea before you even start to write its abstract.</p>
Why do you love this idea?</h2>
Start working on your abstract by taking some quick notes on why you're excited about speaking on this topic. There are no wrong answers! Your reasons might include:</p>


Document a topic you care about in a format that works well for those who learn by listening and watching</li>
Impress a potential employer with your knowledge and skills</li>
Meet others in the community who've solved similar problems before, to advise you</li>
Recruit contributors to a project</li>
Force yourself to finish a project or learn more detail about a tool</li>
Save novices from a pitfall that you encountered</li>
Travel to a conference location that you've always wanted to visit</li>
Build your resume</li>
Or something else entirely!</li>
</ul>
</blockquote>
Starting out by identifying what you personally hope to gain from giving the talk will help ensure that you make the right promises in your abstract, and get the right people into the room.</p>
What's your idea's scope?</h2>
Make 2 quick little lists:</p>

Topics I really want this presentation to cover</li>
Topics I do not want this presentation to cover</li>
</ul>
Once you think that you have your abstract all sorted out, come back to these lists and make sure that you included enough topics from the first list, and excluded those from the second.</p>
Who's the conference's target audience?</h2>
Keynotes and single-track conferences are special, but generally your talk does not have to appeal to every single person at the conference.</p>
Write down all the major facts you know about the people who attend the conference to which you're applying. How young or old might they be? How technically expert or inexperienced? What are their interests? Why are they there?</p>
For example, here are some statements that I can make about the audience at SeaGL:</p>


Expertise varies from university students and random community members to long-time contributors who've run multiple FOSS projects.</li>
Age varies from a few school-aged kids (usually brought by speakers and attendees) to retirees.</li>
The audience will contain some long-term FOSS contributors who don't program, and some relatively expert programmers who might have minimal involvement in their FOSS communities</li>
Most attendees will be from the vicinity of Seattle. It will be some attendees' first tech conference. A handful of speakers are from other parts of the US and Canada; international attendees are a tiny minority.</li>
The audience comes from a mix of socioeconomic backgrounds, and many attendees have day jobs in fields other than tech.</li>
Attendees typically come to SeaGL because they're interested in FOSS community and software.</li>
</ul>
</blockquote>
Where's your niche?</h2>
Now that you've taken some guesses about who will be reading your abstract, think about which subset of the conference's attendees would get the most benefit out of the topic that you're planning to talk about.</p>
Write down which parts of the audience will get the most from your talk --novices to open source? Community leaders who've found themselves in charge of an IRC channel but aren't sure how to administer it? Intermediate Bash users looking to learn some new tricks?</p>
If your talk will appeal to multiple segments of the community (developers interested in moving into DevOps, and managers wondering what their operations people do all day?), write one question that your talk will answer for each segment.</p>
You'll use this list to customize your abstract and help get the right people into the room for your talk.</p>
Still need an idea?</h2>
Conferences with a diverse audience often offer an introductory track to help enthusiastic newcomers get up to speed. If you have intermediate skills in a technology like Bash, Git, LaTeX, or IRC, offer an introductory talk to help newbies get started with it! Can you teach a topic that you learned recently in a way that's useful to newbies?</p>
If you're an expert in a field that's foreign to most attendees (psychology? beekeeping? Cray Supercomputer assembly language?), consider an intersection talk: "What you can learn from X about Y". Can you combine your hobby, background, or day job with a theme from the conference to come up with something unique?</p>
The Anatomy of an Abstract</h1>
There are many ways to structure a good abstract. Here are the 7 elements that I try to always include:</p>


Set the scene with a strong introductory sentence</strong>, which reminds your target audience of your topic's relevance to them. Some of mine have included:</p>


"Rust is a systems programming language that runs blazingly fast, prevents segfaults, and guarantees thread safety."</li>
"Git is the most popular source code management and version control system in the open source community."</li>
"When you're new to programming, or self-taught with an emphasis on those topics that are directly relevant to your current project, it's easy to skip learning about analyzing the complexity of algorithms."</li>
</ul>
</blockquote>
</li>

Ask some questions</strong>, which the talk promises to answer. These questions should be asked from the perspective of your target audience, which you identified earlier. This is the least essential piece of an abstract, and can be skipped if you make sure your exposition clearly shows that you understand your target audience in some other way. Here are a couple of questions I've used in abstracts that were accepted to conferences:</p>


"Do you know how to control what information people can discover about you on an IRC network?"</li>
"Is the project of your dreams ignoring your pull requests?"</li>
</ul>
</blockquote>
</li>

Drop some hints about the format</strong> that the talk will take. This shows the selection commitee that you've planned ahead, and helps audience members select session that're a good fit for their learning styles. Useful words here include:</p>


"Overview of"</li>
"Case study"</li>
"Demonstration"</li>
"Deep dive into"</li>
"Outline X principles for"</li>
"Live coding"</li>
</ul>
</blockquote>
</li>

Identify what background knowledge</strong> the audience will need to get the talk's benefit, if applicable. Being specific about this helps welcome audience members who're undecided about whether the talk is applicable to them. Useful phrases include:</p>


"This talk will assume no background knowledge of..."</li>
"If you've used ____</code> to ____</code>, ..."</li>
"If you've completed the ____</code> tutorial..."</li>
</ul>
</blockquote>
</li>

State a specific benefit</strong> that audience members will get from having attended the talk. Benefits can include:</p>


"Halve your Django website's page load times"</li>
"Get help on IRC"</li>
"Learn from ____</code>'s mistakes"</li>
"Ask the right questions about ____</code>"</li>
</ul>
</blockquote>
</li>

Reinforce and quantify your credibility</strong>. If you're presenting a case study into how your company deployed a specific tool, be sure to mention your role on the team! For instance, you might say:</p>


"Presented by [the original author | a developer | a maintainer | a long-term user]</code> of [the project]</code>, this talk will..."</li>
</ul>
</blockquote>
</li>

End with a recap</strong> of the talk's basic promise, and welcome audience members to attend.</p>
</li>
</ol>
These 7 pieces of information don't have to each be in their own sentence --for instance, establishing your credibility and indicating the talk's format often fit together nicely in a single sentence.</p>
Once you've got all of the essential pieces of an abstract, munge them around until it sounds like concise, fluent English. Get some feedback on helpmeabstract.com</a> if you'd like assistance!</p>
Give it a title</h1>
Naming things is hard. Here are some assorted tips:</p>

Keep it under about 50 characters, or it might not fit on the program</li>
Be polite. Rude puns or metaphors might be eye-catching, but probably violate your conference or community's code of conduct, and will definitely alienate part of your prospective audience.</li>
For general talks, it's hard to go wrong with "Intro to ___</code>" or "___</code> for ___</code> users".</li>
The form "[topic]: A [historymelodramalove story]" is generally reliable. Well, I'm kidding about "melodrama" and "love story"... Mostly.</li>
Clickbait is underhanded, but it works. "___</code> things I wish I'd known about ___</code>", anyone?</li>
</ul>
Good luck, and happy conferencing!</p>


2ish weeks with the Thinkpad 13
2016-05-27T00:00:00+00:00
I recently got a Thinkpad 13 to try replacing my X230 as a personal laptop. Here's the relevant specs from my order confirmation:</p>
</th> </th></tr></thead>

Battery</td> 3cell 42Wh</td></tr>
System Unit</td> 13&S2 IG i5-6200U NvPro</td></tr>
Camera</td> 720p HD Camera</td></tr>
AC Adapter and Power Cord</td> 45W 2pin US</td></tr>
Processor</td> Intel Core i5-6200U MB</td></tr>
Hard drive</td> 128GB SSD SATA3</td></tr>
Keyboard Language</td> KYB SR ENG</td></tr>
Publication Language</td> PUB ENG</td></tr>
Total memory</td> 4GB DDR4 2133 SoDIMM</td></tr>
OS DPK</td> W10 Home</td></tr>
Pointing device</td> 3+2BCP NoFPR SR</td></tr>
Preload Language</td> W10 H64-ENG</td></tr>
Preload OS</td> Windows 10 Home 64</td></tr>
Preload Type</td> Standard Image</td></tr>
TPM Setting</td> Software TPM Enabled</td></tr>
Display Panel</td> 13&S2 FHD IPS AG AL SR</td></tr>
WiFi wireless LAN adapters</td> Intel 8260AC+BT 2x2 vPro</td></tr>
</tbody></table>
I picked mediocre CPU and RAM because the RAM's easy to upgrade, and I'm curious about whether I actually need top-of-the-line CPUs to have an acceptable experience on a personal laptop.</p>
Why the 13?</h1>
I had a few hard requirements for my next personal laptop:</p>

Trackpoint with buttons</li>
Decent key travel (the X1 carbon has 1.86mm</a> and typing on it for too long made my hands hurt)</li>
USBC port</li>
Under $1,000</li>
</ul>
Plus a few nice-to-haves:</p>

Small and light are nice, including charger</li>
Screen not much worse than 1920x1080</li>
Good battery life</li>
Metal case and design for durability make me happy</li>
My house is already full of Thinkpad chargers, so a laptop that uses them helps reduce additional clutter</li>
</ul>
I'll be the first to admit that this is an atypical set of priorities. My laptop is home to Git, Vim, and a variety of tools for interacting with the internet, so the superficial I/O differences matter more to me than the machine's internal specs.</p>
Things I like about the 13</h1>

2.1mm</a> key travel is everything I hoped for. At least, I've used it all day and my hands don't hurt.</li>
Battery life is pretty decent, and battery will be easy to replace when it starts to fail.</li>
Light-enough weight. Lighter charger than other Thinkpads I've had.</li>
Smallest Thinkpad charger that I've ever seen.</li>
Case screws are all captive.</li>
Mystery hole in the bottom case turns out to be a highly convenient hard shutdown button.</li>
Hinges feel pretty solid and hold the screen up nicely.</li>
No keyboard backlight. I dislike them.</li>
4GB of RAM is a single stick, easy to add more (and I'll need to for a smoother web browsing experience; neither Firefox nor Chromium is particularly happy with only 4GB)</li>
A vanilla Ubuntu 16.04 iso Just Worked for installing Linux. It must have shipped with whatever magic signatures were required to play nice with the new security measures, because the install process was delightfully non-thought-provoking.</li>
~7mm plastic bezel between buttons and trackpad reduces likelihood of accidentally moving cursor when clicking.</li>
</ul>
</p>

Screen's the same as my X240, xrandr calls it 1920x1080 294mm x 165mm. This fits 211 columns of a default font, or 127 columns of a font that's comfortably legible when the laptop is on the other side of my desk.</li>
</ul>
Nitpicks about the 13</h1>

Color.</li>
</ul>
When I purchased mine at the end of April, only the silver chassis had a metal lid and shipped with a nice screen by default (the higher-res screen is available in the all-plastic black model for an additional charge). So now I own a non-black laptop for the first time since my Dell Latitude D410 in high school. The screen bezel and keys are black, though, and if I really cared I could probably paint the rest of it.</p>

Power button.</li>
</ul>
</p>
It feels horribly... squishy? There's no satisfying click</code> to tell me when I've pushed it far enough. Holding it for 10 seconds only sometimes shuts the laptop off (though there's a reset switch on the mobo accessible by a paperclip-hole in the bottom panel which forces shutdown instantly when pushed). There's a circle on the pwoer button that looks like it might be an LED, but it never lights up.</p>

Cutesy font.</li>
</ul>
</p>
This is a tiny nitpick, but they've changed the Lenovo logo on the lid, pre-BIOS boot screen, and screen bezel from the already-mediocre font to a super condescending, childish, roundy one. Fortunately the lid one is easily hidden under some stickers.</p>

Bottom panel held on by clips as well as screws.</li>
</ul>
More on this one in the disassembly section below, but I'm afraid they'll break with how often I take my laptops apart.</p>

Mouse buttons feel cheap and plastic-ey.</li>
</ul>
They feel like thin plastic shells instead of solid buttons like on older Thinkpads. I'm not sure precisely why they feel that way, but it's a reminder that you're using a lower-end machine.</p>

Longest side is about 1cm greater than the short side of a TSA xray tub.</li>
</ul>
My X240 fits perfectly along the short end of the tub, leaving room for my shoes beside it. I have to use two tubs or separate my pair of shoes when putting the 13 through the scanner. (see, I wasn't kidding when I said "nitpicks")</p>

The Trackpoint top is not interchangeable with those of older Thinkpads.</li>
</ul>
</p>
The round part is the same size, but the square hole in the bottom is about 2mm to a side rather than the 4mm of the one on an x220 keyboard. Plus the cylinder bit is about 2mm long rather than the x220's 3.5mm, so even with an adapter for the square hole, older trackpoints would risk leaving marks on the screen.</p>

The fan is a little loud.</li>
</ul>
I anticipate that this will get a lot less annoying when I upgrade to 16 or 32GB of ram and maybe tune it in software using thinkfan</code>.</p>
Thinkpad 13 partial disassembly photos</h1>
To get the bottom case off, pull all the visible screws and also remove the 3 tiny rubber feet from under the palm rest. I stuck my tiny rubber feet in a plastic bag and filed it away, because repeated removal would eventually destroy the glue and get them lost.</p>
</p>
The bottom case comes off with a combination of sliding and prying. Getting it back on again requires sliding the palmrest edge just right, then snapping the sides and back on before the palm rest slips out of place. It's tricky.</p>
</p>
The battery is easily removed by pulling out a single (non-captive) screw. It seems to be a thin plastic wrapper around 3 cell phone batteries. The battery has no glue holding it in, just screws.</p>
</p>
Here's its guts, with battery removed.</p>
</p>
Note the convenient hard power cycle button (accessible via a tiny hole in the bottom case when assembled), pair of RAM slots and SSD form factor, and airspace compartment that almost looks intended for hiding half a dozen very small items. The coin cell battery (in sky blue shrink wrap) flaps around awkwardly when the machine is disassembled, but at least it's not glued down.</p>


Reflections on my first live webcast
2016-05-10T00:00:00+00:00
This morning, I participated in the O'Reilly Emerging Languages Webcast</a> with my "Rust from a Scripting Background</a>" talk. Here's how it went.</p>
Preparation</h1>
I was contacted about a month before my webcast and asked to present my OSCON talk as part of the event. I explained why my "How to learn Rust" talk didn't sound like a good fit for the emerging languages webcast, and suggested the "Starting Rust from a Scripting Background" talk that I gave at my local Rust meetup</a> recently as an alternative.</p>
After we agreed on what talk would be suitable, O'Reilly's Online Events Producers emailed me a contract to e-sign. The contract gives O'Reilly the opportunity to reuse and redistribute my talk from the webcast, and promises me a percentage of the proceeds if my recording is sold, licensed, or otherwise used to make them money.</p>
During the week before the webcast, I did a test call in which an O'Reilly representative walked me through how to use the webcast software and verified that my audio was good on the phone I planned to use for the webcast.</p>
A final copy of the slides for the webcast, in my option of PDF or Powerpoint, was due at 5pm the day before the event.</p>
The Software (worked for me on Ubuntu)</h1>
O'Reilly Media provided an application called "Presentation Manager XD" from on24.com that presenters log into during the event.</p>
According to my email from O'Reilly, the requirements for the event are:</p>

Slides - PowerPoint or PDF only please with no embedded video or audio. Screen ratio of 4:3</li>
Robust Internet connection</li>
Clear, reliable phone line.</li>
Windows 7 or 8 running IE 8+, Firefox 22+ or Chrome 27+</li>
Mac OS 10.6+ running Firefox 22+ or Chrome 27+</li>
Latest version of Flash Player</li>
If you plan to share your screen, you will need to install a small application - you will be prompted to install it the first time you log into the platform.</li>
</ul>
Some of these requirements are lies. I used Firefox 46.0 on Ubuntu 14.04. I did rewrite my slides in LibreOffice because it emits better PDFs than the HTML tools I normally use, but I was also looking for an excuse to rewrite them to clean up their organization.</p>
I clicked around in the "Presentation Manager XD" UI and downloaded a file called "ON24-ScreenShare-plugin", then chmod +x</code>'d it and executed it with ./ON24-ScreenShare-plugin</code>. This caused Wine to run, install some Gecko stuff, and start the screenshare plugin sufficiently well to share my screen to the webcast tool.</p>
I had to re-run the plugin in Wine after logging out of and back into my window manager, of course. Additionally, the screenshare window's resizing is finicky. It's fine to grab and drag the highlighted parts of the window's border with the mouse, but the meta+click command with which one usually moves windows in i3 causes the sides of the screenshare window to move independently of each other.</p>
Here's what the webcast UI looked like during streaming, just at the end of the Kotlin talk while I was getting ready to start mine:</p>
</p>
The Talk</h1>
As previously mentioned, I rewrote my talk in LibreOffice Impress --ostensibly to get a prettier PDF, but also because it's been a month or two since I last prepared for it and re-writing helps me refresh my memory and verify that all my facts are up to date.</p>
GUI-based slide editing is downright painful after using rst-based tools for so long, especially because LibreOffice has no good way to embed code samples out of the box. I ended up going with screenshots from the Rust playground, which were probably better than embedded code samples, but relearning how to edit slides like a regular person wasn't a pleasant experience.</p>
I took more notes than I normally do, since nobody on the webcast could see whether I was reciting or reading. I'm glad I did, as having the notes on a physical page in front of me was reassuring and helped me avoid missing any important points.</p>
I rehearsed the timing of each section of my slides individually, since it naturally broke down into 7 or so discrete parts and I had previously calculated how much of my hour to allocate to each section. Most sections ran consistently over time when preparing, yet under time during the actual talk. The lesson here is to rehearse until I'm happy with a section and can make it the same duration twice in a row.</p>
The experience of presenting a talk in a subjectively empty room made me realize just what high-bandwidth communication regular conferences are.</p>
Pros:</p>

No need to worry about eye contact</li>
All the notes you want</li>
Can't see anyone sleeping</li>
Chat channel allowed instant distribution of links</li>
Chat channel allowed expert attendees to help answer questions</li>
Presentation software allowed gracefully skipping slides, rather than the usual paging back and forth with the arrow keys</li>
</ul>
Cons:</p>

Can't take quick surveys by show of hands</li>
Negligible feedback on how many people are there and their body language of engagement/disengagement</li>
Silences are super awkward</li>
Can't see the shy attendees, in order to encourage participation</li>
</ul>
The audience asked fewer questions during the talk than I expected. Fortunately, they came up with plenty of questions at the end -- extra fortunate because I overcompensated on time and finished my slides about 15 minutes before the end of my speaking slot!</p>
Q&A was surprisingly relaxing, as it was totally up to me which questions to answer. I'll admit that I did select in favor of those that I could answer concisely and eloquently, deferring the questions that didn't make as much sense to think about them while I answered easier ones.</p>
tl;dr</h1>
In my experience, presenting a webcast was lower-stress and comparably impactful to a conference talk.</p>
For would-be presenters concerned about their or the audience's appearance, the visual anonymity of a webcast could be a great place to start a speaking career.</p>
Speakers accustomed to presenting in rooms full of humans should expect subtle feedback, like nods, smiles, and laughter, to be totally invisible in a webcast environment.</p>
And if O'Reilly asks you to do a webcast with them, I'd say go for it -- they made the whole experience as seamless and easy as possible.</p>


Paths Into DevOps
2016-05-05T00:00:00+00:00
</p>
Today, Carol asked</a> me about how a current sysadmin can pivot into a junior "devops" role. 10 tweets into the reply, it became obvious that my thoughts on that type of transition won't fit well into 140-character blocks.</p>
My goal with this post is to catalog what I've done to reach my current level of success in the field, and outline the steps that a reader could take to mimic me.</p>
Facets of DevOps</h1>
In my opinion, 3 distinct areas of focus have made me the kind of person from whom others solicit DevOps advice:</p>

Cultural background</li>
Technical skills</li>
Self-promotion</li>
</ul>
I place "cultural background" first because many people with all the skills to succeed at "DevOps" roles choose or get stuck with other job titles, and everyone touches a slightly different point on the metaphorical elephant of what "DevOps" even means.</p>
Cultural Background</h1>
What does "DevOps" mean to you?</p>

Sysadmins who aren't afraid of automation?</li>
2 sets of job requirements for the price of 1 engineer?</li>
Developers who understand what the servers are actually doing?</li>
Reducing the traditional divide between "development" and "operations" silos?</li>
A buzzword that increases your number of weekly recruiter emails?</li>
People who use configuration management, aka "infrastructure as code"?</li>
</ul>
From my experiences starting Oregon State University's DevOps Bootcamp training program, speaking on DevOps related topics at a variety of industry conferences, and generally being a professional in the field, I've seen the term defined all of those ways and more.</p>
Someone switching from "sysadmin" to "devops" should clearly define how they want their day-to-day job duties to change, and how their skills will need to change as a result.</strong></p>
Technical Skills</h1>
The best way to figure out the technical skills required for your dream job will always be to talk to people in the field, and read a lot of job postings to see what you'll need on your resume and LinkedIn to catch a recruiter's eye.</p>
In my opinion, the bare minimum of technical skills that an established sysadmin would need in order to apply for DevOps roles are:</p>

Use a configuration management tool -- Chef, Puppet, Salt, or Ansible -- to set up a web server in a VM.</li>
Write a script in Python to do something more than Hello World -- an IRC bot or tool to gather data from an API is fine.</li>
Know enough about Git and GitHub to submit a pull request to fix something about an open source tool that other sysadmins use, even if it's just a typo in the docs.</li>
Understand enough about continuous integration testing to use it on a personal project, such as TravisCI on a GitHub repo, and appreciate the value of unit and integration tests for software.</li>
Be able to tell a story about successfully troubleshooting a problem on a Linux or BSD server, and what you did to prevent it from happening again.</li>
</ul>
Keep in mind that your job in an interview is to represent what you know and how well you can learn new things. If you're missing one of the above skills, go ask for help on how to build it.</p>
Once you have all the experiences that I listed, you are no longer allowed to skip applying for an interesting role because you don't feel you know enough</strong>. It's the employer's job to decide whether they want to grow you into the candiate of their dreams, and your job to give them a chance. Remember that a job posting describes the person leaving a role, and if you started with every skill listed, you'd probably be bored and not challenged to your full potential.</p>
Self Promotion</h1>
"DevOps" is a label that engineers apply to themselves, then justify with various experiences and qualifications.</p>
The path to becoming a community leader begins at engaging with the community. Look up DevOps-related conferences</strong> -- find video recordings of talks from recent DevOps Days events, and see what names are on the schedules of upcoming DevOps conferences.</p>
Look at which technologies the recent conferences have discussed, then look up talks about them from other events. Get into the IRC or Slack channels of the tools you want to become more expert at, listen until you know the answers to common questions, then start helping beginners newer than yourself</strong>.</p>
Reach out to speakers</strong> whose talks you've enjoyed, and don't be afraid to ask them for advice. Remember that they're often extremely busy, so a short message with a compliment on their talk and a specific request for a suggestion is more likely to get a reply than overly vague requests. This type of networking will make your name familiar when their companies ask them to help recruit DevOps engineers, and can build valuable professional friendships that provide job leads and other assistance.</p>
Contribute to the DevOps-related projects that you identify as having healthy communities. For configuration management, I've found that SaltStack is a particularly welcoming group. Find the source code on GitHub, examine the issue tracker, pick something easy, and submit a pull request fixing the bug. As you graduate to working on more challenging or larger issues, remember to advertise your involvment with the project on your LinkedIn profile</strong>!</p>
Additionally, help others out by blogging what you learn during these adventures. If you ever find that Google doesn't have useful results for an error message that you searched, write a blog post</strong> with the message and how you fixed it. If you're tempted to bikeshed over which blogging platform to use, default to GitHub Pages, as a static site hosted there is easy to move to your own hosting later if you so desire.</p>
Examine job postings for roles like you want, and make sure the key buzzwords appear on your LinkedIn profile wherever appropriate. A complete LinkedIn profile for even a relatively new DevOps engineer draws a surprising number of recruiters for DevOps-related roles. If you're just starting out in the field, I'd recommend expressing interest in every</em> opportunity that you're contacted about, progressing to at least a phone interview if possible, and getting all the feedback you can about your performance afterwards. It's especially important to interview at companies that you can't see yourself enjoying a job at</strong>, because you can practice asking probing questions that tell you whether an employer will be a good fit for you. (check out this post</a> for ideas).</p>
Another trick for getting to an interview is to start something with DevOps in the name. It could be anything from a curated blog to a meetup to an online "book club" for DevOps-related papers, but leading something with a cool name seems to be highly attractive to recruiters. Another way to increase your visibility in the field is to give a talk</strong> at any local conference, especially LinuxFest and DevOpsDays events. Putting together an introductory talk on a useful technology only requires intermediate proficiency, and is a great way to build your personal brand.</p>
To summarize, there are really 4 tricks to getting DevOps interviews, and you should interview as much as you can to get a feeling for what DevOps means to different parts of the industry:</p>

Contribute back to the open source tools that you use</li>
Network with established professionals</li>
Optimize your LinkedIn and other professional profiles to draw recruiters</li>
Be the founder of something.</li>
</ul>
Questions?</h1>
I collect interesting job search and interview advice links at the bottom of my resume repo readme</a>.</p>
I bolded each paragraph's key points in the hopes of making them easier to read.</p>
You're welcome to reach out to me at blog</span> @ edunham.net</span> or @qedunham</span> on Twitter if you have other questions. If I made a dumb typo or omitted some information in this post, either tell me about it or just throw a pull request at the repo</a> to fix it and give yourself credit.</p>


Persona and third-party cookies in Firefox
2016-04-18T00:00:00+00:00
Although its front page claims we've deprecated persona</a>, it's the only way to log into the statusboard</a> and Air Mozilla</a>. For a long time, I was unable to log into any site using Persona from Firefox 43 and 44 because of an error about my browser not being configured to accept third-party cookies.</p>
The support article</a> on the topic says that checking the "always accept cookies" box should fix the problem. I tried setting "accept third-party cookies" to "Always", and yet the error persisted. (setting the top-level history configuration to "always remember history" didn't affect the error either).</p>
Fortunately, there's also an "Exceptions" button by the "Accept cookies from sites" checkbox. Editing the exceptions list to universally allow "http://persona.org</a>" lets me use Persona in Firefox normally.</p>
</p>
Note: The original screenshot is not available in this archive.</em></p>
That's the fix, but I don't know whose bug it is. Did Firefox mis-balance privacy against convenience? Is the "always accept third-party cookies" setting's failure to accept a cookie without an exception some strange edge case of a broken regex? Is Persona in the wrong for using a design that requires third-party cookies at all? Who knows!</p>


Plushie Rustacean Pattern
2016-04-11T00:00:00+00:00
I made a Rustacean</a>. He's cute. You can make one too.</p>
</p>
You'll Need</h1>

A couple square feet of orange polar fleece, or any other orange fabric that won't stretch or fray too much</li>
A handful of stuffing. I cannibalized a throw pillow.</li>
A needle and some orange thread</li>
Black and white fabric scraps and thread, or black and white embroidery floss, for making the face.</li>
Intermediate sewing skills</li>
This pattern</li>
</ul>
The Pattern</h1>
</p>
Get yourself a front, back, underside, and claw drawn on paper, either by printing them out or tracing from a screen. The front, back, and underside should have horizontal symmetry, except for the face placement. Make sure the points marked in red and blue on this pattern are noted on your paper.</p>
Mine measure about 6" wide between the points marked in red.</p>
Sewing vocabulary</h1>

The right side</code> of a fabric is what ends up on the outside of the finished item. The wrong side</code> ends up where you can't see it. Some fabrics have both sides the same; in that case, the wrong side is whichever one you feel like tracing the pattern onto.</li>
seam allowance</code> is some extra fabric that ends up on the inside of the item when you're done. The pattern above does not</strong> include seam allowance. This means that if you cut the fabric along the lines in the pattern, your finished rustacean will be tiny and sad and shaped wrong. You cut the paper along the lines, then trace it onto the fabric, then sew along the lines.</li>
applique</code> is where you sew one piece of fabric onto the surface of another to make a design.</li>
There are a bunch of great youtube videos on basic sewing skills. Watch whichever ones you need.</li>
</ul>
Assembly</h1>

Trace a front, a back, an underside, and the 2 claws onto the wrong side of your fabric with whatever will write on it without bleeding through. Make sure to transfer the blue centerline marks and the red three-point join marks.</li>
Cut out the shapes you just traced, leaving about 1" of margin around them. We'll trim the seams properly later, so don't worry about getting it exact.</li>
Find a couple claw-sized chunks of leftover fabric and pin one to the back of each claw (right sides together, of course).</li>
Sew around both claws, leaving the arm ends open so you can turn them. I find it's easiest to backstitch, and you can get away with stitches up to about 1.5mm apart with normal weight polar fleece.</li>
Trim around the outside of the seams on the claws to leave about 1/4" seam allowance, and clip right up to the stitches in the concave spot. If you backstitched, make sure flip them over before trimming the seams so you don't accidentally cut through the longer stitches.</li>
Turn the claws so the right side of the fabric is out and the seams are on the inside, and stuff them with stuffing or fabric scraps. A pair of wooden chopsticks from a fast food place are a great tool for turning and stuffing.</li>
Put the front and back pieces right sides together so the points marked in red and blue on the pattern line up. Pin them together.</li>
Sew from one red mark to the other along Ferris's spiky back.</li>
Trim around the spikes leaving about 1/4" seam allowance, clipping right up to the seam in the concave spots.</li>
Figure out which side is front (hint, it has only 2 legs rather than 4). Imagine where Ferris's little face will go when he's finished. Now, pin both claws onto the right side</strong> of the front piece, so they'll be oriented correctly when he's done. If in doubt, pin the bottom front in place and turn the whole thing inside out to make sure the claws are right.</li>
Match the center front of the underside with the center of Ferris's front (both have a blue +</code> on the pattern). Be sure the pieces have their right sides together and the claws are sandwiched between them.</li>
Match the points marked with red triangles on each side of the front and underside together and pin them. If the claws are sticking out at this point, go back to step 10 and try again</strong></li>
Sew from one red mark to the other to join Ferris's front to the front of his underside. Put a few extra stitches in the part of the seam where his "arms"/claws are attached, to make sure they can't be pulled out.</li>
Trim around the 2 tiny legs that you've sewn so far, with about 1/8" seam allowance.</li>
Now you can applique his face onto the right side of his front. Or embroider it if you know how. Cut the black and white felt scraps into face-shaped pieces and sew them down, giving Ferris whatever expression you want.</li>
Line up the 4 back legs on the underside and back pieces, and pin them right sides toether. Sew everything except the part marked in green --that's the hole through which you'll turn him inside out.</li>
Trim around those last 4 legs, leaving at least 1/8" seam allowance. Don't cut away any more fabric from the bit marked in green. If you leave a bit of extra fabric around the leg seams, they'll be harder to turn but require less stuffing.</li>
Turn Ferris right side out. Again, chopsticks or the non-pointy end of a barbeque skewer are useful for getting the pointy bits to do the right thing.</li>
Stuff Ferris with the filling. I filled mine quite loosely, because it makes him softer and more huggable. If you overfill his body, his spikes will look silly. If you overfill his legs, they'll stick out in funny directions and not bend right.</li>
Tuck the seam allowance back into the hole through which you stuffed Ferris and sew it shut. Congratulations, you have your own toy crab!</li>
</ol>
The Finished Product</h1>
</p>
He's cute, cuddly, and palm-sized. Lego dude for scale.</p>


Could Rust have a left-pad incident?
2016-03-24T00:00:00+00:00
The short answer: No.</p>
What happened with left-pad</code>?</h1>
The Node community had a lot of drama</a> this week when a developer unpublished a package on which a lot of the world depended.</p>
This was fundamentally possible because NPM offers an unpublish</a> feature. Although the docs for unpublish</code> admonish users that "It is generally considered bad behavior to remove versions of a library that others are depending on!" in large bold print, the feature is available.</p>
What's the Rust equivalent?</h1>
The Rust package manager, Cargo, is similar to NPM in that it helps users get the libraries on which their projects depend. Rust's analog to the NPM index is crates.io</a>.</p>
The best explanation of Cargo's robustness against unpublish</code> exploits is the docs themselves</a>:</p>

cargo yank</strong></p>
Occasions may arise where you publish a version of a crate that actually ends up being broken for one reason or another (syntax error, forgot to include a file, etc.). For situations such as this, Cargo supports a “yank” of a version of a crate.:</p>
$ cargo yank --vers 1.0.1
</span>$ cargo yank --vers 1.0.1 --undo
</span></code></pre>
A yank does not</strong> delete any code. This feature is not intended for deleting accidentally uploaded secrets, for example. If that happens, you must reset those secrets immediately.</p>
The semantics of a yanked version are that no new dependencies can be created against that version, but all existing dependencies continue to work. One of the major goals of crates.io is to act as a permanent archive of crates that does not change over time, and allowing deletion of a version would go against this goal. Essentially a yank means that all projects with a Cargo.lock</code> will not break, while any future Cargo.lock</code> files generated will not list the yanked version.</p>
</blockquote>
As Cargo author Alex Crichton clarified in a GitHub comment</a> yesterday, the only way that it's possible to remove code from crates.io</code> is to compel the Rust tools team to edit the database and S3 bucket.</p>
Even if a crate maintainer leaves the community in anger or legal action is taken against a crate, this workflow ensures that code deletion is only possible by a small group of people with the motivation and authority to do it in the way that's least problematic for users of the Rust language.</p>
For more information on the crates.io package and copyright policies, see this internals thread</a>.</p>
But I just want to left pad a string in Rust??</h1>
Although a left-pad crate</a> was created as a joke, you should probably just use the format! built-in</a> from the standard library.</p>


Reducing SaltStack log verbosity for TravisCI
2016-03-23T00:00:00+00:00
Servo has some Salt</a> configs, hosted on GitHub, for which changes are smoke-tested on TravisCI before they're deployed. Travis only shows the first 10k lines of log output, so I want to minimize the amount of extraneous information that the states print.</p>
My salt state looks like::</p>
android-sdk:
</span>  archive.extracted:
</span>    - name: {{ common.homedir }}/android/sdk/{{ android.sdk.version }}
</span>    - source: https://dl.google.com/android/android-sdk_{{
</span>      android.sdk.version }}-linux.tgz
</span>    - source_hash: sha512={{ android.sdk.sha512 }}
</span>    - archive_format: tar
</span>    - archive_user: user
</span>    - if_missing: {{ common.homedir }}/android/sdk/{{ android.sdk.version
</span>      }}/android-sdk-linux
</span>    - require:
</span>      - user: user
</span></code></pre>
The output in TravisCI is::</p>
ID: android-sdk
</span>Function: archive.extracted
</span>Name: /home/user/android/sdk/r24.4.1
</span>Result: True
</span>Comment: https://dl.google.com/android/android-sdk_r24.4.1-linux.tgz extracted in /home/user/android/sdk/r24.4.1/
</span>Started: 17:46:25.900436
</span>Duration: 19540.846 ms
</span>Changes:
</span>    ----------
</span>    directories_created:
</span>        - /home/user/android/sdk/r24.4.1/
</span>        - /home/user/android/sdk/r24.4.1/android-sdk-linux
</span>
</span>    extracted_files:
</span>        ... 2755 lines listing one file per line that I don't want to see in the log
</span></code></pre>
https://docs.saltstack.com/en/latest/ref/states/all/salt.states.archive.html</a> has useful guidance on how to increase the tar</code> state's verbosity, but not to decrease it. This is because the extra 2755 lines aren't coming from tar</code> itself, but from Salt assuming that we want to know.</p>
terse</code> outputter settings</h1>
The outputter</a> takes several state_output</code> setting options. The terse</code> option summarizes the result of each state into a single line.</p>
There are a couple places you can set this:</p>

Invoke Salt with salt --state-output=terse hostname state.highstate</code></li>
Add the line state_output: terse</code> to /etc/salt/minion</code>, if you're using salt-call</code></li>
Setting state_output_terse</code> is apparently</a> an option, though I can't find any example of a real-world salt config that uses it</li>
</ul>
Setting the terse</code> option in /etc/salt/minion</code> dropped the output of a highstate from over 10,000 lines to about 2500.</p>


Fixing sudo errors from the command line on OSX
2016-03-14T00:00:00+00:00
The first symptom that I had made a terrible mistake showed up in an Ansible playbook:</p>
GATHERING FACTS
</span>***************************************************************
</span>fatal: [...] => ssh connection closed waiting for a privilege escalation password prompt
</span>fatal: [...] => ssh connection closed waiting for a privilege escalation password prompt
</span>fatal: [...] => ssh connection closed waiting for sudo password prompt
</span>fatal: [...] => ssh connection closed waiting for sudo password prompt
</span></code></pre>
That looks like the sudo binary might be broken. To rule out Ansible problems, remote into the machine and try to use sudo:</p>
administrators-Mac-mini:~ administrator$ sudo ls
</span>sudo: effective uid is not 0, is sudo installed setuid root?
</span></code></pre>
This meant that there was a file permissions problem:</p>
working-host administrator$ ls -al /usr/bin/sudo
</span>-r-s--x--x  1 root  wheel  164560 Sep  9  2014 /usr/bin/sudo
</span>
</span>broken-host administrator$ ls -al /usr/bin/sudo
</span>-rwxrwxr-x  1 root  wheel  164560 Sep  9  2014 /usr/bin/sudo
</span></code></pre>
Now the problem is reduced to fixing the permissions. One does not simply sudo to root, because there's no working sudo. However, Apple provides a utility</a> which allows you to enable root login using only the administrator account's permissions:</p>
broken-host administrator$ dsenableroot
</span>username = administrator
</span>user password:
</span>root password:
</span>verify root password:
</span>
</span>dsenableroot:: ***Successfully enabled root user.
</span></code></pre>
The first password is the current one for the administrator account, and the other two should be the same string and will become the root account's password.</p>
After enabling root login, disconnect then SSH into the host as root:</p>
broken-host root# chmod 4411 /usr/bin/sudo
</span></code></pre>
And test that the fix fixed it:</p>
broken-host root# su administrator
</span>broken-host administrator$ sudo ls
</span></code></pre>
Finally, clean up after yourself to inconvenience any future attackers:</p>
broken-host administrator$ dsenableroot -d
</span></code></pre>
Moral of the story: Errant chowns of /usr/bin</code> are just as bad when they come from automation as when they come from humans.</p>


Ansible, Vagrant, and changed host keys
2016-03-08T00:00:00+00:00
Related to this bug</a>, the Vagrant Ansible provisioner</a> seems to ignore some system settings.</p>
The symptom is that when you update a previously used Vagrant box, or otherwise change its host key, Ansible provisioning fails with the error:</p>
fatal: [hostname] => SSH Error: Host key verification failed.
</span>    while connecting to 127.0.0.1:2200
</span>It is sometimes useful to re-run the command using -vvvv, which prints SSH
</span>debug output to help diagnose the issue.
</span></code></pre>
The standard solution would be to forget about the old host key with ssh-keygen -R 127.0.0.1:2200</code> or ignore the change with export ANSIBLE_HOST_KEY_CHECKING=false</code>.</p>
If you trust the box not to be evil and expect its host key to change frequently due to your testing, a fix which the Ansible provisioner does respect is to add ansible.host_key_checking = false</code> to the Vagrantfile, like:</p>
Vagrant.configure(2) do |config|
</span>...
</span>    config.vm.define "hostname" do |prodmaster|
</span>        hostname.vm.provision "ansible" do |ansible|
</span>            ansible.playbook = "provision/hostname.yaml"
</span>            ansible.sudo = true
</span>            ansible.host_key_checking = false
</span>            ansible.verbose = 'vvvv'
</span>            ansible.extra_vars = { ansible_ssh_user: 'vagrant'}
</span>        end
</span>    end
</span>...
</span>end
</span></code></pre>


Vidyo with Ubuntu and i3wm
2016-03-07T00:00:00+00:00
Mozilla uses Vidyo</a> for virtual meetings across distributed teams. If it doesn't work on your laptop, you can use the mobile client or book a meeting room in an office, but neither of those solutions is optimal when working from home.</p>
Vidyo users within Mozilla can download a .deb</code> or .rpm</code> installer from v.mozilla.org</a>. On Ubuntu, it's easy to install the downloaded package with sudo dpkg -i path/to/the/file.deb</code>.</p>
The issue is that when you invoke VidyoDesktop</code> from your launcher of choice (dmenu for me), i3 does what's usually the right thing and makes the client fullscreen in a tile. This doesn't allow the interface to pop up a floating window with the confirm dialog when you try to join a room, so you can't.</p>
mod + shift + space</h1>
Mod was alt</code> by default last time I installed i3, but I've since remapped it to the window key (as IRC clients use alt for switching windows). Some people use caps lock as their mod key.</p>
mod + shift + space</code> makes the window floating</a>, which allows it to pop up the confirmation dialog when you try to join a call.</p>
Float windows by default</h1>
Alternately, stick the line:</p>
for_window [class="VidyoDesktop"] floating enable
</span></code></pre>
in your ~/.i3/config</code>.</p>
Installing Vidyo despite the libqt4-gui error</h1>
Edited as of May 2017: Recent Vidyos depend on a package that's not available in Ubuntu's repos. The easiest workaround is:</p>
sudo dpkg -i --ignore-depends=libqt4-gui path/to/VidyoInstaller.deb
</span></code></pre>


Are we 'are we' yet?
2016-02-26T00:00:00+00:00
The Rust community, being founded and enjoyed by a variety of Mozilians, seems to have inherited the tradition of tracking top-level progress metrics using are we</a> sites.</p>

Are we concurrent yet?</a> tracks the progres of Rust's concurrency ecosystem</li>
Are we web yet?</a> tracks the status of Rust's HTTP stack, web frameworks, and related libraries</li>
Are we IDE yet?</a> provides a list of what features are supported for Rust per IDE, and links to the relevant tracking issues and RFCs</li>
</ul>
If this blog post was an 'are we' page itself, the big text at the top would probably say "Getting There".</p>


Buildbot WithProperties
2016-02-19T00:00:00+00:00
Today, I copied an existing command from a Buildbot configuration and then modified it to print a date into a file.:</p>
...
</span>if "cargo" in component:
</span>    cargo_date_cmd = "echo `date +'%Y-%m-%d'` > " + final_dist_dir + "/cargo-build-date.txt"
</span>    f.addStep(MasterShellCommand(name="Write date to cargo-build-date.txt",
</span>                             command=["sh", "-c", WithProperties(cargo_date_cmd)] ))
</span>...
</span></code></pre>
It broke:</p>
Failure: twisted.internet.defer.FirstError: FirstError[#8, [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.defer.FirstError'>: FirstError[#2, [Failure instance: Traceback: <type 'exceptions.ValueError'>: unsupported format character 'Y' (0x59) at index 14
</span></code></pre>
Why? WithProperties</a>.</p>
It turns out that WithProperties should only be used when you need to interpolate strings into an argument, using either %s</code>, %d</code>, or %(propertyname)s</code> syntax in the string.</p>
The lesson here is Buildbot will happily accept WithProperties("echo 'this command uses no interpolation'")</code> in a command</code> argument, and then blow up at you if you ever change the command to have a %</code> in it.</p>
However, it appears that build steps run as MasterShellCommand</code>s without WithProperties</code> do not display their name</code> in the waterfall, but rather say "running" or "ran".</p>


Using Notty
2016-02-15T00:00:00+00:00
I recently got the "Hey, you're a Rust Person!" question of how to install notty</a> and interact with it.</p>
A TTY was originally</a> a teletypewriter. Linux users will have most likely encountered the concept of TTYs in the context of the TTY1 interface where you end up if your distro fails to start its window manager. Since you use ctrl + alt + f[1,2,...]</code> to switch between these interfaces, it's easy to assume that "TTY" refers to an interactive workspace.</p>
Notty itself is only</strong> a virtual terminal. Think of it as a library meant as a building block for creating graphical terminal emulators. This means that a user who saw it on Hacker News and wants to play around should not ask "how do I install notty", but rather "how do I run a terminal emulator built on notty?".</p>
Easy Mode</h1>
Get some Rust:</p>
curl -sf https://raw.githubusercontent.com/brson/multirust/master/blastoff.sh | sh
</span>multirust update nightly
</span></code></pre>
Get the system dependencies:</p>
sudo apt-get install libcairo2-dev libgdk-pixbuf2.0 libatk1.0 libsdl-pango-dev libgtk-3-dev
</span></code></pre>
Run Notty:</p>
git clone https://github.com/withoutboats/notty.git
</span>cd notty/scaffolding
</span>multirust run nightly cargo run
</span></code></pre>
And there you have it! As mentioned in the notty</a> README, "This terminal is buggy and feature poor and not intended for general use". Notty is meant as a library for building graphical terminals, and scaffolding is only a minimal</strong> proof of concept.</p>
Explanation: Getting Rust</h1>
Since the Rust language is still under active development, many features are available in the Nightly version of the compiler which are not yet available in Stable. If you got Rust from your package manager</a>, you probably are using Stable. To check, run rustc --version</code> and see whether the result says "nightly" in it.</p>
Notty uses some features that're available in Nightly but not Stable. If you try to compile it with Stable, you'll get an error that makes this obvious:</p>
Compiling notty v0.1.0 (file:///home/edunham/code/notty)
</span>src/lib.rs:16:1: 16:16 error: #[feature] may not be used on the stable release channel
</span>src/lib.rs:16 #![feature(io)]
</span>              ^~~~~~~~~~~~~~~
</span>error: aborting due to previous error
</span>Could not compile `notty`.
</span></code></pre>
When you need to switch between Rust versions frequently, multirust</a> is the tool for the job.</p>
Explanation: Getting system dependencies</h1>
I've reproduced the following error messages in full to help out any confused new Rustaceans Googling for them:</p>
Cairo</a> is a graphics library that you can get from your system package manager. If you try to compile notty's dependencies without it, you'll get an error:</p>
Build failed, waiting for other jobs to finish...
</span>failed to run custom build command for `cairo-sys-rs v0.2.1`
</span>Process didn't exit successfully:
</span>`/home/edunham/code/notty/notty-cairo/target/release/build/cairo-sys-rs-1d0cf50d5d2dab2f/build-script-build`
</span>(exit code: 101)
</span>--- stderr
</span>thread '<main>' panicked at '`"pkg-config" "--libs" "--cflags" "cairo"` did
</span>not exit successfully: exit code: 1
</span>--- stderr
</span>Package cairo was not found in the pkg-config search path.
</span>Perhaps you should add the directory containing `cairo.pc'
</span>to the PKG_CONFIG_PATH environment variable
</span>No package 'cairo' found
</span>', /home/edunham/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/cairo-sys-rs-0.2.1/build.rs:9
</span>note: Run with `RUST_BACKTRACE=1` for a backtrace.
</span></code></pre>
The only other gotcha about the dependencies is that errors about gdk</code> actually mean you need to install the libgtk-3-dev</code> package:</p>
failed to run custom build command for `gdk-sys v0.2.1`
</span>Process didn't exit successfully:
</span>`/home/edunham/code/notty/scaffolding/target/release/build/gdk-sys-e1b0a13b32593729/build-script-build`
</span>(exit code: 101)
</span>--- stderr
</span>thread '<main>' panicked at '`"pkg-config" "--libs" "--cflags" "gdk-3.0"` did
</span>not exit successfully: exit code: 1
</span>--- stderr
</span>Package gdk-3.0 was not found in the pkg-config search path.
</span>Perhaps you should add the directory containing `gdk-3.0.pc'
</span>to the PKG_CONFIG_PATH environment variable
</span>No package 'gdk-3.0' found
</span>', /home/edunham/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/gdk-sys-0.2.1/build.rs:17
</span>note: Run with `RUST_BACKTRACE=1` for a backtrace.
</span></code></pre>
Running notty</h1>
Compiling and running scaffolding</code> necessarily builds a bunch of dependencies, some of which throw various warnings. You also might be able to crash scaffolding</code> with an error such as:</p>
thread '<main>' panicked at 'not yet implemented', .../notty/src/datatypes/mod.rs:160
</span></code></pre>
This</a>, along with everywhere else that unimplemented!()</code> occurrs in the notty source code, is an opportunity for you to contribute and help improve the project!</p>


How much knowledge do you need to give a conference talk?
2016-01-19T00:00:00+00:00
I was recently asked an excellent question when I promoted the LFNW CFP</a> on IRC:</p>

As someone who has never done a talk, but wants to, what kind of knowledge do you need about a subject to give a talk on it?</p>
</blockquote>
If you answer "yes" to any of the following questions, you know enough to propose a talk:</p>

Do you have a hobby</strong> that most tech people aren't experts on? Talk about applying a lesson or skill from that hobby to tech! For instance, I turned a habit of reading about psychology into my Human Hacking</a> talk.</li>
Have you ever spent a bunch of hours forcing two tools to work with each other, because the documentation wasn't very helpful and Googling didn't get you very far, and built something useful? "How to build ___ with ___" makes a catchy talk title, if the thing you built</strong> solves a common problem.</li>
Have you ever had a mentor sit down with you and explain a tool or technique, and the new understanding improved the quality of your work or code? Passing along useful lessons from your mentors</strong> is a valuable talk, because it allows others to benefit from the knowledge without taking as much of your mentor's time.</li>
Have you seen a dozen newbies ask the same question over the course of a few months? When your answer to a common question</strong> starts to feel like a broken record, it's time to compose it into a talk then link the newbies to your slides or recording!</li>
Have you taken a really interesting class</strong> lately? Can you distill part of it into a 1-hour lesson that would appeal to nerds who don't have the time or resources to take the class themselves? (thanks lucyw</a> for adding this to the list!)</li>
Have you built a cool thing that over a dozen other people use? A tutorial talk</strong> can not only expand your community, but its recording can augment your documentation and make the project more accessible for those who prefer to learn directly from humans!</li>
Did you benefit from a really great introductory talk when you were learning a tool? Consider doing your own tutorial! Any conference with beginners in their target audience needs at least one Git lesson, an IRC talk, and some discussions of how to use basic Unix utilities. These introductory talks</strong> are actually better when given by someone who learned the technology relatively recently, because newer users remember what it's like not to know how to use it. Just remember to have a more expert user look over your slides before you present, in case you made an incorrect assumption about the tool's more advanced functionality.</li>
</ul>
I personally try to propose talks I want to hear, because the dealine of a CFP or conference is great motivation to prioritize a cool project over ordinary chores.</p>


Buildbot and EOFError
2016-01-16T00:00:00+00:00
More SEO-bait, after tracking down an poorly documented problem:</p>
# buildbot start master
</span>Following twistd.log until startup finished..
</span>2016-01-17 04:35:49+0000 [-] Log opened.
</span>2016-01-17 04:35:49+0000 [-] twistd 14.0.2 (/usr/bin/python 2.7.6) starting up.
</span>2016-01-17 04:35:49+0000 [-] reactor class: twisted.internet.epollreactor.EPollReactor.
</span>2016-01-17 04:35:49+0000 [-] Starting BuildMaster -- buildbot.version: 0.8.12
</span>2016-01-17 04:35:49+0000 [-] Loading configuration from '/home/user/buildbot/master/master.cfg'
</span>2016-01-17 04:35:53+0000 [-] error while parsing config file:
</span>    Traceback (most recent call last):
</span>      File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 577, in _runCallbacks
</span>        current.result = callback(current.result, *args, **kw)
</span>      File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1155, in gotResult
</span>        _inlineCallbacks(r, g, deferred)
</span>      File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1099, in _inlineCallbacks
</span>        result = g.send(result)
</span>      File "/usr/local/lib/python2.7/dist-packages/buildbot/master.py", line 189, in startService
</span>        self.configFileName)
</span>    --- <exception caught here> ---
</span>      File "/usr/local/lib/python2.7/dist-packages/buildbot/config.py", line 156, in loadConfig
</span>        exec f in localDict
</span>      File "/home/user/buildbot/master/master.cfg", line 415, in <module>
</span>        extra_post_params={'secret': HOMU_BUILDBOT_SECRET},
</span>      File "/usr/local/lib/python2.7/dist-packages/buildbot/status/status_push.py", line 404, in __init__
</span>        secondaryQueue=DiskQueue(path, maxItems=maxDiskItems))
</span>      File "/usr/local/lib/python2.7/dist-packages/buildbot/status/persistent_queue.py", line 286, in __init__
</span>        self.secondaryQueue.popChunk(self.primaryQueue.maxItems()))
</span>      File "/usr/local/lib/python2.7/dist-packages/buildbot/status/persistent_queue.py", line 208, in popChunk
</span>        ret.append(self.unpickleFn(ReadFile(path)))
</span>    exceptions.EOFError:
</span>
</span>2016-01-17 04:35:53+0000 [-] Configuration Errors:
</span>2016-01-17 04:35:53+0000 [-]   error while parsing config file:  (traceback in logfile)
</span>2016-01-17 04:35:53+0000 [-] Halting master.
</span>2016-01-17 04:35:53+0000 [-] Main loop terminated.
</span>2016-01-17 04:35:53+0000 [-] Server Shut Down.
</span></code></pre>
This happened after the buildmaster's disk filled up and a bunch of stuff was manually deleted. There were no changes to master.cfg since it worked perfectly.</p>
The fix was to examine master.cfg</code> to see where the HttpStatusPush was created</a>, of the form:</p>
c['status'].append(HttpStatusPush(
</span>    serverUrl='http://build.servo.org:54856/buildbot',
</span>    extra_post_params={'secret': HOMU_BUILDBOT_SECRET},
</span>))
</span></code></pre>
Digging in the Buildbot source reveals that persistent_queue.py</code> wants to unpickle a cache file from /events_build.servo.org/-1</code> if there was nothing in /events_build.servo.org/</code>. To fix this the right way, create that file and make sure Buildbot has +rwx</code> on it.</p>
Alternately, you can give up on writing your status push cache to disk entirely by adding the line maxDiskItems=0</code> to the creation of the HttpStatusPush, giving you:</p>
c['status'].append(HttpStatusPush(
</span>   serverUrl='http://build.servo.org:54856/buildbot',
</span>   maxDiskItems=0,
</span>   extra_post_params={'secret': HOMU_BUILDBOT_SECRET},
</span>))
</span></code></pre>
The real moral of the story is "remember to use logrotate</a>.</p>


Who would you hire?
2016-01-13T00:00:00+00:00
If you're using open source as a portfolio to make yourself a more competitive job candidate, it can feel like you have to start your own project to show off your skills.</p>
In the words of one job seeker I chatted with recently, "I feel like most of my contributions [to other peoples' projects] aren't that significant or noteworthy". Here's a thought experiment to justify including projects to which you contribute, even without a leadership role, on your resume:</p>

Imagine you want to hire a coder.</p>
Candidate A always works alone and refuses to contribute to a project if it doesn't make her look like a rockstar.</p>
Candidate B triages the unglamorous issues that affect multiple users, and steadily produces small, self-contained fixes that avoid introducing new bugs.</p>
</blockquote>
When the situation is framed in these terms, I hope that it's obvious which coder you'd want on your team.</p>
When writing your resume, there's only space to include a few of the many activities in which you invest your time. It's tempting to only include your biggest, highest-profile solo projects, while disregarding those projects to which you've made a small but steady stream of useful contributions.</p>
Reread your resume from the perspective of someone who hasn't met you yet and has only the information in that document available to form a first impression of your character. Which of the 2 hypothetical coders does it make you sound like? Is that how you really are?</p>


Troubleshooting stunnel
2016-01-09T00:00:00+00:00
Today I've learned a few things aout how stunnel</a> works. The main takeaway is that Googling for specific errors in the stunnel log is incredibly unhelpful, resulting in a variety of mailing list posts with no replies. Tracking an error message through the source of the program doesn't lead to any useful comments, either. So here's some SEO bait with concrete troubleshooting suggestions.</p>
I started out, as usual, with a pile of errors:</p>
/usr/local/bin/stunnel
</span>[ ] Clients allowed=2000
</span>[ ] Cron thread initialized
</span>[.] stunnel 5.27 on x86_64-apple-darwin14.0.0 platform
</span>[.] Compiled/running with OpenSSL 1.0.2e 3 Dec 2015
</span>[.] Threading:PTHREAD Sockets:POLL,IPv6 TLS:ENGINE,FIPS,OCSP,PSK,SNI
</span>[ ] errno: (*__error())
</span>[.] Reading configuration from file stunnel.conf
</span>[.] UTF-8 byte order mark not detected
</span>[ ] Initializing service [9987]
</span>[!] Error resolving "127.0.0.1": Neither nodename nor servname known (EAI_NONAME)
</span>[ ] Cannot resolve connect target - delaying DNS lookup
</span>[ ] No certificate or private key specified
</span>[ ] SSL options: 0x03000004 (+0x03000000, -0x00000000)
</span>[.] Configuration successful
</span>[ ] Listening file descriptor created (FD=6)
</span>[!] bind: Address already in use (48)
</span>[!] Error binding service [9987] to 127.0.0.1:9987
</span>[ ] Closing service [9987]
</span>[ ] Service [9987] closed
</span>stunnel startup failed, already running?
</span></code></pre>
[!] Error resolving "127.0.0.1": Neither nodename nor servname known</h1>
This was the biggest wat</a>, and the hardest to track down because the solution is so obvious.</p>
"Error resolving" sounds like the machine hasn't been informed of localhost's existance, so let's check:</p>
$ cat /etc/hosts
</span>127.0.0.1   localhost
</span>255.255.255.255 broadcasthost
</span>::1             localhost
</span></code></pre>
And I can even ping 127.0.0.1</code> successfully. So the message and I have different ideas about what it means to "resolve" an IP.</p>
I found the fix here by diffing the stunnel.conf</code> against that on a working machine, and learned that I'd neglected to specify the correct port number on the destination host.</p>
The solution to the "Error resolving localhost" turned out to be specifying the correct port for the other end of the stunnel</strong>:</p>
$ cat stunnel.conf
</span>pid =
</span>
</span>[9987]
</span>client = yes
</span>accept = 127.0.0.1:9987
</span>cafile = ./cert.pem
</span>verify = 3
</span>connect = 01.23.456.789:9988
</span></code></pre>
Wow. Painfully obvious after you realize what's wrong, and just plain painful before.</p>
[!] Error binding service [9987] to 127.0.0.1:9987</h1>
The "already running?" hint is correct here. This error means stunnel didn't let go of the port despite failing to start on a previous attempt.</p>
Easy fix; check whether it's really stunnel hogging the port:</p>
$ lsof -i tcp:9987
</span>COMMAND PID      USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
</span>stunnel 363        me    6u  IPv4 0x20f17e5e0dd35277      0t0  TCP localhost:dsm-scm-target (LISTEN)
</span></code></pre>
and if so, whack it with a metaphorical hammer:</p>
$ sudo killall stunnel
</span></code></pre>
Tada!</h1>
After getting the destination IP+port combination specified correctly and the old broken stunnel killed, the stunnel starts successfully.</p>


Questions about Open Source and Design
2015-12-21T00:00:00+00:00
Today, I posed a question to some professional UI and UX designers:</p>

How can an open source project without dedicated design experts collaborate with amateur, volunteer designers to produce a well-designed product?</p>
</blockquote>
They revealed that they've faced similar collaboration challenges, but knew of neither a specific process to solve the problem nor an organization that had overcome it in the past.</p>
Have you solved this problem? Have you tried some process or technique and learned that it's not able to solve the problem? Email me (design@edunham.net</code>) if you know of an open source project that's succeeded at opening their design as well, and I'll update back here with what I learn!</p>
In no particular order, here are some of the problems that we were talking about:</p>

Non-designers struggle to give constructive feedback on design. I can say "that's ugly" or "that's hard to use" more easily than I can say "here's how you can make it better".</li>
Projects without designers in the main decision-making team can have a hard time evaluating the quality of a proposed design.</li>
Non-designers struggle to articulate the objective design needs of their projects, so design remains a single monolithic problem rather than being decomposed into bite-sized, introductory issues the way code problems are.</li>
Volunteer designers have a difficult time finding open source projects to get involved with.</li>
Non-designers don't know the difference between different types of design, and tend to bikeshed on superficial, obvious traits like colors when they should be focusing on more subtle parts of the user experience. We as non-designers are like clients who ask for a web site without knowing that there's a difference between frontend development, back end development, database administration, and systems administration.</li>
The tests which designers apply to their work are often almost impossible to automate. For instance, I gather that a lot of user interaction testing involves watching new users attempt to complete a task using a given design, and observing the challenges they encounter.</li>
</ul>
Again, if you know of an open source project that's overcome any of these challenges, please email me at design@edunham.net</code> and tell me about it!</p>


Linode vs AWS
2015-12-03T00:00:00+00:00
I'm examining a Linode account in order to figure out how to switch the application its instances are running to AWS. The first challenge is that instance types in the main dashboard are described by arbitrary numbers ("UI Name" in the chart below), rather than a statistic about their resources or pricing. Here's how those magic numbers line up to hourly rates and their corresponding monthly price caps:</p>



RAM</th>
Hourly $</th>
Monthly $</th>
UI Name</th>
Cores</th>
GB SSD</th>
</tr>
</thead>


1GB</td>
$0.015/hr</td>

$10/mo</p>
</blockquote></td>
1024</td>
1</td>
24</td>
</tr>

2GB</td>
$0.03/hr</td>

$20/mo</p>
</blockquote></td>
2048</td>
2</td>
48</td>
</tr>

4GB</td>
$0.06/hr</td>

$40/mo</p>
</blockquote></td>
4096</td>
4</td>
96</td>
</tr>

8GB</td>
$0.12/hr</td>

$80/mo</p>
</blockquote></td>
8192</td>
6</td>
192</td>
</tr>

16GB</td>
$0.24/hr</td>

$160/mo</p>
</blockquote></td>
16384</td>
8</td>
384</td>
</tr>

32GB</td>
$0.48/hr</td>

$320/mo</p>
</blockquote></td>
32768</td>
12</td>
768</td>
</tr>

48GB</td>
$0.72/hr</td>

$480/mo</p>
</blockquote></td>
49152</td>
16</td>
1152</td>
</tr>

64GB</td>
$0.96/hr</td>

$640/mo</p>
</blockquote></td>
65536</td>
20</td>
1536</td>
</tr>

96GB</td>
$1.44/hr</td>

$960/mo</p>
</blockquote></td>
98304</td>
20</td>
1920</td>
</tr>
</tbody>
</table>
AWS "Equivalents"</h1>
AWS T2 instances have burstable performance. M*</code> instances are general-purpose; C*</code> are compute-optimized; R*</code> are memory-optimized. *3</code> instances run on slightly older Ivy Bridge or Sandy Bridge processors, while *4</code> instances run on the newer Haswells. I'm disergarding the G2</code> (GPU-optimized), D2</code> (dense-storage), and I2</code> (IO-optmized) instance types from this analysis.</p>
Note that the AWS specs page has memory in GiB rather than GB. I've converted everything into GB in the following table, since the Linode specs are in GB and the AWS RAM amounts don't seem to follow any particular pattern that would lose information in the conversion.</p>
Hourly price is the Linux/UNIX rate for US West (Northern California) on 2015-12-03. Monthly price estimate is the hourly price multiplied by 730.</p>
Instance</th> vCPU</th> GB RAM</th> $/hr</th> $/month</th></tr></thead>

t2.micro</td> 1</td> 1.07</td> .017</td> 12.41</td></tr>
t2.small</td> 1</td> 2.14</td> .034</td> 24.82</td></tr>
t2.medium</td> 2</td> 4.29</td> .068</td> 49.64</td></tr>
t2.large</td> 2</td> 8.58</td> .136</td> 99.28</td></tr>
m4.large</td> 2</td> 8.58</td> .147</td> 107.31</td></tr>
m4.xlarge</td> 4</td> 17.18</td> .294</td> 214.62</td></tr>
m4.2xlarge</td> 8</td> 34.36</td> .588</td> 429.24</td></tr>
m4.4xlarge</td> 16</td> 68.72</td> 1.176</td> 858.48</td></tr>
m4.10xlarge</td> 40</td> 171.8</td> 2.94</td> 2146.2</td></tr>
m3.medium</td> 1</td> 4.02</td> .077</td> 56.21</td></tr>
m3.large</td> 2</td> 8.05</td> .154</td> 112.42</td></tr>
m3.xlarge</td> 4</td> 16.11</td> .308</td> 224.84</td></tr>
m3.2xlarge</td> 8</td> 32.21</td> .616</td> 449.68</td></tr>
c4.large</td> 2</td> 4.02</td> .138</td> 100.74</td></tr>
c4.xlarge</td> 4</td> 8.05</td> .276</td> 201.48</td></tr>
c4.2xlarge</td> 8</td> 16.11</td> .552</td> 402.96</td></tr>
c4.4xlarge</td> 16</td> 32.21</td> 1.104</td> 805.92</td></tr>
c4.8xlarge</td> 36</td> 64.42</td> 2.208</td> 1611.84</td></tr>
c3.large</td> 2</td> 4.02</td> .12</td> 87.6</td></tr>
c3.xlarge</td> 4</td> 8.05</td> .239</td> 174.47</td></tr>
c3.2xlarge</td> 8</td> 16.11</td> .478</td> 348.94</td></tr>
c3.4xlarge</td> 16</td> 32.21</td> .956</td> 697.88</td></tr>
c3.8xlarge</td> 32</td> 64.42</td> 1.912</td> 1395.76</td></tr>
r3.large</td> 2</td> 16.37</td> .195</td> 142.35</td></tr>
r3.xlarge</td> 4</td> 32.75</td> .39</td> 284.7</td></tr>
r3.2xlarge</td> 8</td> 65.50</td> .78</td> 569.4</td></tr>
r3.4xlarge</td> 16</td> 131</td> 1.56</td> 1138.8</td></tr>
r3.8xlarge</td> 32</td> 262</td> 3.12</td> 2277.6</td></tr>
</tbody></table>
Comparison</h1>
Linode and AWS do not compare cleanly at all. The smallest AWS instance to match a given Linode type's RAM typically has fewer vCPUs and costs more in the region where I compared them. Conversely, the smallest AWS instance to match a Linode type's number of cores often has almost double the RAM of the Linode, and costs substantially more.</p>
Switching from Linode to AWS</h1>
When I examine the Servo build machines' utilization graphs via the Linode dashboard, it becomes clear that even their load spikes aren't fully utilizing the available CPUs. To view memory usage stats on Linode, it's necessary to configure hosts to run the longview</a> client. After installation, the client begins reporting data to Linode immediately.</p>
After a few days, these metrics can be used to find the smallest AWS instance whose specs exceed what your application is actually using on Linode.</p>
Sources:</p>

linode specs</a></li>
linode pricing</a></li>
AWS specs</a></li>
</ul>


Giving Thanks to Rust Contributors
2015-11-25T00:00:00+00:00
It's the day before Thanksgiving here in the US, and the time of year when we're culturally conditioned to be a bit more public than usual in giving thanks for things.</p>
As always, I'm grateful that I'm working in tech right now, because almost any job in the tech industry is enough to fulfill all of one's tangible needs like food and shelter and new toys. However, plenty of my peers have all those material needs met and yet still feel unsatisfied with the impact of their work. I'm grateful to be involved with the Rust project because I know that my work makes a difference to a project that I care about.</p>
Rust is satisfying to be involved with because it makes a difference, but that would not be true without its community. To say thank you, I've put together a little visualization for insight into one facet of how that community works its magic:</p>
</p>
The stats page is interactive and available at http://edunham.github.io/rust-org-stats/</a>. The pretty graphs take a moment to render, since they're built in your browser.</p>
There's a whole lot of data on that page, and you can scroll down for a list of all authors. It's especially great to see the high impact that the month's new contributors have had, as shown in the group comparison at the bottom of the "natural log of commits" chart!</p>
It's made with the little toy I wrote a while ago called orglog</a>, which builds on gitstat</a> to help visualize how many people contribute code to a GitHub organization. It's deployed to GitHub Pages with TravisCI (eww</a>) and nightli.es</a> so that the Rust's organization-wide contributor stats will be automatically rebuilt and updated every day.</p>
If you'd like to help improve the page, you can contribute to gitstat</a> or orglog</a>!</p>


PSA: Docker on Ubuntu
2015-11-23T00:00:00+00:00
$ sudo apt-get install docker
</span>$ which docker
</span>$ docker
</span>The program 'docker' is currently not installed. You can install it by typing: 
</span>apt-get install docker
</span>$ apt-get install docker
</span>Reading package lists... Done
</span>Building dependency tree       
</span>Reading state information... Done
</span>docker is already the newest version.
</span>0 upgraded, 0 newly installed, 0 to remove and 13 not upgraded.
</span></code></pre>
Oh, you wanted to run a docker container? The docker</code> package in Ubuntu is some window manager dock thingy. The docker</code> binary that runs containers comes from the docker.io</code> system package.</p>
$ sudo apt-get install docker.io
</span>$ which docker
</span>/usr/bin/docker
</span></code></pre>
Also, if it can't connect to its socket:</p>
FATA[0000] Post http:///var/run/docker.sock/v1.18/containers/create: dial
</span>unix /var/run/docker.sock: permission denied. Are you trying to connect to a
</span>TLS-enabled daemon without TLS? 
</span></code></pre>
you need to make sure you're in the right group:</p>
sudo usermod -aG docker <username>; newgrp docker
</span></code></pre>
(thanks, stackoverflow</a>!)</p>


Installing Rust without root
2015-11-17T00:00:00+00:00
I just got a good question from a friend on IRC: "Should I ask my university's administration to install Rust on our shared servers?" The answer is "you don't have to".</p>
Pick one of the two following sets of directions. I'd recommend using Multirust</a>, because it automatically checks the packages it downloads and lets you switch between Rust versions trivially.</p>
Without multirust</h1>
If you just want one version of Rust, this blog post</a> by Valérian Galliat has a fix in 7 lines:</p>
cd ~/.rust
</span>wget https://static.rust-lang.org/dist/rust-nightly-x86_64-unknown-linux-gnu.tar.gz
</span>tar xf rust-nightly-x86_64-unknown-linux-gnu.tar.gz
</span>mv rust-nightly-x86_64-unknown-linux-gnu rust
</span>export LD_LIBRARY_PATH=~/opt/rust/rustc/lib:$LD_LIBRARY_PATH
</span>export PATH=~/.rust/rust/rustc/bin:$PATH
</span>export PATH=~/.rust/rust/cargo/bin:$PATH
</span></code></pre>
If you want rust stable instead of rust nightly, use the URL https://static.rust-lang.org/dist/rust-stable-x86_64-unknown-linux-gnu.tar.gz</code> in the wget</code> step to download the latest stable release.</p>
If you're security-conscious, you might want to verify the integrity of the tarball before inflating it and running its contents. We provide a GPG signature of every tarball, and sha256</code> sums of the tarballs and signatures.</p>
You can construct the URL for shasum or GPG signature by adding the desired extension to the tarball's URL, so for nightly:</p>
https://static.rust-lang.org/dist/rust-nightly-x86_64-unknown-linux-gnu.tar.gz
</span>https://static.rust-lang.org/dist/rust-nightly-x86_64-unknown-linux-gnu.tar.gz.sha256
</span>https://static.rust-lang.org/dist/rust-nightly-x86_64-unknown-linux-gnu.tar.gz.asc
</span>https://static.rust-lang.org/dist/rust-nightly-x86_64-unknown-linux-gnu.tar.gz.asc.sha256
</span></code></pre>
To verify the GPG signature</a>, you'll also need a copy of the Rust project's public key. This key is available through several channels:</p>

on the Rust website</a>, available only over HTTPS.</li>
on keybase.io</a>, correlated to Rust's Twitter account and URL. Don't worry, we authenticated the key by signing a string from Keybase with it locally. We don't trust them to ever see our private key.</li>
on GitHub</a>, in the website's repository.</li>
</ul>
Remember, verifying the signature only guarantees that the tarball you downloaded matches the one that was produced by the Rust project's build infrastructure. As with any piece of software, there exist a variety of threat models from which verifying the signatures cannot completely protect you.</p>
Multirust without root</h1>
Multirust</a> is a tool that makes it easy to use multiple Rust versions on the same system. Although the absolute easiest way to use it is curl -sf https://raw.githubusercontent.com/brson/multirust/master/blastoff.sh | sh</code> (which will interactively request a sudo password partway through), it can be installed without root as well:</p>
git clone --recursive https://github.com/brson/multirust && cd multirust
</span>./build.sh # create install.sh
</span>mkdir ~/.rust
</span>./install.sh --prefix=~/.rust/
</span>echo "PATH=~/.rust/bin:$PATH" >> ~/.bashrc; source ~/.bashrc
</span></code></pre>
If you run into an error like:</p>
install: WARNING: failed to run ldconfig. this may happen when not installing
</span>as root. run with --verbose to see the error
</span></code></pre>
or in verbose mode,:</p>
install: running ldconfig
</span>/sbin/ldconfig.real: Can't create temporary cache file /etc/ld.so.cache~:
</span>Permission denied
</span>install: WARNING: failed to run ldconfig. this may happen when not installing
</span>as root. run with --verbose to see the error
</span></code></pre>
It means you don't have permissions to write to /etc/ld.so.cache</code>. Until this issue</a> gets fixed, the easiest workaround to lacking those permissions is to change the script called by the installer to pass -C</code> to ldconfig</code>:</p>
sed -i 's/   ldconfig/   ldconfig -C ~\/.rust\/ld.so.cache/' build/work/multirust-0.7.0/install.sh
</span></code></pre>
Then you should be able to ./install.sh --prefix=~/.rust</code> without the prior warning. Nasty hack, but the easiest way to get it working today.</p>
This technically breaks rustc</code> (since it's dynamically linked), but if you're building a Rust project or library, you'll be using the statically linked cargo</code> tool and thus won't be affected.</p>
By the way, this is an example of why people who write system utilities like ldconfig</code> should make them able to read their serttings out of environment variables as well as just command-line arguments.</p>
Now you can multirust default nightly</code> to install rust-nightly and configure it as the default, and you're ready to roll!</p>
Testing your Rust installation</h1>
You can now make a package that says "Hello World" in just 5 commands, using a workflow that will scale to packaging and distributing larger projects:</p>
cargo new hello --bin
</span>echo "fn main(){println!(\"Hello World\");}" > hello/src/main.rs
</span>cd hello
</span>cargo build
</span>cargo run
</span></code></pre>
Congratulations, you're running Rust!</p>


Multiple languages on TravisCI
2015-11-12T00:00:00+00:00
Today I noticed an assumption which was making my life unnecessarily difficult: I assumed that if my .travis.yml</code> said language: ruby</code> on the first line, I was supposed to only run Ruby code from it.</p>
Travis lets you run code much more arbitrary than that.</p>
I did a bunch of tests on a toy repo</a> to see what would happen if I ignored my preconceptions about how you can and can't test stuff, and learned some interesting things:</p>

You can install PyPI packages in a test suite that's technically Ruby, or gems in a test suite that's technically Python.</li>
If your project is language:ruby</code>, you need to sudo pip install</code> dependencies. If it's language:python</code>, you can just gem install</code> dependencies without sudo</code>.</li>
If I specify multiple instances of language:</code> or multiple build matrices, Travis uses the language whose build matrix occurs last. If I specify a Python matrix and then a Ruby one, the Ruby matrix will be run.</li>
</ul>
This is especially useful when testing or deployment requires hitting an API whose libraries are most up to date in a language other than that of the project.</p>


Beyond Openhatch
2015-11-04T00:00:00+00:00
Update: I'm now maintaining the issue aggregator list at http://edunham.net/pages/issue_aggregators.html</a></p>
OpenHatch</a> is a wonderful place to help new contributors find their first open source issues to work on. Their training materials are unparalleled, and the "projects submit easy bugs with mentors" model makes their list of introductory issues reliably high-quality.</p>
However, once you know the basics of how to engage with an open source project, you're no longer in the target audience for OpenHatch's list. Where should you look for introductory issues when you want to get involved with a new project, but you're already familiar with open source in general?</p>
An excellent slide deck</a> by Josh Matthews contains several answers to this question:</p>

issuehub.io</a> scrapes GitHub by labels and language</li>
up-for-grabs</a> has an opt-in list of projects looking for new contributors, and scrapes their issue trackers for their "jump in", "up for grabs" or other "new contributors welcome" tags.</li>
If you're looking for Mozilla-specific contributions outside of just code, What can I do for Mozilla?</a> can help direct you into any of Mozilla's myriad opportunities for involvement.</li>
</ul>
Additionally, the servo-starters</a> page has a custom view of easy issues sorted by Servo's project-specific tags.</p>
GitHub Tricks</h1>
If you're looking for open issues across all repos owned by a particular user or organization, you can use the search at https://github.com/pulls</a> and specify the "user" (or org) in the search bar. For instance, this search</a> will find all the unassigned, easy-tagged issues in the rust-lang</code> org. Breaking down the search:</p>

user:rust-lang</code> searches all repos owned by github.com/rust-lang</code>. It could also be someone's github username.</li>
is:open</code> searches only open issues.</li>
no:assignee</code> will filter out the issues which are obviously claimed. Note that some issues without an assignee set may still have a comment saying "I'll do this!", if it was claimed by a user who did not have permissions to set assignees and then not triaged.</li>
label:E-Easy</code> uses my prior knowledge that most repos within rust-lang</code> annotate introductory bugs with the E-easy</code> tag. When in doubt, check the contributing.md</code> file at the top level in the org's most popular repository for an explanation of what various issue labels mean. If that information isn't in the contributing file or the README, file a bug!</li>
</ul>
Am I missing your favorite introductory issue aggregator? Shoot me an email to ___@edunham.net</code> (fill in the blank with anything; the email will get to me) with a link, and I'll add it here if it looks good!</p>


PSA: Pin Versions
2015-10-29T00:00:00+00:00
Today, the website's build broke</a>. We made no changes to the tests, yet a wild dependency error emerged:</p>
Generating... 
</span>
</span>  Dependency Error: Yikes! It looks like you don't have redcarpet or one of
</span>its dependencies installed. In order to use Jekyll as currently configured,
</span>you'll need to install this gem. The full error message from Ruby is: 'cannot
</span>load such file -- redcarpet' If you run into trouble, you can find helpful
</span>resources at http://jekyllrb.com/help/! 
</span>
</span>  Conversion error: Jekyll::Converters::Markdown encountered an error while
</span>converting 'conduct.md':
</span>
</span>                    redcarpet
</span>
</span>             ERROR: YOUR SITE COULD NOT BE BUILT:
</span>
</span>                    ------------------------------------
</span>
</span>                    redcarpet
</span>
</span>The command "jekyll build" exited with 1.
</span></code></pre>
Although Googling the error was unhelpful, a bit more digging revealed that our last working build had been on Jekyll 2.5.3</code> and the builds breaking on a Redcarpet error all used 3.0.0</code>.</p>
The moral of the story is that where the .travis.yml</code> said - gem install jekyll</code>, it should have said - gem install jekyll -v 2.5.3</code>.</p>


SeaGL 2015 Retrospective
2015-10-25T00:00:00+00:00
As well as nominally helping organize the event, I attended and spoke at SeaGL</a> 2015 this weekend. The slides from my talk are here</a>.</p>
My talk drew an audience of perhaps a dozen people on Friday afternoon. I didn't record this instance of the talk, but will probably give it at least one more time and be sure to record then.</p>
One of the more useful tools I learned about is called myrepos</a>. It lets you update all of the Git repositories on a machine at the same time, as well as other neat tricks like replaying actions that failed due to network problems. Its author</a> has written a variety of other useful Git wrappers, as well.</p>
Additionally, VCSH</a> seems to be the "I knew somebody else wrote that already!" tool for keeping parts of a home directory in Git.</p>


Upgrading Buildbot 0.8.6 to 0.8.12
2015-10-14T00:00:00+00:00
Here are some quick notes on upgrading Buildbot.</p>
System Dependencies</h1>
There are more now. In order to successfully install all of Buildbot's dependencies with Pip, I needed a few more apt packages:</p>
python-dev
</span>python-openssl
</span>libffi-dev
</span>libssl-dev
</span></code></pre>
Then for sanity's sake make a virtualenv, and install the following packages. Note that having too new a sqlalchemy</code> will break things</a>.:</p>
buildbot==0.8.12
</span>boto
</span>pyopenssl
</span>cryptography
</span>SQLAlchemy<=0.7.10
</span></code></pre>
Virtualenvs</h1>
Troubleshooting compatibility issues with system packages on a host that runs several Python services with various dependency versions is predictably terrible.</p>
The potential problem with switching to running Buildbot only from a virtualenv is that developers with access to the buildmaster might want to restart it and miss the extra step of activating the virtualenv. I addressed this by adding the command to activate the virtualenv (using the virtualenv's absolute path) to the ~/.bashrc</code> of the user that we run Buildbot as. This way, we've gained the benefits of having our dependencies consolidated without adding the cost of an extra workflow step to remember.</p>
Template changes</h1>
Most of Buildbot's status pages worked fine after the upgrade, but the console view threw a template error because it couldn't find any variable named "categories". The fix was to simply copy the new template from venv/local/lib/python2.7/site-packages/buildbot/status/web/templates/console.html</code> to my-buildbot/master/templates/console.html</code>.</p>
That's it!</h1>
Rust currently has these updates on the development buildmaster, but not yet (as of 10/14/2015) in prod.</p>


Carrying credentials between environments
2015-09-29T00:00:00+00:00
This scenario is simplified for purposes of demonstration.</p>
I have 3 machines: A</code>, B</code>, and C</code>. A</code> is my laptop, B</code> is a bastion, and C</code> is a server that I only access through the bastion.</p>
I use an SSH keypair helpfully named AB</code> to get from me@A</code> to me@B</code>. On B</code>, I su</code> to user</code>. I then use an SSH keypair named BC</code> to get from user@B</code> to user@C</code>.</p>
I do not wish to store the BC</code> private key on host B</code>.</p>
SSH Agent Forwarding</h1>
I have keys AB</code> and BC</code> on host A</code>, where I start. Host A</code> is running ssh-agent</code>, which is installed by default on most Linux distributions. :</p>
me@A$ ssh-add ~/.ssh/AB     # Add keypair AB to ssh-agent's keychain
</span>me@A$ ssh-add ~/.ssh/BC     # Add keypair BC to the keychain
</span>me@A$ ssh -A me@B           # Forward my ssh-agent 
</span></code></pre>
Now I'm logged into host B</code> and have access to the AB</code> and BC</code> keypairs. An attacker who gains access to B</code> after I log out will have no way to steal the BC</code> keypair, unlike what would happen if that keypair was stored on B</code>.</p>
See here</a> for pretty pictures explaining in more detail how agent forwarding works.</p>
Anyways, I could now ssh me@C</code> with no problem. But if I sudo su user</code>, my agent is no longer forwarded, so I can't then use the key that I added back on A</code>!</p>
Switch user while preserving environment variables</h1>
me@B$ sudo -E su user
</span>user@B$ sudo -E ssh user@C
</span></code></pre>
What?</h1>
The -E</code> flag to sudo preserves the environment variables of the user you're logged in as. ssh-agent</code> uses a socket whose name is of the form /tmp/ssh-AbCdE/agent.12345</code> to call back to host A</code> when it's time to do the handshake involving key BC</code>, and the socket's name is stored in me</code>'s SSH_AUTH_SOCK</code> environment variable. So by telling sudo to preserve environment variables when switching user, we allow user</code> to pass ssh handshake stuff back to A</code>, where the BC</code> key is available.</p>
Why is sudo -E</code> required to ssh to C</code>? Because /tmp/sshAbCdE/agent.12345</code> is owned by me:me</code>, and only the file's owner may read, write, or execute it. Additionally, the socket itself (agent.12345</code>) is owned by me:me</code>, and is not writable by others.</p>
If you must run ssh on B</code> without sudo, chown -R /tmp/ssh-AbCdE</code> to the user who needs to end up using the socket. Making them world read/writable would allow any user on the system to use any key currently added to the ssh-agent</code> on A</code>, which is a terrible idea.</p>
For what it's worth, the actual value of /tmp/ssh-AbCdE/agent.12345</code> is available at any time in this workflow as the result of printenv | grep SSH_AUTH_SOCK | cut -f2 -d =</code>.</p>
The Catch</h1>
Did you see what just happened there? An arbitrary user with sudo on B</code> just gained access to all the keys added to ssh-agent on A</code>. Simon</a> pointed out that the right way address this issue is to use ProxyCommand</a> instead of agent forwarding.</p>
No, I really</em> don't want my keys accessible on B</code></h1>
See man ssh_config</code> for more of the details on ProxyCommand</code>. In ~/.ssh/config</code> on A</code>, I can put:</p>
Host B
</span>    User me
</span>    Hostname 111.222.333.444
</span>
</span>Host C
</span>    User user
</span>    Hostname 222.333.444.555
</span>    Port 2222
</span>    ProxyCommand ssh -q -w %h:%p B
</span></code></pre>
So then, on A</code>, I can ssh C</code> and be forwarded through B</code> transparently.</p>


Ansible: Conditional role dependencies
2015-09-10T00:00:00+00:00
I've recently been working on an Ansible role that applies to both Ubuntu and OSX hosts. It has some dependencies which are only needed on OSX. There doesn't seem to be a central document on all the options available for solving this problem, so here are my notes.</p>
Scenario</h1>
The role which must apply to both Ubuntu and OSX hosts builds a Rust compiler capable of cross-compiling to Android, so I call it crosscompiler</code>. To run the crosscompiler</code> role on a Mac, you need the xcode</code> role installed, but applying the xcode</code> role to an Ubuntu host will fail.</p>
A simplified version of this setup looks like:</p>
ansible-configs/
</span>├── galaxy_roles.yaml
</span>├── hosts
</span>├── roles
</span>│   ├── crosscompiler
</span>│   │   ├── defaults
</span>│   │   │   └── main.yaml
</span>│   │   ├── meta
</span>│   │   │   └── main.yaml
</span>│   │   └── tasks
</span>│   │       └── main.yaml
</span>│   └── xcode
</span>│       ├── defaults
</span>│       │   └── main.yaml
</span>│       ├── meta
</span>│       │   └── main.yaml
</span>│       └── tasks
</span>│           └── main.yaml
</span>└── site.yaml
</span></code></pre>
Here are a bunch of different ways to avoid applying a Mac-specific task to an Ubuntu host, or vice versa. Note that any</strong> of the following steps in isolation will solve the problem -- it should not be necessary to use more than one of them.</p>
Check OS on each task of the role</h1>
Add the line when: ansible_os_family == 'Darwin'</code> at the end of each task in roles/xcode/tasks/main.yaml</code>.</p>
This needlessly bloats the code and makes it more difficult to read.</p>
Refactor depended role to ignore non-target platforms</h1>
Move the entire contents of roles/xcode/main.yaml</code> into roles/xcode/osx.yaml</code>, then create a new main.yaml</code> containing:</p>
---
</span>- include: osx.yaml
</span>  when: ansible_os_family == 'Darwin'
</span></code></pre>
This avoids the bloat induced by running the conditional on each task, while accomplishing the same goal. Now the xcode</code> role looks like:</p>
xcode
</span>├── defaults
</span>│   └── main.yaml
</span>├── meta
</span>│   └── main.yaml
</span>└── tasks
</span>    ├── main.yaml
</span>    └── osx.yaml
</span></code></pre>
This is the best solution for a role which might later expand to support additional platforms.</p>
Make the dependency conditional in meta/main.yaml</code> of depending role</h1>
Edit ansible-configs/roles/crosscompiler/main.yaml</code> so that the dependency on xcode</code> reads:</p>
---
</span>dependencies:
</span>  - { role: 'xcode', when: ansible_os_family == 'Darwin' }
</span></code></pre>
This is the best solution when the inner role will only ever target one platform, as is the case with xcode</code>.</p>
Install role conditionally from site.yaml</h1>
Edit ansible-configs/site.yaml</code> to read:</p>
- name: Provision cross-compile hosts
</span>  hosts: xcompilehosts
</span>  roles:
</span>    - { role: xcode, when: ansible_os_family == 'Darwin' }
</span>    - crosscompiler
</span></code></pre>
This is problematic because if I was to distribute the crosscompiler</code> role on the Ansible Galaxy, its dependency logic would not be distributed to other users correctly.</p>
TL;DR</h1>
You can conditionally include dependencies in your roles. It's helpful to end users when galaxy roles only try to apply platform-specific tasks to their target platforms, since you can't be sure how others will use your code.</p>


Apache Licenses
2015-08-28T00:00:00+00:00
At the bottom of the Apache 2.0 License file, there's an appendix:</p>
APPENDIX: How to apply the Apache License to your work.
</span>
</span>...
</span>
</span>Copyright [yyyy] [name of copyright owner]
</span>
</span>...
</span></code></pre>
Does that look like an invitation to fill in the blanks to you? It sure does to me, and has for</a> others</a> in the Rust community as well.</p>
Today I was doing some licensing housekeeping and made the same embarrassing</a> mistake</a>.</p>
This is a PSA to double check whether inviting blanks are part of the appendix before filling them out in Apache license texts.</p>


X240 trackpoint speed
2015-08-24T00:00:00+00:00
The screen on my X1 Carbon gave out after a couple months, and my loaner laptop in the meantime is an X240.</p>
The worst thing about this laptop is how slowly the trackpoint moves with a default Ubuntu installation. However, it's fixable:</p>
cat /sys/devices/platform/i8042/serio1/serio2/speed
</span>cat /sys/devices/platform/i8042/serio1/serio2/sensitivity
</span></code></pre>
Note the starting values in case anything goes wrong, then fiddle around:</p>
echo 255 | sudo tee /sys/devices/platform/i8042/serio1/serio2/sensitivity
</span>echo 255 | sudo tee /sys/devices/platform/i8042/serio1/serio2/speed
</span></code></pre>
Some binary search themed prodding and a lot of tee: /sys/devices/platform/i8042/serio1/serio2/sensitivity: Numerical result out of range</code> has confirmed that both files accept values between 0-255. Interestingly, setting them to 0 does not seem to disable the trackpoint completely.</p>
If you're wondering why the configuration settings look like ordinary files but choke on values bigger or smaller than a short, go read about sysfs</a>.</p>


Folklore and fallacy
2015-08-17T00:00:00+00:00
I was a student employee at the OSU Open Source Lab, on and off between internships and other jobs, for 4 years. Being part of the lab helped shape my life and career, in almost overwhelmingly positive ways. However, the farther I get from the lab the more clearly I notice how being part of it changed the way I form expectations about my own technical skills.</p>
To show you the fallacy that I noticed myself falling into, I'd like to tell you a completely made-up story about some alphabetically named kangaroos. Below the fold, there'll be pictures!</p>
</p>
Once upon a time, some kangaroos lived in a desert. One day, for some inscrutable marsupial reason, a bunch of young kangaroos got together to practice jumping. Since they're not particularly creative creatures, they called the group jumping school.</p>
</p>
Every kangaroo who came to jumping school started out only being able to jump 1 foot, and got better at a rate of 1 foot per year, and stayed for 4 years.</p>
A kangaroo called Aggie was one of the school's first students. She came in only able to jump 1 foot, but she improved by 1 foot per year.</p>
</p>
At the start of the second year that the school was around, a kangaroo named Bill joined. Bill could only jump 1 foot when he started, and improved at a rate of 1 foot per year. But Aggie could always jump 2 feet farther than Bill while she was still in school, because they were both improving at the same rate.</p>
</p>
Nobody new joined for a while, and Aggie left after her fourth year to go cross roads in front of unwary motorists, but Bill stayed in school and kept improving. When she left school, Aggie could jump 5 feet. She knew she'd worked hard, and could always jump farther than Bill, so she felt pretty good about herself.</p>
At the start of the school's fourth year, after Aggie had left, a new student named Claire joined. Claire could only jump 1 foot at first, but improved at a rate of 1 foot per year.</p>
</p>
Claire and Bill would chat about school sometimes, and Claire observed that Bill could always jump 2 feet farther than her. When she commented on it, Bill said "If you think I can jump far, you should have seen Aggie! She was a student here before you came, and she could always jump 2 feet farther than me!".</p>
At the end of the school's 6th year, Bill finished up and went away to fight in boxing matches. (When Bill left, he could jump 5 feet. He always suspected he could have done better, since he remembered Aggie always being able to jump just a bit farther than him.)</p>
A new student, Dave, joined after Bill left. Dave started out being able to only jump 1 foot and was able to jump 5 feet by the time he'd been at the school for 4 years. Dave knew that Claire could always jump 2 feet farther than him while they were in school together. Dave heard stories from Claire about Bill and Aggie, who could both jump even farther than her!</p>
</p>
2 years after Dave joined, Claire left jumping school for a full-time job tearing up farmers' crops and gardens. She knew she'd tried her best, and she could jump 5 feet after she'd been at school for 4 years, but she also knew that Bill had always been a better jumper than her and Aggie had been even better than Bill. Her 5 feet didn't seem particularly impressive, since (she even double-checked her math on it!) Aggie would have been able to jump 4 feet farther, so that must have meant Aggie was a student who could jump 9 feet.</p>
A couple of years later, Dave was finally finished with school. He'd come in only being able to jump 1 foot, and left being able to jump 5 feet! But he wasn't sure if this was better or worse than normal, so he thought about the other kangaroos who'd also gone to the school.</p>
</p>
Dave knew that Claire was a student, and she was able to jump 2 feet farther than him whenever they studied together. Bill was a student who could jump 2 feet farther than Claire, and Aggie was a student who could jump 2 feet farther than Bill! Since Dave could jump 5 feet, he concluded that Claire could jump 7 feet, Bill could jump 9, and Aggie could jump 11! Not only was Dave the worst of the lot, he wasn't even half as good as the school's first student! He felt pretty bad about his accomplishments, and wondered why students were getting worse every year.</p>
Epilogue</h1>
</p>
Once upon a somewhat later time, Jumping School had a reunion and all of the former students attended. Claire and Dave were a little afraid of meeting Aggie, since they'd heard such impressive stories of how far she could jump. All 4 kangaroos tested how far they could jump, just for old times' sake, and they found that they could all jump about 6 feet! They compared stories about their experiences since leaving school, and found that their rates of improvement had slowed down as they got closer to the limit of how far their species was able to jump.</p>
In reality, red kangaroos only jump about 1.8 meters</a>, which is the only factually accurate part of this entire story.</p>
The Moral</h1>
I think that a similar effect distorted my perception of my own competence when I compared myself to past OSL students. One can spot the fallacy pretty easily when everything is spelled out with cute photos: Relative skill levels don't translate reliably into absolute ones over time. It's tricker to spot the same fallacy in real life, but you might have an easier time now that you've seen the pattern once before.</p>
Photo credits, in order:</p>

https://en.wikipedia.org/wiki/File:Kangaroo_Island_kangaroos.jpg</a></li>
https://www.flickr.com/photos/chrissamuel/5366854530</a></li>
https://commons.wikimedia.org/wiki/File:Flying-kangaroo.jpg</a></li>
https://www.flickr.com/photos/kaptainkobold/3007004396</a></li>
https://en.wikipedia.org/wiki/File:-_fighting_red_kangaroos_1.jpg</a></li>
http://opencage.info/pics/files/800_11526.jpg</a></li>
https://www.flickr.com/photos/moonfish/3147269483</a></li>
https://www.flickr.com/photos/chrissamuel/5480184535</a></li>
</ul>


RustCamp videos are available
2015-08-17T00:00:00+00:00
The videos from RustCamp are available here</a>.</p>
I asked Gankro what was up with the milkshake thing in his talk</a>, and learned about this meme</a>.</p>


Don't Starve
2015-08-09T00:00:00+00:00
It was a lazy Sunday afternoon and I wanted to play Don't Starve. This actually ended up meaning about 3 hours of intermittent troubleshooting and 1 hour of games, because Linux.</p>
Get the files</h1>
I bought Don't Starve from the Humble Bundle store, although there are other methods of obtaining it which strike a different balance between cost and convenience.</p>
The downloaded file is dontstarve_x64_july21.tar.gz</code>.</p>
This Just Works</h2>
$ yaourt -S libcurl-compat
</span>$ tar -xvf dontstarve_x64_july21.tar.gz
</span>$ cd dontstarve/bin
</span>$ LD_PRELOAD=libcurl.so.3 ./dontstarve
</span></code></pre>
Below the fold is the troubleshooting process I went through to make it look so easy. Hopefully it'll be of assistance to those searching for the errors that I ran into!</p>
This part has SEO for all the intermediate errors</h2>
Try to run the script</h1>
$ cd dontstarve
</span>$ ./dontstarve.sh
</span>sh: symbol lookup error: sh: undefined symbol: rl_signal_event_hook
</span>sh: symbol lookup error: sh: undefined symbol: rl_signal_event_hook
</span>sh: symbol lookup error: sh: undefined symbol: rl_signal_event_hook
</span>sh: symbol lookup error: sh: undefined symbol: rl_signal_event_hook
</span>sh: symbol lookup error: sh: undefined symbol: rl_signal_event_hook
</span>Fontconfig error: "/etc/fonts/conf.d/10-scale-bitmap-fonts.conf", line 70: non-double matrix element
</span>Fontconfig error: "/etc/fonts/conf.d/10-scale-bitmap-fonts.conf", line 70: non-double matrix element
</span>Fontconfig warning: "/etc/fonts/conf.d/10-scale-bitmap-fonts.conf", line 78: saw unknown, expected number
</span>
</span>(updater:2135): Pango-WARNING **: failed to choose a font, expect ugly output. engine-type='PangoRenderFc', script='latin'
</span>
</span>(updater:2135): Pango-WARNING **: failed to choose a font, expect ugly output. engine-type='PangoRenderFc', script='common'
</span>./dontstarve.sh: line 2: unexpected EOF while looking for matching ``'
</span>./dontstarve.sh: line 4: syntax error: unexpected end of file
</span></code></pre>
Look what doesn't like me! The fonts are sad. Spoiler: You don't actually need those fonts at all.</p>
Much Frustration</h1>
It turns out that the ./dontstarve.sh</code> in the root of the unzipped dontstarve</code> directory is solely an updater, which is totally irrelevant</strong> if you just want to play whatever version of the game they happened to ship you. It launches an updater that requests a game key and has hideously broken fonts and generally sprouts new errors like a Hydra every time you think you're making progress.</p>
Try running the correct dontstarve</code> executable</h1>
The correct script to run actually lives in bin/dontstarve.sh</code>, but if you try to run it from outside that directory, it can't find other files:</p>
$ bin/dontstarve.sh
</span>$ bin/dontstarve.sh: line 3: ./dontstarve: No such file or directory
</span></code></pre>
Cute. So instead:</p>
$ cd bin
</span>$ ./dontstarve.sh
</span>./dontstarve: /usr/lib/libcurl.so.4: version `CURL_OPENSSL_3' not found (required by ./dontstarve)
</span></code></pre>
OR:</p>
$ bin/dontstarve # Spoiler, this will still be broken later
</span>bin/dontstarve: /usr/lib/libcurl.so.4: version `CURL_OPENSSL_3' not found (required by bin/dontstarve)
</span></code></pre>
Fixing the libcurl</code> error</h1>
Get libcurl-compat</code> from the AUR:</p>
$ yaourt -S libcurl-compat
</span></code></pre>
At the end of its installation process, it'll give you a helpful little warning about preloading the library:</p>
Sometimes you have to preload library!
</span> e.g. if you see this message:
</span> /usr/lib/libcurl.so.4: version `CURL_OPENSSL_3' not found'
</span> Do this:
</span> LD_PRELOAD=libcurl.so.3 youprogname 
</span></code></pre>
Run dontstarve</code> with the right libcurl</code></h1>
$ LD_PRELOAD=libcurl.so.3 bin/dontstarve
</span></code></pre>
Hello Segfaults</h1>
And then we get a nice reproduceable segfault, ending in:</p>
ERROR: Missing Shader 'shaders/font.ksh'.
</span>Assert failure '0' at ../source/renderlib/OpenGL/HWEffect.cpp(86)
</span>
</span>Assert failure 'BREAKPT:' at ../source/renderlib/OpenGL/HWEffect.cpp(86)
</span>
</span>Assert failure 'datasize + mReadHead <= mBufferLength' at ../source/util/reader.h(28)
</span>
</span>Assert failure 'BREAKPT:' at ../source/util/reader.h(28)
</span>
</span>Segmentation fault (core dumped)
</span></code></pre>
This is NOT</strong> the cue to go shave a graphics card yak, despite what Googling the error would lead one to believe. This is the cue to:</p>
$ cd bin
</span>$ LD_PRELOAD=libcurl.so.3 ./dontstarve
</span></code></pre>
If it's stupid, but it works, it ain't stupid. Running the executable from the right directory makes the game work on an X230 and on an X1 Carbon, so in my (un)professional opinion, it's got nothing to do with special fancy graphics drivers.</p>
It works!</h1>
Scroll way back up to the top for the short version. Have fun, and Don't Starve!</p>
P.S.</h1>
I tried this with the x32 version. It goes all:</p>
~/Downloads/dontstarve/bin $ LD_PRELOAD=libcurl.so.3 ./dontstarve
</span>bash: ./dontstarve: No such file or directory
</span></code></pre>
The script and executable have the same permissions when unzipped from the 32-or 64-bit tarballs... Just, the 32-bit one doesn't work. There's probably a good reason for this, possibly related to the fact that I'm running on a 64-bit system, but since the x64</code> variant of the game runs just fine I didn't dig into the x32</code>'s malfunctions any deeper.</p>


How many Rust channels are there?
2015-07-31T00:00:00+00:00
I'm using search.mibbit.com</a> to count these. All have at least one user in them as of 4pm PST 2015-07-31.</p>
There are 53</strong> Rust-related channels on irc.mozilla.org</code>.</p>
List below the fold.</p>
41 General and Project channels:</p>
##rustfmt
</span>#cargo
</span>#hematite (Minecraft-in-Rust)
</span>#hyper (an HTTP library, https://github.com/hyperium/hyper)
</span>#iron (a web framework, https://github.com/iron/iron)
</span>#mio
</span>#rust
</span>#rust-api
</span>#rust-apidesign
</span>#rust-audio
</span>#rust-bikeshed
</span>#rust-bots
</span>#rust-casino
</span>#rust-community
</span>#rust-config
</span>#rust-crypto
</span>#rust-data
</span>#rust-design
</span>#rust-dev
</span>#rust-diverse
</span>#rust-fsnotify
</span>#rust-gamedev
</span>#rust-internals
</span>#rust-lang
</span>#rust-learners
</span>#rust-libs
</span>#rust-music
</span>#rust-newspeak
</span>#rust-osdev
</span>#rust-politics
</span>#rust-tls
</span>#rust-tools
</span>#rust-triage
</span>#rust-tty
</span>#rust-war
</span>#rust-webdev
</span>#rust-workshop
</span>#rust-ww
</span>#rust_offtopic
</span>#rustaudio
</span>#servo
</span>#winapi
</span></code></pre>
3 social channels:</p>
#rust-chat
</span>#rust-offtopic
</span>#rust-offtopic-offtopic
</span></code></pre>
9 apparently language- or location-specific channels:</p>
#rust-br
</span>#rust-de
</span>#rust-fr
</span>#rust-hu
</span>#rust-learners-de
</span>#rust-nyc
</span>#rust-ru
</span>#rust-seattle
</span>#rust.fi
</span></code></pre>


Good times
2015-07-28T00:00:00+00:00
People sometimes say "morning" or "evening" on IRC for a time zone unlike my own. Here's a bash one-liner that emits the correct time-of-day generalization based on the datetime settings of the machine you run it on.</p>
case $(($(date +%H)/6)) in 0|1)m="morning";;2)m="afternoon";;3)m="night";;esac; echo good $m
</span></code></pre>
How?</h1>
First, check if the feature is already implemented. man date</code> and try not to giggle. Search for morning</code>. It's not there.</p>
So we need a switch/case</a>:</p>
case EXPRESSION in CASE1) COMMAND-LIST;; CASE2) COMMAND-LIST;; ... CASEN) COMMAND-LIST;; esac
</span></code></pre>
And the expression we're switching on will be the current hour:</p>
date +%H
</span></code></pre>
My first attempt does not work</strong> because I expect too much of Bash:</p>
case $(date +%H) in [0-12]) m="morning";;[13-18]) m="afternoon";;[19-21])m="evening";;*)m="night";;esac; echo $m
</span></code></pre>
It fails because the "ranges" are actually just shell patterns</a>.</p>
I could either expand my script to handle all hours, or compress ranges of hours down into something that can be expressed by patterns. The latter sounds shorter and easier. I want to divide the current hour by 6, to tell which quarter of the day I'm in.</p>
A bit of trial and error reveals that a syntax that allows me to do math on the result of date</code> is:</p>
$(( $(date +%H)/6 ))
</span></code></pre>
because it's shorthand for assigning the result of the math into a variable and using it immediately. This only adds a few characters to the one-liner:</p>
case $(($(date +%H)/6)) in 0|1)m="morning";;2)m="afternoon";;3)m="night";;esac; echo $m
</span></code></pre>
That's it!</p>


Printing
2015-07-20T00:00:00+00:00
The office printers have instructions for setting them up under Windows, Mac, and Ubuntu. I had forgotten how to wrangle printers, since the last time I had to set up new ones was half a decade ago when I first joined the OSL.</p>
Setting up printers on Arch is easy once you know the right incantations, but can waste some time if you try to do it by skimming the huge wiki page</a> rather than either reading it thoroughly or just following these steps:</p>
Install the CUPS client</strong>:</p>
$ yaourt -S libcups
</span></code></pre>
Add a magic line to /etc/cups/cups-files.conf</strong>:</p>
SystemGroup username
</span></code></pre>
With your username on the system, assuming you have root and will log in as yourself in the dialog it prompts for. That line can go anywhere in the file.</p>
Make the daemon go</strong>:</p>
$ sudo systemctl enable org.cups.cupsd.service
</span>$ sudo systemctl start org.cups.cupsd.service
</span></code></pre>
Visit the web interface</strong> at http://localhost:631</a>.</p>
Then you have a GUI sufficiently similar to the one in the instructions for Ubuntu!</p>
There is no GUI client for CUPS to install. If you find yourself mucking about with gpr</code>, xpp</code>, kdeprint</code>, or /etc/cups/client.conf</code>, you have gone way too far down the wrong rabbit hole.</p>


Outage postmortem: Replacing Rust Buildbot's outdated cert
2015-07-17T00:00:00+00:00
At the end of the day on July 14th, 2015, the certificate that Rust's buildbot slaves were using to communicate with the buildmaster expired. This broke things</a>. The problem started at midnight on July 15th, and was only fully resolved at the end of July 16th. Much of the reason for this outage's duration was that I was learning about Buildbot as I went along.</p>
Here's how the outage got resolved, just in case anyone (especially future-me) finds themself Googling a similar problem.</p>
Troubleshooting</h1>
Dave Huseby pointed out the problem on IRC when the slaves that he runs were unable to connect to the buildmaster:</p>
16:48:23 <&brson> edunham: dhuseby said this earlier <huseby> it seems like the verify=3 in the stunnel config is the problem
</span>16:48:39 <&brson> if he changed 'verify' to some other value in the stunnel config it worked
</span></code></pre>
A quick check fo the stunnel docs</a> shows that verify=3</code> is the strictest setting, and will fail if the locally installed cert isn't right. This supports the hypothesis that our cert might be expired. On the buildmaster, I found the cert and examined its metadata:</p>
$ find . -type f -name "*.pem"
</span>$ openssl x509 -noout -issuer -subject -dates -in certname.pem 
</span></code></pre>
On the old cert, the results contained:</p>
$ openssl x509 -noout -issuer -subject -dates -in rust-bot-cert.pem 
</span>issuer= /
</span>    O=Rust Project/
</span>    OU=Bot/
</span>    CN=bot.rust-lang.org/
</span>    emailAddress=admin@rust-lang.org
</span>subject= /
</span>    O=Rust Project/
</span>    OU=Bot/
</span>    CN=bot.rust-lang.org/
</span>    emailAddress=admin@rust-lang.org
</span>notBefore=Jul 14 02:28:50 2012 GMT
</span>notAfter=Jul 14 02:28:50 2015 GMT
</span></code></pre>
This tells me that the cert was created in 2012 and had its expiry set for the seemingly distant future of 2015.</p>
Make a New Cert</h1>
To determine whether the old key had a passphrase on it, go openssl rsa -check -in keyname.pem</code>. It writes the private key to your terminal if there's no password, or prompt for a passphrase if the key has one.</p>
The stunnel docs</a> give most of the relevant incantation. Since no file in our buildbot directory is named precisely stunnel.conf</code>, make cert</code> doesn't quite work right. But it works fine to manually run a variant of the command given in the docs:</p>
$ openssl req -new -x509 -days 3650 -nodes -out cert.pem -keyout key.pem
</span></code></pre>
That prompted me for a variety of information, which I entered where applicable and left blank where it wasn't. The metadata is primarily for the benefit of others verifying that a cert belongs to the correct person, which isn't a relevant concern in our use case.</p>
I then backed up the old key and cert (although they're no longer usable, they contain a bunch of metadata that I didn't know whether I'd need later) and moved the new key and cert to match the old ones' original file names.</p>
Finally, I updated the repository</a> with the new certificate.</p>
Make Buildbot spin up AMIs with the new cert</h2>
This was the tricky bit. Since the slave image does not pull updates to its copy of the Rust Buildbot github repo when it boots, the file had to be statically edited and then the AMIs re-saved. But Buildbot makes its instance requests based on AMI ID, and the IDs are unique to a particular image. So the workflow goes:</p>

Figure out which AMI Buildbot will spin up for a given job</li>
Spin up an instance of the AMI</li>
Remote into it and manually update the cert</li>
Save the instance into a new AMI, noting its ID</li>
Update Buildbot's configuration with the new ID</li>
</ul>
Figure out which AMI Buildbot will use</h1>
The AMI IDs used for spot requests are stored in /home/rustbuild/rust-buildbot/master/slave-list.txt</code> on the buildmaster. From that file I determined that we only had 4 unique AMIs in use:</p>
ami-b74fa1f3 -- windows
</span>ami-7fd23e3b -- generic linux 
</span>ami-381e197d -- android
</span>ami-dbac5f9f -- centos5 (builds snapshots that work with ancient Linux)
</span></code></pre>
The slave list also told me that all requests would be made for instance type c3.2xlarge</code>.</p>
Spin up an instance of the AMI</h1>
Since there were only 4, I did this manually. If it was a recurring task or there had been more AMIs, I would have automated this part of the process.</p>
In the AWS Console, go to EC2, then click the AMIs link under "Images" at the left.</p>
Search for the AMI ID in the search box. Only one AMI is found, because IDs are unique. Then click the big blue Launch button up at the left.</p>
</p>
The only "gotcha" in the ensuing 7-step process is making sure to put the instance into the correct security group. I spun my temporary instances up into the same group as a host I know I can get to from the Bastion server with my credentials, to reduce the number of steps I'd have to troubleshoot if they were difficult to access.</p>
It also helps to tag the spot request with the name of the AMI it was created from, when processing several at once.</p>
Remote into the instance and update the cert</h1>
For Linux-flavored instances, this just meant ssh rustbuild@00.00.00.00</code> (using the instance's public IP, visible in the main instances list) from the Bastion. I then found the old cert on the host, and verified that it matched the old cert on the buildmaster. Checking that the certs matched was as simple as running md5sum cert.pem</code> on both and visually comparing the results, and reassured me that I was overwriting the correct file.</p>
Getting into the Windows hosts requires using RDP</a> after setting up an SSH tunnel to the bastion. Since I was on airport wifi at the time, I had a teammate stick the cert onto the Windows instances instead.</p>
The cert that stunnel</code> actually uses on a Windows host with our configurations actually lives at C:\Program Files (x86)\stunnel\cert.pem</code>, not in the actual repo like on all the sensible operating systems. Although there exists a C:\bot\cert.pem</code>, replacing it does not cause stunnel</code> to connect successfully.</p>
Save the instance into a new AMI</h1>
Check the box by the instance's name in the EC2 instances list, then follow the menus around for Actions -> Image -> Create Image. Note the new AMI's ID, and replace all instances of the old AMI's ID with the new one in the buildmaster's slave-list.txt</code>.</p>
Kick Buildbot a bit</h2>
After the AMIs and FreeBSD, Bitrig, and Mac builders all had the new cert, I restarted Buildbot on the buildmaster and reran its script for creating the stunnels. Althoug it didn't gracefully pick up where it had left off on partially built pull requests, closing then re-opening the PRs caused it to notice them and resume building successfully.</p>
Hopefully TaskCluster gets OSX support soon, so we can start switching off of Buildbot.</p>
Prevent it from happening again</h1>
After I first published this post, Gerv</a> pointed out that the correct final step would be "Add an alarm to the shared IT calendar for a month before the new cert expires". In my case, the analog to that alarm is "Make sure we move away from Buildbot in less than a decade". However, if you're reading this post to solve a similar problem in an infrastructure that will still exist at the date of the cert's expiry, you should automate a reminder so that you or your successor doesn't get the same unpleasant surprise.</p>


Airport Wifi
2015-07-16T00:00:00+00:00
Many "free" wifi hotspots give you a limited time per computer. If you're traveling light and forgot to bring extra devices, it's easy to give a Linux laptop multiple personalities:</p>
$ ip link
</span>    1: lo
</span>    2: wlp4s0
</span>    3: enp0s25
</span>$ ip link set dev wlp4s0 down
</span>$ macchanger -r wlp4s0
</span>$ ip link set dev wlp4s0 up
</span></code></pre>
... And then connect to the wifi and jump through its silly captive portal hoops again!</p>
Changing your MAC address occasionally can be part of a healthy security diet, making your device slightly more difficult to track, as well.</p>


Interactive Rust Examples in Static Pages
2015-07-13T00:00:00+00:00
Rust by Example</a> has a little box where readers can interact with some example Rust code, run it using the playground</a>, and see the results in the page. As a sysadmin I'm loath to recommend that anybody trust the playground for anything, but as a nerd and coder I recognize that it's super cool and people want to use it.</p>
There are 2 ways to stuff a Playground into your website: The easy way, and the "right" way. Here's how to do it the easy way, and where to look for examples of the hard way.</p>
The Easy Way</h1>
There are these cool things called iframes</a> that basically let you put websites in your websites...</p>
</p>
All you have to do is stick this one line in your page's source, and modern browsers will go load and inject the specified page -- in this case, the playpen. The exact technique for injecting raw HTML will differ based on your blogging platform. With Tinkerer, the source will look like this:</p>
.. raw:: html 
</span>
</span>    <iframe src="https://play.rust-lang.org/" style="width:100%; height:400px;"></iframe>
</span></code></pre>
That'll render with the default contents of the playpen. Note that it'll someties try to be clever and load the code that the viewer most recently opened in it when loaded with no arguments:</p>
</iframe>
Custom Code in the Playpen</h1>
When you load the Playpen, you can specify the ID of a gist</a> whose contents should appear in the text area. Let's say we're doing a simple hello world:</p>
// This code is editable and runnable!
</span>fn main() {
</span>    println!("Hello from edunham's blog!");
</span>}
</span></code></pre>
I uploaded it</a> and noted the unique hash on the URL, df9d98db4c29089e93b8</code>. That's the magic number that I'll stick on the end of the playpen link to see the gist, like https://play.rust-lang.org/?gist=df9d98db4c29089e93b8</a>. Here's the raw HTML for an iframe that loads the playpen with the contents of that gist:</p>
<iframe src="https://play.rust-lang.org/?gist=df9d98db4c29089e93b8" style="width:100%; height:400px;"></iframe>
</span></code></pre>
It'll render like this:</p>
</iframe>
The Hard Way</h1>
I also tried pulling out the requisite HTML, Javascript, and CSS from the main Rust site</a>. If I come back later and get it working, I'll update this post with instructions, but till then the iframe technique is substantially easier. If you get it working the way the main site did using a Sphinx-based blogging platform, please let me know!</p>
Thanks</h1>
As usual in my web-related endeavors, I accidentally nerd-sniped</a> Mythmon</a> with this problem, and he spent quite a while helping me troubleshoot it and teaching me about Firefox's developer tools.</p>


Rust's Packaging Status Across Distros
2015-07-07T00:00:00+00:00
One of many questions facing the Rust infrastructure team right now is "What's our packaging situation?". We don't have a centralized source of information on what version of Rust is available in which systems' package managers, and we don't even know where to find that information.</p>
This post is the notes I've taken in researching Rust's packaging status across distributions.</p>
I last updated this post on 8/17/2015.</p>
In Summary</h1>
Distro</th> Rust's Status</th> Cargo's Status</th></tr></thead>

Debian</td> 1.0.0 in unstable</td> Requested</td></tr>
Ubuntu</td> Community package</td> Community package, bad docs</td></tr>
Gentoo</td> 1.1.0 in index</td> Community package</td></tr>
OpenSUSE</td> 1.0.0 in index</td> Recent version in index</td></tr>
Arch</td> multirust in index</td> 2 community packages; 1 bad</td></tr>
Fedora</td> Community package</td> No separate package</td></tr>
Slackware*</td> Community package</td> No separate package</td></tr>
Windows</td> in chocolatey</a></td> No separate package</td></tr>
Mac</td> in Brew</a></td> No separate package</td></tr>
FreeBSD</td> Community port (?)</td> No separate package</td></tr>
</tbody></table>
There are an unreasonable number of tiny Linux distributions out there, but they fall into a handful of "families". For a visualization of the history of various distros, check out this map</a>. I've elided CentOS and RHEL as "flavors of Fedora", and a bunch of popular new Ubuntu flavors (ElementaryOS, Mint, etc.) as "compatible with Ubuntu packages, and probably smart enough to Google for Ubuntu docs if there aren't any specific to their OS".</p>
Debian</h1>
Debian, Ubuntu, Kubuntu, Mint, and Knoppix all use .deb</code> packages installed through package managers that wrap the dpkg</code> tool.</p>
The Debian Stable</a> package list has dedicated sections for some languages, but Rust is too young to be among them. Rust does not appear in the Jessie stable package list</a>, the Debian testing package list</a> or the Debian unstable package list</a>. Debian bug #786432</a> is a request for Cargo to be packaged, which describes what Cargo will need to do to be packageable. Bug #689207</a> tracks the status of packaging rustc</code> for Debian.</p>
There's a rustc package in Debian Unstable</a>.</p>
The Debian Wiki has a rust packaging page</a> to track the status of packaging Rust for Debian.</p>
Ubuntu</h1>
The official Ubuntu</a> package repo has no mention of Rust or Cargo in the trusty</a> or vivid</a> package lists.</p>
The rust-ci.org help</a> tells users to install Cargo from Corey Richardson's PPA</a> (last updated July 2014, build currently failing) and Rust from Hans Hoel's PPA</a>, in which rust-stable</code>, rust-nightly</code>, and cargo-nightly</code> appear to be up to date.</p>
Googling for how to install Rust on Ubuntu also turns up a page on vultr.com</a> directing users to intsall Rust using rustup.sh</code>. Another</a> of the top hits also instructs users to use rustup.sh</code>.</p>
Gentoo</h1>
There's a Rust 1.1.0 package in the Gentoo package index</a>. On GitHub, Heather</a> has packaged a Rust overlay for Gentoo and appears to be actively developing it, although the build is currently failing.</p>
There's a cargo portage overlay</a> but cargo is not provided through the package index.</p>
OpenSUSE</h1>
Rust 1.0.0 is packaged for OpenSUSE</a>. Duncan Mac-Vicar</a> has a blog post on trying Rust on OpenSUSE from 2014. My search for "How to install Rust on OpenSUSE" also turned up a youtube video</a> on installing Rust from the nightly tarball. It seems to just be a walkthrough of what happens when you follow Rust's installation guidelines, but kudos to the OpenSUSE community for catering to diverse learning styles.</p>
An OpenSUSE Cargo package</a> is also available in what appears to be the main index.</p>
Arch</h1>
The Arch Wiki</a> has a very useful page on Rust that includes instructions on cross-compiling. There's also a community package</a> of rust 1.1.0. The Arch User Repository (AUR) provides Cargo through a choice of cargo-bin</a> or cargo-git</a>, though the latter hasn't been updated since February.</p>
I later learned that you need to install the multirust</a> package to get Rust and Cargo.</p>
Fedora</h1>
Fedora has this copr</a> build system thing in which one apparently finds community packages. At least, I found rust-binary</a> (poorly documented and last built 6 months ago) and rust-unofficial</a> (last built 18 months ago) in it. A blog post about building Rust on Fedora</a>, posted in 2013 but with good SEO for queries about "how to install Rust on Fedora", recommends building from source. There's no rust</code> or cargo</code> in the CentOS 6 package list</a>, either.</p>
Fedora is tracking the official effort to package rustc</code> in redhat bug #915042.</p>
A stackoverflow question</a> from late last year asked about how to build Rust for Redhat 5. The fedora-rust</a> repo on GitHub provides a sporadically-updated Docker image with Rust installed.</p>
FreeBSD</h1>
The FreeBSD Wiki</a> tracks the status of the Rust port. The port overview page</a> tracks which version is available, which is the latest Rust version available in July 2015. There's also a Rust on FreeBSD</a> internals thread that tracks other relevant information and tools.</p>
Slackware*</h1>
According to the 2014 r/linux survey</a>, 53 of the 10,292 respondants used Slackware on a desktop or server. A variety of search queries for Rust and Slackware turn up Rust's generic Linux installation docs as the first hits, and one github issue from a slackware user</a>. Extensive digging reveals a Rust 1.1.0 package in the SlackBuilds repository</a>.</p>
* The fact that I researched whether anyone had packaged Rust for Slackware is in no way intended to imply any plans for ever supporting such a package. I was just curious. Really.</p>
Thanks!</h1>
I appreciate readers Huon Wilson and Seo Sanghyeon taking the time to point out useful links for this post! Both mentioned the presence of the Debian Unstable package, and Seo informed me of redhat bug #915043</a> as well.</p>


Mozilla Onboarding
2015-07-06T00:00:00+00:00
On April 16th, I threw an application toward this job posting</a> on careers.mozilla.org. I doubted whether I'd be qualified, but I reminded myself that most people apply to jobs where they meet only 80% of the critiera. I could, with some creative redefinition ("of course an internship at Intel is a year of relevant experience!"), meet every listed criterion. So I applied, since the worst they could say was no.</p>
Here's a rundown my experience getting interviewed and onboarded, which might be of interest to current Mozilla employees curious about how things have changed since they joined, and to anyone interested in working there.</p>
Application</h1>
The recruiter replied to my application only 3 hours after I submitted it. This was certainly an improvement since the 6 months of ultimately fruitless bureaucratic limbo when I applied for a Mozilla internship a few years ago!</p>
I found out later that the 3-hour turnaround time included a member of the Servo team reading my resume and verifying that my background looked relevant to their interests, which makes it all the more impressive.</p>
Phone Screens</h1>
On April 20th, I had a quick phone chat with my recruiter. It was totally non-tecnical, and I summarized it to my friends afterward as "Did you read your own resume? Did you read the job description? Ok, cool!". I'm sure they were screening for other stuff as well, perhaps basic social skills and an acceptable level of fluency in spoken English, but that screen was not nearly as scary as I'd been prepared for.</p>
My first techncial phone screen was on 4/22. A second recruiter organized the details of the call, and I talked with Brian Anderson about my background and the Rust programming language's DevOps needs. He was particularly interested in my work with Chef and Jenkins in production at my Urban Airship job. After this phone screen, the title on the job posting got changed from "Operations Engineer" to "DevOps Engineer".</p>
On 4/24, I talked with Lars Bergstrom about the Servo project's needs. Learning about the place that project was at compared to Rust started giving me some perspective into the similarities between the DevOps Engineer role and my work juggling systems administration for a variety of open source projects at the OSU Open Source Lab.</p>
My final phone screen was with Jack Moffitt, on 4/29. This conversation was less a review of my experiences and more a brainstorming session about techniques for speeding up the run time of Rust's continuous integration tests. Although I wasn't familiar with the specific scenario yet, I was able to apply my knowledge of CI and Git techniques to propose some possible solutions that the team hadn't considered yet.</p>
The On-Site Interviews</h1>
I interviewd on-site at Mozilla's San Francisco office on May 4th. My day was scheduled from 10am to 3pm, and I spoke with 5 different Mozilians from various teams. Ordinarily these interviews would have been scheduled at least a week later, but a friend and I had been planning a road trip down to the bay area anyway, so we shifted the dates a bit to fit with the interviews.</p>
Again, I asked the recruiters whether they had any advice on what to expect from the whiteboarding component of the interviews, and again they gave no answer.</p>
Two interviews were with managers higher up in Mozilla Research's organizational hierarchy. One was very concerned about community and avoiding the problems of "not invented here", and felt unfamiliar with most "DevOps" related terminology. In that interview, I drew pictures on the whiteboard to explain the similarities between public and private clouds, and we talked a lot about open source community. The other manager wasn't hesitant to air her concerns about my relative lack of corporate experience, which gave me a chance to explain the applicability of my open source leadership background and OSU Open Source Lab experience to the DevOps Engineer role.</p>
My technical interview with a Rust team member during my onsite interviews was more of a friendly chat about the infrastructure and how to improve its performance. After the first couple of interviews, I went to lunch with a variety of research and devops team members. We chatted about the infrastructure, and I enjoyed the chance to get to know my prospective coworkers in a less formal setting. I find it important to interact with people in a setting where it's socially appropriate for them to complain about their jobs -- the nature of those complaints (or lack thereof) is one of the most useful cues about a project's real culture rather than the polite facade it necessarily puts on for newcomers.</p>
After lunch, I had the only interview that I'd consider particularly technical. My interviewer framed a question about designing an algorithm to copy a linked list in memory and asked me to describe a solution (psuedocode optional), rather than worrying about the details of writing in any specific language. The question had a trick to it and I would have solved it much faster if I'd seen it before, but my interviewer fed me hints and helped me stay on the right track. This collaborative approach made the interview feel like I was solving a real problem with a co-worker, rather than the "we want to see what you're like when we terrify you to the breaking point" of some other companies' interview styles. The tone of that interview cemented my impression that Mozilla had the collaborative, non-malicious culture that I was looking for.</p>
My final interview was with a TaskCluster developer. Since we'd already spoken at lunch (and apparently I'd made a sufficient impression of my technical skills), he told me "Instead of an interview, I'm going to sell you TaskCluster". We then went through a slide deck on how TaskCluster is designed and its intent to replace BuildBot, and I got a lot of my introductory questions about the tool out of the way.</p>
The Offer</h1>
On May 6th, I had a quick phone chat with my original recruiter to discuss compensation. I had already talked with a manager for my prospective role about Mozilla's bonus and benefits package, which allowed me to do the math on what kind of base salary would result in a total compensation package comparable to the other offers I had on the table. I discussed these numbers and my reasoning behind them with my recruiter. When doing the math on base salary, I also privately calculated the ranges of offer for which I would want to try to negotiate higher, and what ranges I'd be happy to simply accept without haggling.</p>
On May 8th, we had another call during which my recruiter disclosed my specific offer. The number is comfortably competitive for my skills and role -- although it doesn't precisely match my other offers in salary, the difference is less than the benefit to my career of working at a company that understands and values open source.</p>
I received my offer letter on May 12th, verified that the confidentiality agreement was sane (it protects the user data that Mozilla gathers with peoples' permission, rather than trying to restrict employees' open source contributions), and signed and returned the paperwork.</p>
Later that day, the Mozilla onboarding system started sending me emails. It walked me through setting up accounts, ordering my work laptop, filing tax paperwork, and all the other bureaucratic minutae associated with starting a new job. The system handed out onboarding tasks in small groups on a set schedule, all with friendly and enthusiastic messages explaining what was going on with that set of forms. This made the whole process much easier than other places I've onboarded, where one spends several hours shut in a room with an HR person doing paperwork on the first day.</p>
Time Passes...</h1>
Between my offer and start date, I found a new place and moved into it, as well as completing the onboarding tasks as they trickled out of the automated onboarding system. I picked up my laptop from Mozilla IT on May 22, then took it home and installed Arch Linux. I tried Windows</a> just to give it a chance, but it rapidly failed my informal tests of usability.</p>
When I came into the office to pick up my laptop, our wonderful front desk ninja of all trades Katt took my photo and issued me a badge. This was somewhat surprising, as nobody had warned me that I'd be getting photographed that day, but it was ultimately convenient since it provided me with a badge to enter the office on my first day.</p>
I also sent my SSH and GPG public keys to a coworker who started setting up my access to the systems I now administer.</p>
First Day</h1>
My first day was actually a Tuesday, 5/26, since that Monday was memorial day. I attended a mandatory IT orientation in which they explained fancy new technologies such as IRC and gmail filters, from 8:30 to 11am. It could have been a lot shorter if they'd packaged it as wiki pages or individual videos to allow us to skip or skim the topics we already knew, but I guess they find it important to put a personal touch on the orientation by having a real live human present it. I'm sure that if I was more extraverted or less symbiotic with my computer, I would have appreciated it a lot more.</p>
The things that people say about a "firehose of information" are all true. There's a comprehensive but somewhat obsolete wiki, a less thorough but closer to up to date jira, and a bunch of public documentation as well. I've been careful to sign up for only those mailing lists which coworkers tell me are directly relevant to my job right now, or else the deluge of facts would be even worse.</p>
First Month Retrospective</h1>
I've learned that there are benefits and drawbacks to working at a company that I associate with the most friendly and knowledgeable people I've ever met. The benefits are that my coworkers are amazing and inspire me to be the best professional and open source community member that I can. The "drawback", if one can call it that, is that everyone suddenly assumes I'm just as friendly and knowledgeable as everyone else. The title of "Mozilla Employee" seems at times to be a glowing crown emblazoned with "Ask Me Firefox Questions", but I've so far been able to redirect everyone onto my more knowledgeable peers to get their problems solved. And on the whole, there's no better way of becoming that archetype of friendliness and knowledge than to suddenly have the entire world expecting me to embody it.</p>
I've always struggled, as I'm sure you might as well, with accurately assessing my own techncial comptence. At Mozilla, I never feel like the smartest person in the room, and yet I bring specialized knowledge to my team that it would lack without me. This balance keeps me constantly learning, while reassuring me that my contributions are valuable, which is close to optimal for my overall happiness.</p>
The Rust and Servo teams and communities have been incredibly welcoming to me, and they communicate in ways that I find easy to work with.</p>


One free ticket to Rust Camp
2015-06-30T00:00:00+00:00
UPDATE</strong>: The ticket has been claimed by a local student! Thanks everyone for helping boost the signal about this.</p>
Right now, there are 11 tickets being given away by community members to help women</a> and minorities</a> attend Rust Camp</a>.</p>
I have one spare ticket, which I got for a friend who ended up having other commitments on the day of the conference. I would like to give this ticket to someone who would otherwise be unable to attend Rust Camp, regardless of whether or not they're a minority. To maximize the amount of impact that all our free tickets have, I'd ask that anyone qualified for one of Graydon's tickets</a> appeal to him first, so that the ticket I'm giving away can help someone who wouldn't have another option to attend.</p>
I also happen to have a voucher from United Airlines for $500 worth of flights, so if you could not attend Rust Camp without travel assistance within that amount, let me know.</p>
To recap: Are you a Rust community member who couldn't attend without a sponsored ticket? Are you unable to get another free ticket? Send me an email, rustcamp</code> at edunham.net</code>. Since I'm relatively new to the community and don't really know who's who yet, I'd love to see a link or two to your involvement with Rust -- a blog post, tweet, GitHub project, StackOverflow question/answer, or anything else. I'll update or take down this post once the ticket has been claimed.</p>


How to find a Buildbot slave's IP
2015-06-26T00:00:00+00:00
Today I got a seemingly ordinary request from a community member who volunteers a build slave for Rust's buildbot:</p>
my builder is behind a firewall that just cycled IP's
</span>and I don't know what it is
</span>edunham: can you get the IP address of the bitrig builder for me?
</span>I have admin access to the builders website but it doesn't list the IP addresses of builders
</span></code></pre>
No officially supported technique to find slave IPs</h1>
I logged into the buildmaster and checked the slave's logs. I found that although slaves reported a variety of information to the master, nothing appears to log their IPs.</p>
I consulted the Buildbot docs, other users, and a project maintainer, but nothing could tell me a "correct", supported way to find a slave's IP.</p>
The Workaround</h1>
I know that the slave whose IP I want is running builds successfully, which means it has an stunnel</a> back to the master.</p>
There are few enough slaves in this setup that it'll be faster to manually check what host each stunnel corresponds to than to automate it with a script.</p>
I logged into the buildmaster, assumed the identity of the buildbot user, and used netstat</code> to get a list of the open stunnels:</p>
buildbot@buildmaster:~$ netstat -tupn | grep stunnel
</span>(Not all processes could be identified, non-owned process info
</span> will not be shown, you would have to be root to see it all.)
</span>tcp        0      0 10.190.147.69:9988      54.193.203.23:49167     ESTABLISHED 1038/stunnel4
</span>tcp        0      0 127.0.0.1:58123         127.0.0.1:9989          ESTABLISHED 1038/stunnel4   
</span>tcp        0      0 127.0.0.1:33051         127.0.0.1:9989          ESTABLISHED 1038/stunnel4   
</span>tcp        0      0 10.190.147.69:9988      54.219.68.106:36176     ESTABLISHED 1038/stunnel4   
</span>tcp        0      0 10.190.147.69:9988      63.245.221.32:54336     ESTABLISHED 1038/stunnel4 
</span>...
</span></code></pre>
The IP of the host I'm looking for will be somewhere in the second column of addresses. Since there were only a couple more than I've shown you up there, it was straightforward to test each plausible candidate individually.</p>
To see whether a given IP is the slave who moved, I can use reverse DNS to approximate what hostname it belongs to. The host</code> command from dnsutils</code> is a convenient wrapper to reverse DNS when you give it an IP:</p>
$ host 54.193.203.23
</span>23.203.193.54.in-addr.arpa domain name pointer ec2-54-193-203-23.us-west-1.compute.amazonaws.com.
</span>
</span>$ host 63.245.221.32
</span>32.221.245.63.in-addr.arpa domain name pointer corp.mtv2.mozilla.com.
</span></code></pre>
So the first of those IPs is one of our EC2 buildslaves, and the second is a mac sitting under my coworker's desk in the Mountain View office. There was one IP that corresponded to neither AWS nor the Mozilla network, and it belonged to the missing build slave.</p>
It's also possible, with a bit more typing, to perform reverse DNS lookups</a> with dig</code>. You just reverse the order of the blocks of digits in the IP address, and add .in-addr.arpa</code> to the end. For instance, abc.def.ghi.jkl</code> would become jkl.ghi.def.abc.in-addr.arpa</code>.</p>


Moving a Jekyll site from GitHub Pages to Amazon S3
2015-06-25T00:00:00+00:00
The rust-lang.org web site used to be hosted on GitHub Pages. This gave it excellent uptime and made deploying changes easy, but did not support HTTPS.</p>
The Options</h1>
I identified 3 major options for how we could alleviate users' anger at not seeing the little lock icon in their URL bars:</p>

Route all requests to the site through our nginx proxy, which has SSL set up

This is problematic because it introduces a new single point of failure in the system, if the proxy host goes down.</li>
</ul>
</li>
Stick a CDN, CloudFlare, between the GitHub page and the world.

This will encrypt communications between the CDN and the user, but not between CloudFlare and GitHub. This means the lock icon in the URL bar is present, but actively lying to the user about the security of the page.</li>
</ul>
</li>
Move the site to Amazon S3 and have CloudFront distribute content securely to users.

This is similar to the GitHub/CloudFlare model, with the important difference that there's negligible opportunity to MITM between S3 and CloudFront</li>
The inconvenience is that I have to rebuild the functionality of GitHub pages where pushing to the repo automatically rebuilds the site.</li>
</ul>
</li>
Update: A comment on July 1 in the github thread</a> pointed out that surge.sh</a> offers static web publishing with Jekyll for free, or with custom SSL for $13/month. It could probably be deployed automatically from Travis, and might be a better fit than AWS for other use cases.</li>
</ul>
Since I want to improve security without harming performance or adding unnecessary new moving parts to Rust's infrastructure, the third option is distinctly preferable.</p>
New Workflow</h1>
The only change visible to site authors will be that content is added to the master</code> branch rather than to gh-pages</code>.</p>
Under the hood, there are several pieces to this transition:</p>

Make Travis rebuild the site on every commit to master</code>, and upload the successfully built site to S3</li>
Send out a CloudFront invalidation when the site is rebuilt, to guarantee that users see the newest version of pages</li>
Update the docs to explain the new workflow, and do some housekeeping like adding a custom error page instead of the GitHub pages 404.</li>
</ul>
Make Travis build site and upload to S3</h1>
In S3, create a bucket with a descriptive name for the files to end up in. Remember that having dots in bucket names won't work with CloudFront, and bucket names have to be globally unique. Mine is called www-rust-lang-org</code>, since it exists to hold the www.rust-lang.org site.</p>
Follow the directions to create an IAM user</a> for Travis. Download the credentials and keep them somewhere safe -- you'll later encrypt them to allow Travis to use them to push your site to s3. Set the Travis account's policies</a> to allow it to upload to the bucket you created, deploy a CloudFront invalidation, and not touch anything else in your infrastructure. It's best practices</a> to give the account the minimum permissions with which it can get its job done, to minimize the harm that will occur if someone breaks the encryption of its credentials.</p>
On the Travis site</a>, push the button that turns on CI for your repository. In the repo settings menu of the user interface (travis-ci.org/owner/repo/settings), make sure that building pull requests is turned off (or the build will fail on PRs, since for security reasons a PR cannot decrypt the encrypted environment variables).</p>
Then follow the s3 deployment guide</a> to create your .travis.yml</code> file. During the interactive prompt for encryption, make sure that the Travis CLI uses the key for the correct repository, by passing the -r owner/repo</code> flag. Each repo has a unique keypair, and a value encrypted with your fork's key cannot be decrypted once your changes are merged to the organization. A couple quick local tests have revealed that Travis seems to guess what repo it's encrypting for from the GitHub URL of the current repository's origin</code> remote.</p>
Push or PR your changes to the repo for which the credentials are encrypted, and verify that the correct files showed up in the bucket. Then set up CloudFront</a>.</p>
Make Travis automatically invalidate old pages</h1>
The benefit to CloudFront is that it gives excellent availability by caching pages geographically close to users. The problem with this is that caching is hard, and can cause users to see stale pages after a site update unless we inform the CDN that it should invalidate cached copies of changed pages.</p>
Ideally, I want Travis to send out a CloudFront invalidation immediately after uploading fresh pages to S3. Travis will be running a Ruby environment because the site is built with Jekyll, so I found a ruby gem</a> which automatically generates Cloudfront invalidations.</p>
Since I'll be trusting the gem with credentials to my infrastructure, I compared its source to the invalidation API instructions</a> to make sure it wasn't doing anything obviously malicious or incorrect.</p>
There are 3 new instructions to add to .travis.yml</code> in order to make that gem do its thing: decrypting the invalidation instructions, installing the invalidator, and running the invalidation.</p>
Craft a _cf_s3_invalidator.yml</code> file according to the instructions in the ruby gem, then encrypt it with the command:</p>
travis encrypt-file -r owner/repo  _cf_s3_invalidator.yml --add
</span></code></pre>
The --add</code> flag dumps the decryption command into the before_install</code> step of your .travis.yml. Read the output of the travis command in your terminal, because it's important: You need to add the encrypted file, but not the original, to git.</p>
Add these lines to .travis.yml</code> to install the invalidator and run it when necessary:</p>
after_success: cf-s3-inv                                                        
</span>script: jekyll build  
</span></code></pre>
Custom Error Page</h1>
To have Jekyll build a custom error page with a picture on it, put your picture in the directory with your other images and write a simple HTML page to display the error. Mine looks like this</a>.</p>
In the settings for your S3 bucket, specify error.html</code> as your custom error page.</p>
In the settings for your CloudFront distribution, customize which error page gets returned for various HTTP response codes.</p>


DMARC
2015-06-17T00:00:00+00:00
Today, the security alias for a site I administer got an automated message pointing out that we lacked a DMARC record. Here's what I learned about how to set up and test them.</p>
How do clients detect forged emails?</h1>
There are two main ways for an email client to check whether a message really came from the server it claims it's from:</p>

DKIM</a>, or DomainKeys Identified Mail, attaches a digital signature to each message and publishes the public key for that signature in the DNS. The client looks up the alleged origin server's public key, and if it can decrypt the signature, that means the message was signed with the corresponding private key (and thus ostensibly originated on a server controlled by the same person who controlls the DNS).</li>
SPF</a>, or Sender Policy Framework, uses TXT records to publish a list of hosts from which legitimate messages will be sent. The client compares the origin of a message to that list, and if the message came from one of the listed hosts, the client knows that it was sent from a machine that the person who controls the DNS considers authorized.</li>
</ul>
Of course there are other techniques as well, such as signing a message with the sender's GPG key and then having the recipient manually look up the sender in the trustweb, but DKIM and SPF are the most widely used automatic systems for spam detection.</p>
What happens when they find a fake?</h1>
This is where DMARC</a> comes in. With just DKIM and SPF, the alleged sender (ie, the domain who's either sending the mail or being spoofed) has no way to tell clients what to do when messages fail both authenticity checks.</p>
DMARC is simply a TXT record published in a server's DNS that tells clients what the person controlling that DNS wants them to do if a message fails both DKIM and SPF checks.</p>
What's in a DMARC record?</h1>

Version, which as of 2015 will only ever be v=DMARC1</code>. This field is for future-proofing in case the protocol needs to change someday.</li>
Policy, which will be one of p=none</code>, p=quarantine</code>, or p=reject</code>. The policy tells clients what to do with messages that appear to be spam.</li>
Percent, which tells clients how often to check messages from this domain. pct=100</code> will ensure that all messages are checked.</li>
Reporting address is a URI that specifies who clients should tell about messages which failed both SPF and DKIM. This will probably look like rua=mailto:dmarcreports@yourdomain.tld</code>.</li>
</ul>
What policy should I use?</h1>
If you choose reject</strong>, you're asking clients to throw away all messages from your domain which fail both SPF and DKIM authentication. Only use this setting if you're extremely confident that users will never need to see the contents of unauthenticated or incorrectly authenticated mail that comes from legitimate servers on your domain.</p>
Use quarantine</strong> if messages that fail both SPF and DKIM authentication should be marked as spam but delivered by clients. This can be a good compromise between ensuring that users are notified of messages that fail authentication, yet letting legitimate but poorly-authenticated messages get to somewhere the users can see them if they check their spam folders.</p>
The none</strong> setting is for testing purposes. If you're just starting out with DMARC, setting your record to none</code> for the first week or two will allow you to see clients' reports of which legitimate emails from your domain are failing their DKIM and SPF checks. It's a good idea to leave your DMARC policy set to none</code> until you have DKIM, SPF, or both for every legitimate service that sends emails from your domain. Remember that a message only has to pass one authentication method to be considered not spam -- DMARC is only relevant to the messages which fail both</em>.</p>
Deploying DMARC</h1>
First, pick the email address to which reports of apparently-spoofed messages should be sent. It's a good idea to create a new email alias (this should be trivial if you already control your own DNS), so that you can add other administrators if your team grows.</p>
Next, figure out what the record should say. It might look like this:</p>
v=DMARC1;p=none;pct=100;rua=mailto:dmarcstuff@yourdomain.com
</span></code></pre>
Once you have the record, add it to your DNS as a TXT record for _dmarc.yourdomain.com</code>. Then wait a few minutes for the world's DNS servers to get the memo about your new record, and verify that it's correctly deployed!</p>
Verify the DMARC record</h1>
From the command line, you can use dig txt _dmarc.somedomain.tld</code> to see what that domain's policies are. For instance, Google's policy is to quarantine messages and inform a mailauth-reports</code> address:</p>
$ dig txt _dmarc.google.com
</span>...
</span>;; ANSWER SECTION:
</span>_dmarc.google.com.  484 IN  TXT "v=DMARC1\; p=quarantine\; rua=mailto:mailauth-reports@google.com"
</span>...
</span></code></pre>
If you prefer a pretty online interface, you could use a free online tool like dmarcian.com</a> instead.</p>
If you happen to have sufficiently dark-gray hat handy, you could try spoofing a message to yourself from your own domain:</p>
$ mail -r spoofed@yourdomain.tld you@gmail.com
</span></code></pre>
This will require no small amount of fiddling with firewalls to get working, since many personal systems are configured by default to be unable to send mail.</p>
Alternately, you can just wait until legitimate mail gets sent from your domain, then see what clients report back. The pct=100</code> directive asks clients to report on all mail recieved from your domain, regardless of whether it passed or failed.</p>
Next Steps</h1>
Leave the p=none</code> setting in place until all the systems which you expect to send legitimate emails will have sent something. Then audit the logs emailed to you by client mail hosting providers and see whether any legitimate messages failed both DKIM and SPF.</p>
Fix the systems which sent those poorly-authenticated messages, check the logs from after your fix to make sure no messages are failing both authenticity tests any more, and then increase the p</code> setting to either quarantine</code> or reject</code> by editing the TXT record.</p>


Downloading an S3 bucket
2015-06-17T00:00:00+00:00
Since I'm curious about how often files are downloaded from S3, I enabled logging on the buckets serving them and directed the logs into a bucket which I created to hold them. Then I wanted to move everything on that logs bucket to my local machine, so I could poke around in the logs and ascertain the best way to turn them into useful information.</p>
Install the CLI</h1>
Stackoverflow</a> pointed me toward the awscli</code> Python module. Since I don't keep a system pip, I installed it to a virtualenv:</p>
$ virtualenv2 v
</span>$ source v/bin/activate
</span>$ pip install awscli
</span></code></pre>
Tell the CLI your credentials</h1>
It just needs your Access Key ID and Secret Access Key. :</p>
$ aws configure
</span></code></pre>
Download the desired bucket</h1>
Decide where you want to save the bucket contents to, then sync it:</p>
$ aws s3 sync s3://bucket-of-logs ~/bucket-logs/
</span></code></pre>
That's it!</h1>
It's surprisingly easy to download the contents of an entire S3 bucket, considering that the documentation on how to do it is spread out over several different sources and buried in comprehensive reference guides</a>.</p>


Deleting spam logs
2015-06-15T00:00:00+00:00
Some spammers got onto the Mozilla network, scraped a major channel's user list, and PMed everybody requests to join their network from almost 1,000 different nicks. Here's how I tidied up afterwards.</p>
Close spurious PM buffers</h1>
Simple fix from the irssi docs</a>:</p>
/set autoclose_windows on
</span></code></pre>
This will automatically close any PM buffer where the other person (or bot) is no longer online. Another cool setting is:</p>
/set autoclose_query 172800
</span></code></pre>
That will close any private message window that's been silent for more than 2 days. You can tweak the number of seconds higher or lower to fit your use case and prevent it from deleting real conversations.</p>
Delete logs by length</h1>
This particular spammer pasted one or two lines per log, and I happen to know that I have had no one-line conversations on that network whose logs I wish to keep. Here's a slightly ugly chunk of shell to delete every log whose length was between 1 and 5 lines (inclusive):</p>
~/irclogs/moz$ wc -l * | grep "[1-5] [a-zA-Z]" | cut -d " " -f 9 | xargs rm
</span></code></pre>
Pipes are our friends! Just run the commands one at a time to see what they do:</p>

wc -l</code> counts the lines in the file, with a return of the form # filename</code></li>
grep</code> finds things which match the regular expression. [1-5]</code> matches any single digit between 1 and 5, the space matches a space, and the [a-zA-Z]</code> matches the first letter of the name of the log.</li>
cut</code> selects one column of output. The -d</code> part says that I want the columns to be space-delimited, and the -f 9</code> selects the 9th column from the output. It's 9 becuase wc</code> pads its output with a bunch of spaces before the single digit of file length.</li>
xargs</code> takes the thing you piped into it and feeds it as an argument into rm</code>. In this case, cut</code> spits out a list of filenames, so rm</code> will go through and remove each one.</li>
</ul>
Delete logs by contents</h1>
Since it's the same line in every spam log, I can also delete all PM logs that contain the spam message:</p>
~/irclogs/moz$ grep -l --exclude="#*" "join my irc network irc.cooldudeirc.com " * | xargs rm
</span></code></pre>

grep</code> finds all files containing the stated string. The -l</code> says "only return file names, not the part of the contents that matched" and the --exclude</code> pattern makes the search return only private message logs (since channel logs start with a #</code>).</li>
xargs</code> does the same thing as before.</li>
</ul>


Installing Playpen on Ubuntu
2015-06-10T00:00:00+00:00
One of the more egregious inconsistencies in Rust's architecture is that play.rust-lang.org</code> lives on an Arch box, while everything else is Ubuntu. Before the team has a dedicated operations person, the argument for using Arch was that playpen</a> comes pre-packaged</a> for it, whereas one has to build it oneself on Ubuntu.</p>
But standardizing the infrastructure to one OS is a really cool thing to do, since it requires that much less thought and effort to do any update that affects every system (I'm looking at you, security patches).</p>
The Goal</h1>
I want to install playpen on an Ubuntu host. For testing purposes, it's a DigitalOcean droplet, but eventually it'll be an AWS EC2 instance.</p>
Additionally, I'd like to automate the installation process to make it repeatable. For this, I'm using ansible</a>.</p>
I'm currently using Ubuntu 15.04, since it has the libsystemd-dev</code> package available and I'll be updating to the 16.04 LTS when it comes out at the end of 15.04's life anyways.</p>
The Playbook</h1>
---
</span>- hosts: play
</span>  remote_user: root
</span>  tasks:
</span>  - name: Create .ssh directory
</span>    file:
</span>        # The ansible_ssh_user is specified in the hosts file
</span>        path=/.ssh/
</span>        state=directory
</span>        owner={{ ansible_ssh_user }}
</span>        group={{ ansible_ssh_user }}
</span>  - name: Install known_hosts file
</span>    copy:
</span>        src=known_hosts
</span>        dest=/.ssh/known_hosts
</span>        owner={{ ansible_ssh_user }}
</span>        group={{ ansible_ssh_user }}
</span>  - name: Clone Playpen
</span>    git:
</span>        repo=https://github.com/thestinger/playpen.git
</span>        dest=/home/{{ ansible_ssh_user }}/playpen_source
</span>  - name: Install prerequisite packages for building playpen
</span>    apt:
</span>        pkg={{ item }}
</span>        state=present
</span>    with_items:
</span>        - make
</span>        - pkg-config
</span>        - clang
</span>        - libglib2.0-dev
</span>        - gcc
</span>        - libseccomp-dev
</span>        - libsystemd-dev
</span>  - name: Make Install Playpen
</span>    # Note: Playpen requires Linux 3.8 or later. 3.16 counts as "later".
</span>    command: make install
</span>        chdir=/home/{{ansible_ssh_user }}/playpen_source/
</span>        creates=/usr/local/bin/playpen
</span></code></pre>
As I update it, you can see the changes here</a>. It assumes that you have a hosts file, which groups the hosts which need playpen installed as play</code> and specifies the ansible_ssh_user</code> for them.</p>
My hosts file currently looks like this:</p>
[server]
</span>play ansible_ssh_host=104.236.16.38 ansible_ssh_user=root
</span></code></pre>
I apply the playbook like this:</p>
$ ansible-playbook -i hosts playpen.yml
</span></code></pre>
Setup</h1>
I create a DigitalOcean droplet, $5/month tier, specifying Ubuntu 15.04 LTS and my SSH key.</p>
I add the SSH key to my keyring with ssh-add ~/.ssh/keyname</code>.</p>
I note the droplet's IP address in the DigitalOcean console, and edit my Ansible hosts file to add it.</p>
I attempt to SSH to root at that IP, to check that it's available and to install Python (Ansible's only prerequisite on managed nodes). If I get a remote host identification has changed</code> error, I run ssh-keygen -R <ip></code> to remove the old key. This is a pretty common error when you delete a droplet then recreate another immediately, since DigitalOcean reuses IP addresses.</p>
Run the Playbook</h1>
Once Python is installed on the DigitalOcean droplet and its IP is up to date in my hosts file, I can install Playpen with:</p>
$ ansible-playbook -i hosts playpen.yml
</span></code></pre>
Here's a closer look at what it does. :</p>
---
</span>- hosts: play
</span>  remote_user: root
</span>  tasks:
</span></code></pre>
This is Ansible boilerplate. It specifies which hosts to run the commands on, what user to log into those hosts as, and then starts the list of tasks. The commands I'd have to run by hand every time without Ansible are all listed in the tasks section, whose breakdown follows. :</p>
- name: Create .ssh directory
</span>  file:
</span>      # The ansible_ssh_user is specified in the hosts file
</span>      path=/.ssh/
</span>      state=directory
</span>      owner={{ ansible_ssh_user }}
</span>      group={{ ansible_ssh_user }}
</span></code></pre>
We're going to do a HTTPS clone from GitHub in a couple steps, but first we need to tell the host to recognize GitHub's public SSH key. This is that "The authenticity of host 'whatever' can't be established. Are you sure you want to continue?" message that you get in the habit of blindly typing Y</code> to during the first time you interact with a given server from a clean install... But you can't just type Y</code>, because it's all automated and awesome, so instead we teach the server to recognize GitHub so the host isn't unknown. :</p>
- name: Install known_hosts file
</span>  copy:
</span>      src=known_hosts
</span>      dest=/.ssh/known_hosts
</span>      owner={{ ansible_ssh_user }}
</span>      group={{ ansible_ssh_user }}
</span></code></pre>
The known_hosts</code> file in my repo contains a copy of GitHub's public key. The copy</code> module simply copies a file from the Ansible repo over onto the remote server. Now the server getting the install will recognize GitHub when it tries to clone the Playpen repo, and we can continue. :</p>
- name: Clone Playpen
</span>  git:
</span>      repo=https://github.com/thestinger/playpen.git
</span>      dest=/home/{{ ansible_ssh_user }}/playpen_source
</span></code></pre>
The git</code> module works just like command-line git. This clones the repo, in this case into /home/root/playpen_source</code> because that seems like as good a place as any to put it. :</p>
- name: Install prerequisite packages for building playpen
</span>  apt:
</span>      pkg={{ item }}
</span>      state=present
</span>  with_items:
</span>      - make
</span>      - pkg-config
</span>      - clang
</span>      - libglib2.0-dev
</span>      - gcc
</span>      - libseccomp-dev
</span>      - libsystemd-dev
</span></code></pre>
I constructed the list of dependencies by trial and error: Cloned the repo on a clean Ubuntu install, ran make install</code>, read the errors, installed the dep that threw the error. Over and over. This is how it feels to be a log tailer running in an infinite loop. But the good news is that since I did this, you don't have to! Just copy my list of dependencies if you're lazy, or try it yourself and you'll get the same thing.</p>
The with_items</code> directive in Ansible is pretty neat: it iterates over the list it's handed, and runs the command (in this case, apt</code>) on every item. It's a nice concise way to install a lot of stuff. :</p>
- name: Make Install Playpen
</span>  # Note: Playpen requires Linux 3.8 or later. 3.16 counts as "later".
</span>  command: make install
</span>      chdir=/home/{{ansible_ssh_user }}/playpen_source/
</span>      creates=/usr/local/bin/playpen
</span></code></pre>
This changes directory into /home/root/playpen_source/</code> and runs make install</code>. The bit about creates</code> tells Ansible that /usr/local/bin/playpen</code> gets generated by this command, so if that file exists at the start of the run, Ansible doesn't need to bother re-doing the entire make install</code> step.</p>
And there we have it!</h1>
I'm still pretty new to Ansible, so if you have suggestions for improving this script, file an issue</a> or email me, anything <at> edunham <dot> net</code>!</p>


Display Defaults
2015-06-08T00:00:00+00:00
I've been running Arch on my work laptop and it's pretty much working. However, I have a nice external monitor on my desk, and I keep having to manually configure the output to it with arandr</code>. Here's how I made it configure itself by default when X starts.</p>
First, make the layout</h1>
Arranging monitors is one of the few places where I prefer using a GUI over the command line. I use arandr</code>, a graphical xrandr</code> frontend, to organize my monitors.</p>
Once the displays are set as desired, I go to layout</code> -> save as</code> and save it as ~/.screenlayout/default.sh</code>. That file contains:</p>
#!/bin/sh
</span>xrandr --output VIRTUAL1 --off --output eDP1 --mode 2560x1440 --pos 0x1200
</span>--rotate normal --output DP1 --off --output HDMI2 --mode 1920x1200 --pos
</span>312x0 --rotate normal --output HDMI1 --off --output DP2 --off
</span></code></pre>
Then, execute the command on login</h1>
Since arandr</code> outputs the xrandr</code> command needed to set the layout correctly, I took the lazy way out and just dumped the xrandr</code> incantation from ~/.screenlayout/default.sh</code> into the start of ~/.xinitrc</code>. I might someday come back and replace it with a more sophisticated script to detect whether the second monitor is plugged in, but it's good enough for now.</p>
That's it!</h1>
Yet I had to dig that knowledge out of the forums</a> and xrandr wiki page</a>. But seriously, arandr</code> and .xinitrc</code> are all it takes. It was much easier than I expected.</p>


Playing with Ansible
2015-06-08T00:00:00+00:00
Although I currently expect that I'll end up choosing Salt for work, I've gotten nerdsniped</a> by the apparent simplicity and power of Ansible. Since I'm trying to make a habit of narrating my first encounters with various tools, here's a short novel of 0 through cloning a repo.</p>
Starting Out</h1>
Google hands me the intro doc</a> right away. I noticed this when researcing my config management comparison post as well -- Googling for a given topic results in more useful docs and less marketing with Ansible than CFEngine, Puppet, or Chef.</p>
I install Ansible with yaourt -S ansible</code>. This installs it:</p>
$ ansible --version
</span>ansible 1.9.1
</span>  configured module search path = None
</span></code></pre>
I wonder what that search path info is for... I'm sure the docs will tell me when I need it.</p>
So, I need an /etc/ansible/hosts</code> First, I need a host to put in it. Since I use DigitalOcean for my VPS and have some free credit lying around from the GitHub Education Pack</a>, I go spin up a $5/month droplet to play with. I make sure to check the box to add my SSH key, and add the key to my agent locally, to avoid hassles in the future.</p>
Write /etc/ansible/hosts</code></h1>
I grab the droplet's IP address from the digitalocean console</a>, then I drop it into /etc/ansible/hosts</code>. I wonder for a minute whether there's any way to alias the host or generally refer to it by something more friendly than an IP address, thn I remember that there ways of grouping hosts in order to address them and the tutorial gets to that later on.</p>
Oh yeah, and I did that thing I always do and forgot to open the file in /etc/</code> with sudo, hitting that familiar error:</p>
E45: 'readonly' option is set (add ! to override)
</span></code></pre>
The fix is to write with:</p>
:w !sudo tee %
</span></code></pre>
This says "write the file; shell out to sudo tee</code>, give it the contents of the whole file (%</code>) as an argument" (thanks stackoverflow</a>!)</p>
Ping the Host(s)</h1>
Next tutorial step is ansible all -m ping</code>. I fumbled the typing on the first try, and learned something from the error:</p>
$ ansible all -m png
</span>162.243.134.126 | FAILED => module png not found in configured module paths
</span></code></pre>
This tells me that the -m ping</code> is actually running a module on it. I took a look at the source</a> to see if the ping module is easy to find to read, but I'm not familiar enough to know exactly where to find it. A bit more clicking around suggests it's probably somewhere in here</a>. Oh hey, I found it! It's only in like the second place I checked</a>. It's short and easy to read. The docstring is pretty interesting; the C(path)</code> syntax appears to be some kind of cross-referencing directive. So we make a module with some boilerplate in it, import the basic utils, then blindly call main()</code>. This is totally something I could write if I needed to.</p>
So, when I spell ping</code> correctly, I get:</p>
$ ansible all -m ping
</span>The authenticity of host '162.243.134.126 (162.243.134.126)' can't be
</span>established.
</span>ECDSA key fingerprint is
</span>SHA256:pMoI7FvPgBiFqosItd7rmlHABpiKWBToM/asCOgbAh8.
</span>Are you sure you want to continue connecting (yes/no)?
</span></code></pre>
and then tell it yes. Cool story, standard SSH stuff.</p>
And then... :</p>
$ ansible all -m ping
</span>162.243.134.126 | FAILED => SSH Error: Permission denied
</span>(publickey,password).
</span>while connecting to 162.243.134.126:22
</span>It is sometimes useful to re-run the command using -vvvv, which prints
</span>SSH debug output to help diagnose the issue.
</span></code></pre>
Well okay then, I guess that makes sense considering that the user on the remote box is root and locally I'm edunham. Let's see... I could totally look this up, but let's blindly guess that the -u</code> flag is what's necessary...:</p>
$ ansible all -u root -m ping
</span>162.243.134.126 | success >> {
</span>    "changed": false,
</span>    "ping": "pong"
</span>}
</span></code></pre>
Ah-ha, it behaves like a proper little Unix utility with guessable flags! Good social engineering tactic there, Ansible, making me feel all clever...</p>
Let's install a thing!</h1>
With this initial success, I'm going to deviate from the tutorial a bit and see how Ansible handles trying to install a package from source. I've decided to build a mockup of play.rust-lang.org</code>, since it's one of the SPOFs that's scaring me the worst about Rust's infrastructure at the moment. It's an Arch box that hasn't been updated in a while, running Arch because this tool called playpen</a> comes packaged for Arch but you have to biuld it yourself on Ubuntu. (Yes, the devs were doing the ops work before I got there).</p>
So, I want to install playpen from source. First, I guess, I should probably install Git from the package manager, so that Ansible can clone playpen.</p>
The tutorial's next step is to run an echo command, so I guess I could repurpose it into an apt-get command, but that seems very wrong. Let's see what's next...</p>
The inventory</a> section of the intro comes next, and it explains how to name groups of hosts. Turns out that happens in /etc/ansible/hosts</code> as well... I'd really rather not keep the metadata on how things are grouped up over in /etc/</code>. I feel like it might be better to put the inventory in the config repo... and stackoverflow</a> points out that one can pass the -i</code> flag to specify a custom inventory location. The best practices doc (thank you, Ansible, for having a best practices doc that's actually easy to find) has a section on content organization</a>, which on the one hand doesn't say much about keeping a copy of the hosts file, but on the other hand reassures me by not forbidding it either. I'm just a little bit worried about keeping the grouping metadata of the hosts file from getting lost, since running commands on the correct</em> hosts is a core feature of any good CM tool.</p>
So, change workflow a little:</p>
$ cp /etc/ansible/hosts ~/repos/toy-ansible/hosts
</span>$ cat hosts
</span>[server]
</span>play ansible_ssh_host=162.243.134.126
</span>$ ansible play -u root -m ping -i hosts
</span>play | success >> {
</span>    "changed": false,
</span>    "ping": "pong"
</span>}
</span></code></pre>
Okay, now I can keep this metadata in the repository if I want to. Still not totally sure what best practices will be here for my particular use case; maybe using DNS; maybe storing the exact IPs in a file that never gets committed but leaving hosts as a skeleton to document what goes where if anyone else tries to set up a copy; maybe publishing it and just trusting AWS firewall to do what i tell it to. Because if ansible gets run from or via the bastion, I can leave SSH access just as locked down as it's always been.</p>
So. One PR to fix confusing wording</a> later, I'm back to figuring out the next file to stick in my repo to explain to the Ansible world that this "play" host needs to have Git installed on it.</p>
...okay, that's a lot of /etc/ansible/whatever</code> files and dirs in the tutorial. Maybe I'm supposed to be keeping all of /etc/ansible</code> in Git, rather than my arbitrary repos place? Maybe there's some prefix in an environment variable that I can set so I odn't have to keep passing -i</code> every time?</p>
Okay, tutorial. All this stuff about managing many hosts is cool and I'll come back to it later, but can we get on with the single host case already?</p>
And no, tutorial, I do NOT want to learn about ad-hoc commands before playbooks. Okay, you can shut everything down on Christmas, but that will make people Quite Unhappy. I want to live in a world where special snowflakes and one-offs are always a bad thing, so I'm jumping straight to the playbooks</a> section.</p>
Playbooks</h1>
Ok, so I'm just really bad at recognizing YAML. I claimed elsewhere that I didn't recognize the syntax of Ansible playbooks, which is true, but that's my fault and not theirs.</p>
Their sample playbook makes sense! Let's try writing something of my own... oh wait, I don't know what file extension nor location it belongs with. Fine, guess I've gotta actually keep reading the docs for awhile.</p>
The Next Day</h2>
So, I come back and find the playbooks tutorial</a> conveniently open in a tab. Cool, that's what to put in a playbook... but what do I call it? Dig around for best practices, don't find any, file another bug</a>, call it server.yml</code> because who cares. Guess from the sample playbook how to translate from Yum to Apt.</p>
install Git</h1>
We're going to jump right into trying to install Playpen from source, because I find I learn the most from doing things wrong.</p>
First, I'll try to install Git. From the tutorial, I'm guessing this should work:</p>
---
</span>- hosts: play
</span>  remote_user: root
</span>  tasks:
</span>  - name: install Git
</span>    apt:
</span>        pkg=git
</span>        state=latest
</span></code></pre>
Now, the right thing to do here would be see whether it runs, but I'm going to do the wrong thing and try to figure out how to install Playpen from source as well. Let's just pretend that changing too many things at once is a test of the quality of those inevitable error messages I'm going to induce.</p>
shave a yak and install known_hosts</h1>
There's a post about installing redis</a> on the Google group, and the guy who wrote most of Ansible chimes in with some advice on best practices (though the thread is from 2013, so it may be totally obsolete by now). Looking for the right way to do a git clone</code> through Ansible reveals that it sometimes gets stuck</a>, usually when known_hosts</code> is missing. Looks like I get to learn how to put a file in place, before learning to git clone.</p>
I'm not totally sure if the boilerplate about ansible_ssh_user</code> that I'm copying is actually going to accomplish my goal, but we'll see when I run it. Since I'm edunham</code> on the machine where I'm running Ansible and root</code> on the remote system, it'll be obvious to which user that variable referred.</p>
Now my server.yml</code> looks like this:</p>
---
</span>- hosts: play
</span>  remote_user: root
</span>  tasks:
</span>  - name: install Git
</span>    apt:
</span>        pkg=git
</span>        state=latest
</span>  - name: Install known_hosts file
</span>    copy:
</span>        src=known_hosts
</span>        dest=/home/${ansible_ssh_user}/.ssh/known_hosts
</span>        owner=${ansible_ssh_user}
</span>        group=${ansible_ssh_user}
</span></code></pre>
I then moved my laptop's ~/.ssh/known_hosts</code> to a backup location, tried to pull from github, added the key, and copied the now-much-shorter ~/.ssh/known_hosts</code> into my Ansible repo.</p>
Trial and error and error</h1>
After putting my local backup back into place so my laptop knows more hosts than just GitHub, it's time to see whether Ansible can apply that playbook:</p>
$ ansible-playbook server.yml
</span>
</span>PLAY [play]
</span>*******************************************************************
</span>skipping: no hosts matched
</span>
</span>PLAY RECAP
</span>********************************************************************
</span></code></pre>
Huh, I clearly did something wrong... Oh, that's right, it wouldn't know which host play</code> is because I moved the ansible hosts</code> file into the repo!:</p>
$ ansible-playbook -i hosts server.yml
</span>
</span>PLAY [play]
</span>*******************************************************************
</span>
</span>GATHERING FACTS
</span>***************************************************************
</span>fatal: [play] => SSH Error: Permission denied (publickey,password).
</span>while connecting to 162.243.134.126:22
</span>It is sometimes useful to re-run the command using -vvvv, which
</span>prints SSH debug output to help diagnose the issue.
</span>
</span>TASK: [install Git]
</span>***********************************************************
</span>FATAL: no hosts matched or all hosts have already failed -- aborting
</span>
</span>
</span>PLAY RECAP
</span>********************************************************************
</span>to retry, use: --limit @/home/edunham/server.retry
</span>play                       : ok=0    changed=0 unreachable=1    failed=0
</span></code></pre>
And that would be a failure to add the relevant ssh key after reboot. I add the ssh key to my agent, and this time it works:</p>
$ ansible-playbook -i hosts server.yml
</span>
</span>PLAY [play] *******************************************************************
</span>
</span>GATHERING FACTS ***************************************************************
</span>ok: [play]
</span>
</span>TASK: [install Git] ***********************************************************
</span>changed: [play]
</span>
</span>TASK: [Install known_hosts file] **********************************************
</span>failed: [play] => {"checksum": "926e119d8b84c44f4790c47436967ada72e05ba3", "failed": true}
</span>msg: Destination directory /home/${ansible_ssh_user}/.ssh does not exist
</span>
</span>FATAL: all hosts have already failed -- aborting
</span>
</span>PLAY RECAP ********************************************************************
</span>           to retry, use: --limit @/home/edunham/server.retry
</span>
</span>play                       : ok=2    changed=1    unreachable=0    failed=1
</span></code></pre>
Okay, the ansible_ssh_user</code> stuff was indeed screwed up. If I Google a bit more, I find that I actually skipped something importat in the inventory intro</a>: setting the user for the host. So I add ansible_ssh_user=root</code> to the hosts file, and try again... and it fails again. Even when I substitute root</code> for ${ansible_ssh_user}</code> in the playbook, it fails the same way. Looks like it's not automatically creating the directory for me.</p>
Create a directory</h1>
So, I get to make .ssh</code> by hand. Cool story. So now that task looks like this:</p>
- name: Install known_hosts file
</span>  file:
</span>      path=/home/${ansible_ssh_user}/.ssh/
</span>      state=directory
</span>      owner=${ansible_ssh_user}
</span>      group=${ansible_ssh_user}
</span>  copy:
</span>      src=known_hosts
</span>      dest=/home/${ansible_ssh_user}/.ssh/known_hosts
</span>      owner=${ansible_ssh_user}
</span>      group=${ansible_ssh_user}
</span></code></pre>
for which I'm rewarded with the error ERROR: multiple actions specified in task: 'file' and 'Install known_hosts file'</code>. So I give a separate name to each action, try again, and get a new error:</p>
failed: [play] => {"failed": true, "gid": 0, "group": "root", "mode": "0755",
</span>"owner": "root", "path": "/home/${ansible_ssh_user}", "size": 4096, "state":
</span>"directory", "uid": 0}
</span>msg: chown failed: failed to look up user ${ansible_ssh_user}
</span></code></pre>
At this point, since Google wasn't giving me helpful results in a timely manner, I pinged a friend on IRC and he suggested an alternate syntax, which works:</p>
---
</span>- hosts: play
</span>  remote_user: root
</span>  tasks:
</span>  - name: install Git
</span>    apt:
</span>        pkg=git
</span>        state=latest
</span>  - name: Create .ssh directory
</span>    file:
</span>        path=/home/{{ ansible_ssh_user }}/.ssh/
</span>        state=directory
</span>        owner={{ ansible_ssh_user }}
</span>        group={{ ansible_ssh_user }}
</span>  - name: Install known_hosts file
</span>    copy:
</span>        src=known_hosts
</span>        dest=/home/{{ ansible_ssh_user }}/.ssh/known_hosts
</span>        owner={{ ansible_ssh_user }}
</span>        group={{ ansible_ssh_user }}
</span></code></pre>
I'm glad it works like that, because it feels more like all the other Python-flavored templating that I've touched (Flask, Django, etc.). And it's definitely a Python habit, but I prefer brain damage.</p>
I wonder if it'd be okay to put spaces around the =</code> signs... one :%s/=/ = /g</code> and an ansible-playbook</code> invocation later, I find that adding spaces causes it to fail with an erorr like:</p>
fatal: [play] => a duplicate parameter was found in the argument string ()
</span></code></pre>
Not freaking helpful. But next time I see it, I'll recognize it as too many spaces rather than just being totally confused.</p>
Time to clone playpen</h1>
Now I should, in theory, have the ability to clone a repo from github. Time to test this hypothesis.... I'm starting by reading then copying from the git module docs</a>.</p>
Let's try it:</p>
- name: Clone Playpen
</span>  git:
</span>      repo=git@github.com:thestinger/playpen.git
</span>      dest=/home/{{ ansible_ssh_user }}/playpen_source
</span></code></pre>
And it fails, because it got a GitHub host key that wasn't recognized. I guess there are actually several keys:</p>
# ssh-keygen -r github.com
</span>github.com IN SSHFP 1 1 7bc4945739c3552b9de0260f4524e05329587dea
</span>github.com IN SSHFP 1 2 b040403fc0992ef0bf9144d8aaa25049d8564839821eb592d7338399e456609c
</span>github.com IN SSHFP 2 1 ce76002677a077bf43dabb446a23e86bb127c8c3
</span>github.com IN SSHFP 2 2 858dec00d20192cf334f54c96cccd5900f4540c2975e5d24c8236c255618c10c
</span>github.com IN SSHFP 3 1 261f7e4445378789267eb92c744e9e0d32a5f98d
</span>github.com IN SSHFP 3 2 a4ca08ec5bcf801885aa8b08b5deeb9a51c006988a5814e833f6ac08e81b021f
</span></code></pre>
In investigating which key I actually added to known_hosts</code>, I discover a couple of worrisome things:</p>
First, my initial fumbling managed to create a /home/${ansible_ssh_user}</code> directory, which is empty. Ha ha, I guess? I've manually removed it.</p>
Second, although Ansible claims to have installed the known_hosts</code> file, I can't actually see it. Actually after a bit more digging, it turns out that I was just being dumb at Unix. The system has a /.ssh</code> created by DigitalOcean, which contains only my authorized_keys</code> file. Ansible successfully created the /home/root/.ssh/known_hosts</code> file, but cd</code> as root takes you to /</code> rather than to /home/root</code>. Duh.</p>
Turns out that if I throw the known_hosts</code> file into /.ssh</code>, I can move on to the next error:</p>
failed: [play] => {"cmd": "/usr/bin/git ls-remote '' -h refs/heads/HEAD", "failed": true, "rc": 128}
</span>stderr: Permission denied (publickey).
</span>fatal: Could not read from remote repository.
</span>
</span>Please make sure you have the correct access rights
</span>and the repository exists.
</span>
</span>msg: Permission denied (publickey).
</span>fatal: Could not read from remote repository.
</span>
</span>Please make sure you have the correct access rights
</span>and the repository exists.
</span></code></pre>
Should've been using the HTTPS url, since I don't want to add a public key from this machine to anybody's github account, and I'll never need to push back to the repo from my play server.</p>
Success!</h1>
With that fix, it's now successfully changed! The working playbook looks like this:</p>
---
</span>- hosts: play
</span>  remote_user: root
</span>  tasks:
</span>  - name: install Git
</span>    apt:
</span>        pkg=git
</span>        state=latest
</span>  - name: Create .ssh directory
</span>    file:
</span>        # The ansible_ssh_user is specified in the hosts file
</span>        # For this host I'm using root, so .ssh location is special
</span>        path=/.ssh/
</span>        state=directory
</span>        owner={{ ansible_ssh_user }}
</span>        group={{ ansible_ssh_user }}
</span>  - name: Install known_hosts file
</span>    copy:
</span>        src=known_hosts
</span>        dest=/.ssh/known_hosts
</span>        owner={{ ansible_ssh_user }}
</span>        group={{ ansible_ssh_user }}
</span>  - name: Clone Playpen
</span>    git:
</span>        repo=https://github.com/thestinger/playpen.git
</span>        dest=/home/{{ ansible_ssh_user }}/playpen_source
</span></code></pre>
Next Up</h1>
My next steps will be to install the tools necesassary to build Playpen, and get Ansible to build it. I probably won't keep up the "let's-play" blog style, because it exponentially increases the amount of typing involved, and if you read this far you're either a creepy stalker, thoroughly sick of hearing about my flailing at Ansible, or possibly both.</p>


Configuration Management Comparison
2015-06-05T00:00:00+00:00
Let's just say that it's pretty clear why my team at Mozilla decided to hire an operations specialist when they did. For the infrastructure which supports the Rust programming language, I get the relatively rare (compared to just hacking on an existing deployment) privilege of deploying configuration management from the ground up.</p>
As usual, this means I'm overthinking everything. It seems important to choose the right tool, when in reality it's really only critical to choose a tool that fulfills all the requirements and has a few compelling reasons to be selected over the others.</p>
The System</h1>
The infrastructure has a relatively small number of hosts, and the part which will do the most scaling is deployment of build workers.</p>
Currently, the infrastructure consists of several EC2 instances:</p>

Bastion for SSH access from outside the system</li>
Nginx proxy which routes traffic to the correct hosts</li>
Play, the server which hosts sandboxes</li>
Buildbot leader</li>
Buildbot workers</li>
</ul>
Additional infrastructure includes a stack of Mac Minis under a desk, which perform OSX builds.</p>
The Requirements</h1>
I'm looking for a configuration management solution which supports a variety of platforms:</p>

OSX (preferably multiple versions)</li>
Windows</li>
FreeBSD</li>
Linux</li>
</ul>
For the relevant operating systems, the right configuraiton management solution will Just Work on both AWS and a cross-platform virtual machine and/or container solution. Containers would be lighter weight for contributors running Linux to test infrastructure changes on, but I think it's important to make sure that people running OSX or Windows are able to play with a toy copy of the infrastructure as well.</p>
The right configuration management solution will be available under an open source license, and I'll also be assessing the product's maturity, implementation language, and project culture.</p>
A good solution will balance security with openness. There's some information which is "secret" in that it allows access to a given instance of the infrastructure, and will be unique per instance. It's important that others could spin up their own copies of our infrastructure if they desired, but equally important that only authorized people can access the instance associated with our domain and signing key. This means that a good configuration management solution will make it easy to separate "secret" data from the rest.</p>
As well as making sure that each solution checks the above boxes, the solution's docs should make it easy to find out:</p>

How are commands distributed to hosts (master or masterless?)</li>
How will a repository of configurations be structured?</li>
How do I install a package from the package manager, create a file, and start a service?</li>
How do I install a package from source?</li>
How do I get a report of which files were changed when the tool last ran?</li>
</ul>
The Contenders</h1>
I'll be looking at 5 of the most popular open source configuration management solutions currently available: Ansible, CFengine, Chef, Puppet, and Salt.</p>
Is It Open Source?</h1>
Ansible's license</a> is GPLv3.</p>
CFEngine's license</a> can be GPLv3 or Commercial Open Source License (COSL) depending on the user's preference.</p>
Chef's license</a> is Apache v2.</p>
Puppet's license</a> is Apache v2.</p>
Salt's license</a> is Apache v2.</p>
Age, Language, and Community</h1>
</th> Ansible</th> CFEngine</th> Chef</th> Puppet</th> Salt</th></tr></thead>

Age</td> 3yrs</td> 22yrs</td> 6yrs</td> 10yrs</td> 4yrs</td></tr>
Language</td> Python</td> C</td> Ruby</td> Ruby</td> Python</td></tr>
People</td> 1,060</td> 71</td> 385</td> 376</td> 1,123</td></tr>
Issues</td> 535/4371</td> (Redmine)</td> 337/589</td> (Jira)</td> 2645/6384</td></tr>
% Open</td> 10.9%</td> 40.3%</td> 36.4%</td> 9.4%</td> 29.3%</td></tr>
Commits</td> 14,366</td> 12,387</td> 12,721</td> 20,210</td> 54,144</td></tr>
% by 1</td> 18.6%</td> 20.6%</td> 15.4%</td> 17.7%</td> 13.3%</td></tr>
% by 6</td> 34.2%</td> 53%</td> 39.2%</td> 37.4%</td> 34.2%</td></tr>
</tbody></table>
The "age" is how long from when it was founded to 2015, based on the "first release" dates found on wikipedia</a>.</p>
The "language" is what that Wikipedia page reports it being implemented in, confirmed by examining the GitHub language stats.</p>
The "people" stat is the number of unique contributors that GitHub identifies as having contributed to the main tree (the same repos as the licenses were linked from above).</p>
The "issues" stat is the number of open vs the number of closed issues in their GitHub issue tracker, if that's what they're using. Otherwise, I noted what issue tracking tool they're using instead.</p>
Percent open is the number of open issues divided by the total number of issues (open and closed). As far as I could tell, the CFEngine Redmine</a> appears to have 754 open issues of 1871 issues total, and the Puppet Jira</a> appears to have 1,260 issues open out of 13475 total.</p>
Of the total commits reported by GitHub, I used the contributor graphs to calculate what percentage of the commits were by the top 1 most prolific contributor, and by the top 6. I had no mathematically compelling reason to choose 6 as the threshhold, but it tends to encompass people who've joined later in the project's lifecycle and become prolific contributors as well as those who were there from the very beginning.</p>
Note</p>
There's an obvious spectrum of workflows contrasting CFEngine, Chef/Puppet, and Salt/Ansible. The newer tools are clearly using a more open development process and successfully getting contributions from more people. I'm not sure exactly what the numbers on open tickets say about a project's workflow, but they're interesting to look at.</p>
It was interesting to examine the GitHub contribution graphs for these projects, because they offer a visual representation of the ebb and flow of contributions that people have put into a given project. The puppet graph</a> tells a story of lak starting the project then leaving and hlindberg taking over. The ansible graph</a> shows a similar situation, in which mpdehaan left around October, as jimi-c transitioned in to take their place. The salt graph</a> is quite different, in that thatch45 has been slowly and steadily contributing from the very beginning of the project but hasn't moved on in the way that the original authors of other tools did.</p>
Cross-Platform?</h1>
All the tools I'm looking at support Linux for both the server (if their model has a server) and the client machines. The real question is whether they'll support Mac, Windows, and BSD clients gracefully and in a well-documented manner.</p>
Ansible:</strong> Works on OSX if Xcode and Python are installed, and can support Windows using Powershell instead of SSH. Works on FreeBSD. First hits were blogs, though official docs are easy to find. Provides a module for EC2 support.</p>
CFEngine:</strong> Can be installed on Mac with a Homebrew recipe. Community edition (the free kind) can be installed with Cygwin after a bit of fiddling around with dependencies; commercial version supports Windows out of the box. Works on FreeBSD. First hits were blogs and sales pitches for enterprise edition. Has a demo for EC2, so I guess that means it works?</p>
Chef:</strong> They provide an installer</a> which allegedly Just Works. The installer and some docs were the first hits when I searched. Provides knife ec2</code> plugin.</p>
Puppet:</strong> Has official support on Mac and Windows, and looks like good community support on FreeBSD. Official docs were the first hits for Mac and Windows, and Puppet-specific community forum was first hit for FreeBSD. First hit for EC2 support looks like a sales pitch, though their auto-generated AMIs</a> thing looks neat.</p>
Salt:</strong> Has official support for Windows, Mac, and FreeBSD. Mac support has options of Homebrew, MacPorts, and pip. For Windows installation, they provide an exe. Same section of the docs was first hit to all 3 searches. "Salt Cloud" offers an ec2</code> provider.</p>
Secrets and Security</h1>
On the whole, any configuration management will result in a more secure system than none. Most vulnerabilities come from running old, unpatched versions of common utilities with known bugs, and config management makes it easy to keep systems up to date. However, any program that you run on a server can itself have bugs which introduce vulnerabilities.</p>
Ansible</strong>: To my inexpert sensibilities, the agentless model (you don't run an Ansible daemon on each managed machine, unlike the other CM tools) seems like it could limit the impact of an error in the tool itself.</p>
For keeping certain variables secret, you can keep them in another file</a> and add it to your .gitignore</code>. There's also a vault playbook</a> to automate encrypting those files which contain sensitive information.</p>
CFEngine</strong>: The best practices manual basically says get thee to a security policy</a>, which doesn't really help. Googling didn't help either, so I asked some colleagues, who responded by asking whether "don't use CFEngine" was an option. If there exists a best practice, it's so inconvenient to find that it's relatively unhelpful.</p>
Chef</strong>: Chef supports encrypted data bags</a>, which can then only be edited via knife or the management console. There are also a variety of other options</a>. An interesting solution is citadel</a>, which uses AWS as a trusted third party to control which nodes get access to secrets, but in turn requires the entire infrastructure to be AWS-based. Although relying solely on AWS would be possible at this point in the Rust infrastructure's development, I'm reluctant to needlessly commit us to sticking with it in the future.</p>
Puppet</strong>: The hiera-gpg</a> and hiera-eyaml</a> tools result in sufficiently secure files that major open source projects like Apache's infrastructure</a> are comfortable with publishing them.</p>
Salt</strong>: Salt uses pillars</a> to expose data to target minions. There's also a gpg renderer</a> for encrypting data to be stored in source control, much like all the other modern encryption solutions I'm examining.</p>
The Paradigms</h2>
Here's the elevator pitch for what each tool purports to do, and how they do it.</p>
Ansible</h1>
You write a YAML description of your inventory, or which hosts should be available. Then you describe tasks to perform in each role, and write Playbooks which map between hosts and roles. You can then run Ansible from any machine (there's no dedicated master, nor daemon on the machines being managed) and it uses SSH to remote into the nodes and execute the commands which all those YAML files described.</p>
Because of its simplicity, Ansible claims fewer consistency guarantees than other tools, and provides minimal debugging data about files that it changed.</p>
CFEngine</h1>
When CFEngine was first created, configuration management as we know it today did not exist. As the history page</a> explains, by 2003 the code base had morphed into something that nobody fully understood, so it was rewritten into CFEngine 3. In a nutshell, CFEngine allows you to write promises that describe the desired state of a machine. You can read more at the CFEngine 3 quickstart</a>, but be warned that the abstract yet condescending tone of the piece makes reading it feel like trying to learn Haskell.</p>
Chef</h1>
You write Ruby to write recipes describing everything necessary to configure a system, then gather those recipes into cookbooks. A community tool, foodcritic</a>, is available to test cookbooks for common errors and stylistic enforcement. Cookbooks also have run-lists, which specify the order in which recipes are applied.</p>
Chef-client runs on every host being managed, and queries the chef-server to determine which changes should be applied.</p>
Puppet</h1>
You describe the desired state of your machines in Puppet's DSL, nodes request information from the puppetmaster, the master provides the information, then the nodes run an agent which applies whatever changes are necessary to bring the machine into the described state. The puppet architecture</a> is at the highest level pretty similar to CFEngine's, with the massive advantage that you don't have to go learn promise theory in order to understand it.</p>
Salt</h1>
Salt can be run masterless on each minion, but in production is run from a master. You write .sls</code> files describing the desired state of each minion, which can optionally use data out of pillars</a>. I don't recognize the language used to describe salt states as being a standard I've worked with before, but it's simple and blatantly obvious.</p>
My Opinions</h2>
Ansible</strong>: The minimalism and simplicity of Ansible seem seductive, but the same traits that make it easy to learn will make it less adapted to handling the edge cases which inevitably emerge at scale.</p>
I really like the idea of Ansible and will consider switching to it once the community tools surrounding it are more mature. Googling and asking other users has failed to turn up any good way to report exactly what changes to a file Ansible overwrote when it ran. Although I don't need such fine-grained reporting right away, the fact that they're missing indicates that building nice-to-have features into my configuration management and monitoring later on would require spending a lot of time hacking on the tool itself.</p>
CFEngine</strong>: I worked extensively with CFEngine 2 at the OSU Open Source Lab, and my resulting confidence that CFEngine 3 really can't be as hard as it looks is the only reason I reread that quickstart a few more times until it started making sense. Considering the other options available, I think it'd be needlessly cruel to point a novice contributor to the infrastructure at these docs and say "first, learn the tool".</p>
Although CFEngine is far better than no configuration management at all, it can't compete with the other tools on this list for simplicity, documentation quality, and ease of collaborating on configurations written in it.</p>
Chef</strong>: I spent a lot of time modifying Chef configs for established systems during my time at Urban Airship, and on the whole their infrastructure automation workflow was great. Looking back at that success in more detail, I'm pretty sure that it resulted from using any</em> modern configuration management in conjunction with appropriate levels of automation-inspiring "laziness", rather than from any trait unique to Chef.</p>
It's a little tempting to take the tool that's good-enough and familiar, but there were enough annoyances associated with it (starting with the need to go learn Ruby) that I'm willing to set it aside in favor of simpler, more contributable alternatives.</p>
Puppet</strong>: It is really hard for me to make an "unbiased" comparison of Puppet to Salt, because the local Portland tech scene includes the headquarters of Puppet Labs and I'm friends with the authors of the puppet book</a>. This means that when I run into problems with Puppet and complain on IRC, I tend to get immediate answers from expert users. However, other contributors to our infrastructure wouldn't necessarily have this advantage. In my experience Googling for answers to Puppet best practices questions, the results are comparably "we want to sell you a solution" to the CFEngine and Chef docs.</p>
Salt</strong>: Servo is already using salt</a> to manage its infrastructure, and I'd like to eventually end up with Rust and Servo using the same configuration management solution. I'm more comfortable reading Python than Ruby, and of the models that rely on a master server, Salt is the newest, least commercial-feeling, and most straightforward to learn.</p>
Salt vs Ansible</h2>
So now I have a problem: Two great tools, either of which would be astronomically better than the current state.</p>
When I look farther ahead at automating the Rust infrastructure, I anticipate that I'll be doing a lot of work with parallelizing buildbot runs (to speed things up and also prepare for an eventual migration to TaskCluster</a> once its features catch up to our needs). There's a lot of overhead involved in setting up a host as a buildbot worker, compiling and setting up depenencies for the test suite, so it would be nice to pick a solution that makes it easy to build containers or AMIs.</p>
It looks like both salt</a> and ansible</a> play nicely with Packer, which would be a good choice for constructing build workers with all the dependencies already in place.</p>
Fortunately, there's a blog post by missingm</a>, which links repos that perform identical tasks using both Salt and Ansible. I agree with the blog post's conclusion that the Ansible playbook is easier to read than the Salt states, but the real challenge is to determine how hard each type of configuration will be to write well.</p>


Recording Screencasts on Arch
2015-06-01T00:00:00+00:00
Today I learned that there's a trick to getting sound to work on Arch using recordmydesktop</a>.</p>
The Issue</h1>
yaourt -S recordmydesktop
</span>recordmydesktop test.ogv
</span></code></pre>
Was met with the classic error:</p>
Couldn't open PCM device hw:0,0
</span>Error while opening/configuring soundcard hw:0,0
</span>Try running with the --no-sound or specify a correct device.
</span></code></pre>
Troubleshooting</h1>
I tried passing all the devices from /dev/snd</code>, and none of them worked.</p>
To see whether a graphical UI might install some silent dependency, I installed qt-recordmydesktop</code> and ran it. This failed to record in the same way as before, but an advanced settings tab mentioned that jackd</code> was not running.</p>
I then manually tried to start jack</a> by invoking jackd</code> at the command line, and hit a useful error message emitted by the sanity check</a>.</p>
The Fix</h1>
recordmydesktop</code> was unable to work out of the box because the user as whom I invoked it was not a member of the audio group:</p>
usermod -a -G audio username
</span></code></pre>
made the error disappear, and now recordmydesktop</code> works with the default device.</p>
Epilogue</h1>
My first successful recording, test.ogv, is about 5 seconds of me wondering aloud where the program went, cursing at it, and typing loudly to killall recordmydesktop</code>.</p>


Oh, Windows...
2015-05-23T00:00:00+00:00
I got a shiny new Thinkpad X1 Carbon 3rd Gen for my new job. It came with Windows pre-installed. Out of morbid curiosity and willingness to consider giving this shiny new allegedly-less-terrible Win8 thing a chance, I booted it up into the default Windows installation before wiping everything to install arch.</p>
Windows: Hi there! Tell us a bunch of information!</p>
Me: Okay...</p>
Windows: Log in to your Microsoft account!</p>
Me: Can I skip this? I can't skip this? I have no such thing. Let's stick some bogus credentials in and see how it fails.</p>
Windows: Aww, those credentials were wrong. Want to just skip this step?</p>
Me: YES! I would swear that link wasn't there before. Okay... sets up local account</em></p>
Windows: I'm going to reboot now.</p>
Me: Ah, good ol' classic Windows.</p>
Windows: reboots quickly, then screen gradients from color to color while installing apps</em></p>
Me: Whoa, trippy!</p>
Windows: shows relatively familiar-looking Windows-ey desktop</em></p>
Me: Hmm, my skills from middle school might still apply. opens internet exploder, goes to mozilla.com</em></p>
Windows: Here's the mozilla site! Here's a big button for downloading Firefox.</p>
Me: Thank you, Moz, for knowing that's why I'm here.</p>
Windows: You wanna install this? You sure?</p>
Me: Yes, sure I'm sure. Oh, and let's see if we can partition disks politely from that little tool that used to be around here somewhere. clicks the thing where the start menu should be</em></p>
Windows: Hello human, I hear that humans find squares and bright colors aesthetically satisfying! Here are many bright colors, and many squares!</p>
Me: Ogod make it stop... they warned me about this... slams window key</em></p>
Windows: Aww, okay, here's your desktop back.</p>
Me: types "windows 8 how to partition disks" into exploder address bar, since ff is still downloading</em></p>
Windows: Here are some Bing hits for what you want! First one is some disk-partition.com thing.</p>
Me: Okay... reads</em>... windows+r, type diskmanagement.msc</code>. Yay, it's almost like a fake command prompt!</p>
Windows: You have 5 partitions! 1,000MB, 260MB, 217GB Windows, 12.42GB Recovery, and 7GB OEM!</p>
Me: Congrats. Let's delete that recovery stuff; I like living dangerously.</p>
Windows: Nope. Not gonna show you those buttons; you might hurt yourself.</p>
Me: Alrighty then, is there sudo or anything? checks help docs</em></p>
Windows: Haha nope. Also, have an error about how you're trying to run the system updater twice at once, even though you didn't invoke either.</p>
Me: Fine then, have it your way. wipes entire disk and installs Arch Linux</em></p>


Open Infrastructure
2015-05-20T00:00:00+00:00
My New Ops Job</h1>
Next week, I'm starting work as the only DevOps Engineer on the Mozilla Research team. While I've held a variety of superficially similar jobs in the past few years, this one offers a special opportunity to apply the values I appreciate as a software developer to the infrastructure design and maintenance which represents my work as an ops guy.</p>
Wait, what's DevOps?</h1>
Many of the operations engineers whom I look up to regard the term "DevOps" as an absurd buzzword. I share their antipathy toward its use alongside "cloud" to mean some mysterious panacea of modernization, often spouted by the same technologically illiterate individuals who'd assume a hypervisor to be some sort of new-and-improved sunshade.</p>
Although it could do with a less abused name, the modern systems administration skills that companies are seeking when they open a job req for "DevOps Engineer" deserve to be differentiated from "old-school operations".</p>
In my opinion, the most useful meaning of "DevOps" is to describe to the paradigm in which infrastructure is code. In this sense, the operations team are necessarily developers, since their main goal is to write code which describes the tasks that they would have had to do manually in the olden days. This change of perspective encourages a "DevOps" team to apply best practices which were unattainable back when every server was a special snowflake: Testing, deployment automation, version control of configurations, and code review, to name only a few examples.</p>
There are those who use the vagueness of "DevOps" to justify asking one person (or group) to perform the roles of an entire development team and an entire operations team simultaneously. This could concievably have some benefits, such as improved communication between teams, but it mostly misses the point. I've talked to many of my peers unfortunate enough to end up in the misguided type of "DevOps" role, and learned that they tend to spend about 90% of their time with one of the two hats on and only switch to the other when they have to.</p>
So, Infrastructure is Code...</h1>
Yes! And that's where it all gets interesting. Mozilla's values of openness and the Research team's experimental nature mean that the code I'll write to solve infrastructure problems has no secrets in its design</em>. There are a handful of necessary secrets related to our instantiation</em> of the code, such as the AWS credentials of accounts whose bill Mozilla pays, but they don't have nearly the crippling impact on openness that a design</em> secret would.</p>
At a typical software company, enabling anyone else to precisely duplicate your product is a Really Bad Idea (TM). Any company that sells software has to have some secrets -- people won't usually pay for code that they could get for free. There are a few software-related companies which appear to operate quite profitably without secrets, but examine them more closely and you'll find that they're actually selling a service</em> like support. When a company has some secret in their code's design, concern for protecting it trickles down and metamorphoses into concern for hiding the details of the infrastructure necessary to deploy the code as well.</p>
If you debunk the illusion that sharing their infrastructure will harm the company, operations teams are still extremely reluctant to share their code with others. The typical excuse for opening only a choice library or two, rather than the entire ops codebase, is embarrassment at its quality. Cloaked in the altruistic guise of "I don't think our code would help anyone else, and might lead them astray", the underlying sentiment is one of regret that a given infrastructure project never met those lofty criteria to which its authors aspired. It's the same psychological effect that discourages amateur artists -- we gain the ability to recognize good</em> art (or code) more quickly than the ability to produce it ourselves. And in the case of ops, we often ship code which works but could be greatly improved if only more time was available. Publicly sharing work that one knows was only "good enough" is scary and embarrassing, and I don't fault people for their reluctance to do it.</p>
...So we should be sharing it!</h1>
At least, I think so. I think it's rare and wonderful to have an important infrastructure project with no reason to keep its code secret, and I have hoped from the beginning that whoever gets this Mozilla Research DevOps Engineer job will take advantage of the opportunity. It's entirely a selfish wish, and I have it because I find myself teaching devops concepts to newbies a lot and wanting some really good, real-world repositories to point them at as examples. I was pretty outspoken about these opinions throughout the interview process, so I can only conclude that the team which decided to hire me concurrs.</p>
To recap, being Mozilla Research's very own in-house "DevOps" Engineer is a unicorn of a job because it has cleared the first hurdle that prevents infrastructure-as-code from being open source, and has a running start and a tailwind toward the second. The challenge we've already overcome is that of design</em> secrets, critical company information which would be leaked by sharing even a sanitized version of the infrastructure, and we've overcome it by not having any. The major remaining challenge is that publishing work done in a hurry is scary, because it might be bad and might make me look bad to have it out there with my name on it.</p>
If We Build It, Will They Come?</h1>
That is, in my opinion, the hardest question of all. If I publish the infrastructure that I'm building in real time, warts and all, will anyone benefit?</p>
If the infrastructure works at all, and includes sufficient documentation for anyone to set up their own instance of it if they so desire, then students wishing to learn about "DevOps" in the real world have already benefitted.</p>
Selfishly, though, I'd like to get some contributions back. In my experience with the kinds of code that do and don't get contributors, here are some factors which I think will be important to gaining contributions from people who aren't me:</p>

Relevance. Mozilla Research is a great place to be for this, because Rust is a shiny new programming language with quite a bit of attention on it, so the code that describes its continuous integration and hosting is likely to be of interest to people experimenting with it.</li>
Prestige. People want to get commits into a project that their peers and prospective employers have heard about. With the right promotion, the build infrastructure for Rust and Servo piggyback on the projects' fame.</li>
Prove Someone Wrong. If I offer the infrastructure as a canonical example of how to correctly use the tools it's built on, I will offend and anger the power users and authors of those tools, and they will very likely point out many places where my code needs improvement. Sufficient repetitions of this cycle should cause the code to converge into a state which both works and appears to be a good example of how to use the tools. If it's offered as a real-world example in the tools' tutorials and documentation, that puts a bunch of newbie eyes on the code, and the resulting questions are a great way to identify additional areas for improvement.</li>
</ul>
So in the best case I'll get a bunch of help and advice. In the worse case, where everyone totally ignores it, I'll still be better off because the imagined public eye will push me to document my decisions better and adhere to higher standards of code quality.</p>


Starting Rust: Introduction
2015-05-07T00:00:00+00:00
When discussing my experience (or lack thereof) with the Rust programming language during a recent interview, I learned that experienced developers might be interested in a stream-of-consciousness, "let's-play" type narration of my learning curve through the language.</p>
There's an excrutiatingly detailed account of it below the fold. You've been warned.</p>
Setting</h1>
It's 8:30pm by my internal clock, 9:30pm local time, and I'm chilling in a hotel room in Provo, Utah and feel like doing something useful. So, it's time to learn some Rust!</p>
I travel with a Thinkpad X201, a nice solid box with a small screen, good keyboard, XKCD mouse, and habit of getting a bit toasty when asked to do any heavy lifting.</p>
My Background</h1>
I first touched C about 8 years ago, then started Python and Java 5 years ago, and taught a bit of C++ (though I still won't claim proficiency in it) 4 years ago. I use Python for most things, and C when I have to. In the past 5 years I've played with (but not built anything particularly notable in) a couple Lisps, Haskell, Prolog, Forth, Go, Javascript, and the E derivative Monte.</p>
I skimmed a Rust tutorial the other day and got about as far as noticing that the syntax for guards reminds me of the E paradigms that I've learned through Monte, and overall it looks more or less like any other C-derived syntax. I'm wondering whether it will have arrows in it (their use in Monte has been explained a few times to me, but not in a way that's stuck yet), and if so, whether they'll be explained well.</p>
Finding a Tutorial</h1>
I Google technical topics when logged into my primary account in the same browser. This helps me find pages in my history across devices easily, and gives it a contextual clue that when I say "patchwork" I'm more likely to be solving a Django problem than a quilting one. Results look like this...</p>
</p>
...And I mutter under my breath about someday getting around to writing that lightning talk on naming things. No, I'm not interested in the video game.</p>
I guess I'll try the first hit first, though I'll bet it'll redirect me to the second. If I get frustrated or stuck, the third will always be there, but honestly I'd forget that it exists if I wasn't writing this down right now.</p>
So... https://doc.rust-lang.org/tutorial.html</a> is a deprecation notice in favor of the book. That's cute, I guess. Oh, and scripts are blocked... I guess I'll globally allow rust-lang.org, though I rather wonder why it needs js on a mere redirection page. There, I allowed it, and the fonts got prettier. Was that really necessary?</p>
Ok, that link about the book takes me to https://doc.rust-lang.org/book/</a> . That's... not the book that showed up in Google hits. Okay then? Skim the usual first page.</p>
The Book, Introduction</h1>
Huh, there are 7 sections, starting with the introduction, then a bunch of links. I remember those! I clicked straight through to the "syntax and semantics" thing last time I looked at it, because I was in a huge hurry. Scroll down a little; it has contributing guide.</p>
When last I looked, I assumed the intro ended after the contributing section, but now that I'm being a bit more thorough so I don't look silly in a blog I scroll down further. Looks like there's an introduction thing, picking out the key points of the language.</p>
Do I have to read that intro, or should I just grab one of those 7 links? Do I have to read the links in order? May I jump around? I see now why Knuth directed the readers of TAOCP with psuedocode and flowcharts.</p>
I guess this brief intro thing will answer the "why" questions I might run into later.</p>
So, yeah, it starts with code:</p>
fn main() {
</span>    let mut x = vec!["Hello", "world"];
</span>}
</span></code></pre>
Huh, copying that code caused me to mouse over the code block, which pops up a little arrow-ish icon in the upper right of the block... That arrow takes me to a rather odd URL:</p>
http://play.rust-lang.org/?code=fn%20main%28%29%20{%0A%20%20%20%20let%20mut%20x%20%3D%20vec![%22Hello%22%2C%20%22world%22]%3B%0A}%0A
</span></code></pre>
Whoa, it's some kind of in-browser Rust environment! That's bloody awesome, except "Firefox can't find the server at play.rust-lang.org". Womp womp, sad trombone.</p>
Wait, no, dropping "play.rust-lang.org" into the browser takes me to the https</code> version of the site, which is totally up and happy. Really, no redirect? I sincerely doubt any security implications of redirecting everythign to the one working site</em> would really be that serious compared to the annoyance implications of having the book's examples broken. Maybe I should just fix the book?</p>
And... now when I re-click the arrow thingy, it does redirect. Is it the hotel wifi's fault, or the server? Probably the wifi. Oh well.</p>
So yeah, that syntax... The !</code> is cute; I suppose it might be an assignment but I'm so used to seeing it as negation that I'm tempted to read "vec not hello world"...</p>
Get distracted by macros</h1>
Ok, so it makes a Vec<T></code>. I think I saw that type<thing></code> notation in... was it Go? I'm going to tentatively read it as "a vector of things of type T", though that could be totally wrong.</p>
Oh, the bang was a macro. So... constructors == macros? Close enough. I tend to think of macros as being like the #define</code> in the C preprocessor, since that's where I first met them -- macros in LaTeX (I guess I should've listed that among languages I've touched, but it's really more markup?) also just expand into a chunk of code. Hmm, maybe I should actually click on "macro" where it's blue.</p>
Nope, I should not have clicked on that. It just takes me to https://doc.rust-lang.org/book/macros.html</a> , which is a huge long tangent... skim the first page... ooh, "syntactic abstraction". So this all could still be a fancy way of saying "expand this out into the other thing", just like the other places I've seen it? Scroll, scroll... yeah, okay. That's not hugely more syntax than LaTeX macros, maybe less, and yes there are arrows (or perhaps "oh no"?), and I'd better get back to the bit that tells me what the syntax means before continuing with the part about expanding syntax into other syntax (yo dawg!).</p>
Return to introduction</h1>
Pop that tangent off the stack, and I'm back to introduction. Why am I tired... it's only 9pm laptop time, 10pm local time. Maybe it's all the brain-ing and the typing. Oh well, I'll finish the intro for this post at least.</p>
Ok, now we're learning about mutable and bindings. I know about mutable vs immutable from some obnoxious mistakes early in my Python days; bindings are just a thing... are bindings perfectly equivalent to assignment? I think not; I think I may have gotten at least one lecture at some point on the difference, but I'll just accept that they act sufficiently like assignment for the time being (since bindings</code> isn't a link to anything) and keep going.</p>
And the bit about not needing type annotations is a whole paragraph bragging about the fact that it doesn't force you to type stuff out when the stuff in question is so stupid-simple that even a computer could do it reliably. Well done. (mildly sarcastic applause, but thanks for telling me)</p>
So, stack vs heap stuff. Scroll back up to the code, seek where the heck they're getting the knowledge of where stuff's allocated from... Oh, I wasn't actually expected to know that yet. For some reason the way the words were put together made me feel like it was telling me about some bit of syntax which indicated where things went, but no.</p>
Yes, I grok stacks vs heaps; I have 95% of a bachelor's of Computer Science. No need to tangent down the rust-specifics rabbit hole just now.</p>
Okay, you're finally getting around to the business of ownership. Ok, stuff about when x goes out of scope... do you realize you've never mentioned what scopes are? Luckily I already know that I can usually assume that a scope is represented by a set of {}</code>, and you've given me no reason to suspect things would be different here. the wording of "x goes out of scope" is a bit funny, like x "goes" anywhere once its scope is over... I thought things out of their scopes just weren't there</em>, just went poof. But okay. Yay, no gc, no malloc, hypothetically all solvable by a sufficiently smart compiler. And the determinism is a huge plus for testing anything.</p>
Okay, next line. Ownership system, borrowing... aaand borrowing is blue. Great, another hugely long page of complex stuff I don't know yet... though I guess I should click before judging. Yep, long page. Starting with 'meta'. And I'm supposed to have read the ownership page first, and I'd bet that for that I'm supposed to have finished the introduction...</p>
More Grumbling About Format</h1>
It's the best and worst part of static, linked-up tutorials, how they can express the circular dependencies of knowledge inherent to breaking into any new knowledge domain. It can be a cheat around teaching in what I regard as the right way, which is to handle the circle by spiralling out from 0 knowledge: explain the minimum knowledge of the term inline at first, then go into medium detail next time you come around to it, then greater detail later on. That's how writing a talk (and presenting the info sequantially in real time) forces you to do it, but those constraints are gone on the web. Oh well.</p>
And Back</h1>
Back to the book. Ok, borrowed bindings don't de-allocate when they go out of scope... but it's not the binding</em> that de-allocates, it's the compiler. Such pedantry...</p>
Baby's first Rust error. I see why you introduced borrowing up there, after all. Sure, we pointed the intrinsically immutable y at x, so we kinda lose any benefit from making x mutable or we'd break y's immutability promises... Ok, it's a big wall on how you took away a foot-gun. That was nice of you. I think I'm okay with this.</p>
And a bit more on what that "going out of scope" bit means, quite specifically! That's neat. You even finally get around to mentioning what scopes are, a little.</p>
The End (of the introduction)</h1>
Today I learned that typing all this up takes an awful lot of typing. I think I'll go retrieve one of the icecream bars that I stashed in the hotel fridge's freezer, reward myself for a job thoroughly (albeit less than grammatically) done, and hit section 2 (Getting Started) tomorrow.</p>


Command-line Keyboard Shortcuts
2015-04-16T00:00:00+00:00
Just another installment of "How did I not know that already?"</p>
Backstory</h1>
I came across a copy of the first edition of How Linux Works</a> recently, and have been reading through it to see whether I missed anything important in my hodge-podge of formal and informal open source training.</p>
It's nice to be reassured that I have the right idea for all the basics that it's covered so far. Along the way, it's answering a few of those un-Googleable questions that aren't really a big enough deal to warrant remembering to ask someone more knowledgeable about them.</p>
Today I Learned</h1>
Ctrl+A</code> jumps the cursor to the beginning of the line, and Ctrl+E</code> jumps to the end. (The Home</code> and End</code> keys also work in my shell, but they're slower and more awkward since they require moving one's hands from home row)</p>
Ctrl+W</code> deletes the word in which the cursor is currently located (words being defined as strings separated by spaces), and Ctrl+U</code> deletes the entire line.</p>
Perhaps less usefully, there are also control characters to subsitute for arrow keys, with easy mneumonics for them:</p>
Sequence</th> Arrow</th> Mnemonic</th></tr></thead>

ctrl+B</code></td> left</td> Back</td></tr>
ctrl+F</code></td> right</td> Forward</td></tr>
ctrl+P</code></td> up</td> Previous</td></tr>
ctrl+N</code></td> down</td> Next</td></tr>
</tbody></table>
Other Relevant Facts</h1>
The book hasn't covered these yet, but if the above was new to you, make sure you know these things too:</p>
Ctrl+R</code> enters reverse-i-search</code> mode, in which you type any substring of the command you want. As you type, it will match the most recent command in which the thing you typed appears.</p>
The up and down arrow keys scroll through your command history. Page-up jumps to the first command in your history, and page-down jumps to the last. All that history lives in ~/.bash_history</code> by default, and the HISTFILESIZE</code> and HISTSIZE</code> options in ~/.bashrc</code> configure how much history gets saved.</p>
Ctrl+C</code> kills the running program, Ctrl+D</code> sends the EOF</code> (end-of-file) character, and q</code> quits you out of pretty much any program with :</code> instead of your usualy $</code> prompt.</p>


The life cycle of a conference talk
2015-04-15T00:00:00+00:00
Although lines of code are the most convenient metric of success in open source, I'm proud of those of my own contributions that have taken the form of documentation, teaching, and outreach.</p>
One such skill that I've been working on improving over the past couple years is public speaking, specifically educational talks on a variety of open source related topics. At my current skill level, I'm finding it increasingly important to understand what works and what doesn't in the process that I use when creating my talks. Below the fold will be a few pages of my thinking "out loud" about the topic.</p>
The Idea</h1>
"I should give a talk about that!"</p>
That sentence is my cue to write the idea down somewhere, because otherwise it will disappear. If I have any idea of what the talk's content should include, it goes straight into a .rst</code> file in my slides repo</a>. Whenever I see an interesting article or teaching tool that relates to the topic, I drop the link into that file.</p>
Outline</h1>
The few hours when an idea is shiney and new and still makes sense in the context where it originated are the best time to outline a talk. I think of the outline as the brainstorming phase of story development: concepts from the outline often end up as sections of the final talk, but those which don't fit can easily be thrown out along the way.</p>
The better-developed I can get my outline, the easier it is to continue development of the idea later. It also helps me to write abstracts when submitting the talk to conferences.</p>
Abstract</h1>
The abstract is that single-paragraph summary of a talk which convinces a conference's committee that the talk would be a good fit, and then appears in the program if the talk is selected.</p>
A good abstract is essential to the success of my talks, because it guides the rest of the development process. The title and abstract are how I make sure that only people interested in what I have to say will show up to my talk. They're the promises that the finished slide deck needs to fulfill.</p>
With this in mind, I've identified some common traits in the abstracts which get accepted (vs the many talks I've had rejected from conferences).</p>
A good abstract:</p>

Starts with a hook that will catch the reader's attention</li>
Demonstrates understanding of the audience's needs</li>
Asks questions from the audience's perspective, which the talk will answer</li>
Summarizes why the presenter is qualified to teach the subject</li>
Identifies any prerequisite knowledge that the audience will need to get the most benefit from the talk, or reassures beginners that their questions will be welcome</li>
Ends with a recap of what the audience can expect to learn and an encouragement to attend</li>
</ul>
These factors can make it seem tempting to promise more than the talk can deliver. Don't fall into that trap! If you have any doubt about your slides' ability to fulfill a promise made in the abstract, go write the slides for that part of the talk and make sure they're sufficient before continuing.</p>
Application</h1>
Once I have an outline, abstract, and title for a talk, I'm ready to look for conferences whose attendees might be interested in it.</p>
I use the site Call To Speakers</a> to find new conferences, as well as applying to everything in my local area which overlaps with my knowledge and interests.</p>
I typically submit 2-5 applications to conferences I've spoken at before and enjoyed, or interesting new conferences which:</p>

Occur on dates when I expect to be available</li>
Are near home or offer travel assistance for speakers (though this will be a much less pressing concern with a grownup job than it was during school)</li>
Seem likely to interest the kinds of people whose company I enjoy, such as open source nerds</li>
Have a reasonable code of conduct that shows they're prepared to handle both real and imagined problems in a way that's fair and reasonable</li>
</ul>
When To Apply</h1>
A conference's Call for Proposals (CFP) typically closes 3-9 months before the date of the conference, with speakers getting notified 1-2 months after that. Dates vary for each conference, of course.</p>
After attending and enjoying a conference, I put "check for <conference> CFP dates" on my calendar for a reasonable-sounding time the next year. If there's no update on CFP timing when my calendar emails me a reminder of that event, I move it forward a month and check again then.</p>
Slides</h1>
After a talk gets accepted to a conference for the first time, I convert its outline into a slide deck. The purpose of the slides is twofold:</p>

Slides are a good excuse for speaker notes on my screen, which help reassure me that I haven't forgotten any of the important content</li>
They give the audience something to look at, to reduce their tendency to look at their computers or phones. Ideally, the picture on each slide complements the topic I'm discussing on it, and slides change frequently enough to prevent anyone from getting bored.</li>
</ul>
My favorite slide decks have averaged close to only 1 word per slide. My usual slide decks are pictures with section titles above them, interspersed with the occasional graph or bulleted list.</p>
Google's search features for finding images licensed for reuse are extremely useful in compiling this type of slides. Wikimedia images are especially nice because their pages have the citation pre-written for you.</p>
Rehersal</h1>
This is the part of speaking which I hate most, and tend to procrastinate badly on. When the slides seem about 75% done, I fire up Audacity</a> and record my entire talk as if I'm presenting it at the conference.</p>
It's horrible. It's always horrible. There's a certain quota of horrible-ness in a talk that's guaranteed to be there at first, and the only way to get it out is through rehersals.</p>
I don't take notes while recording the rehersal, since that would mess up my estimate of how long the talk takes, but I do jot down some notes on slide order and which transitions were awkward immediately after finishing the talk.</p>
Analysis</h1>
Okay, I lied about rehersal being the worst part of preparing a talk; that honor actually goes to the part where I have to listen to the recording.</p>
This is the part where I have to be my own poor audience, enticed by the talk's abstract into stumbling into a room and being subjected to an hour of my shrill, obnoxious voice. I don't like this part. But I make it feel productive by taking notes, either on paper or in my slides themselves, on how to make it slightly less terrible.</p>
These improvements tend to fall into three categories:</p>

Ordering and transitions. I rarely get my content into an order that flows optimally on the first try, and have to re-learn the transitions between sections every time I rearrange my slides.</li>
Pruning extraneous content. When I really like a topic, I often want to digress into cool stories every slide, and listening to those digressions is the best way to identify which are useful and which sound irrelevant.</li>
Adding facts and citations to substantiate the claims that I'm likely to make when discussing a topic.</li>
</ul>
Repeat</h1>
Unfortunately, a single rehersal is never enough. The more times I practice a talk, the better it gets. It's two years and a dozen talks into my speaking career, and I still cringe at having to listen to my own voice, so I doubt it'll get better. But it's worth it, to create finished talks that others benefit from and enjoy.</p>
Finishing Touches</h1>
Shortly before my presentation, I put the canonical copy of my slides at talks.edunham.net/conferencename/talkname</code>. I also add that URL to the intro and Q&A slides, so that viewers can find the slides easily.</p>


Culling the GitHub notification spam
2015-04-12T00:00:00+00:00
Long ago, I set up an email filter to make GitHub notifications skip my inbox. This made it easy to ignore the volume of irrelevant notifications that I was getting, and I only noticed them when they got in the way of a search for the one or two useful emails that GitHub sends.</p>
However, it also means that I won't find out if something legitimately notification-worthy happens, such as a new contributor opening a pull request to one of the handful of projects I actually care about.</p>
Here are the steps one can take to improve that signal-to-noise ratio.</p>
Quit Auto-Subscribing</h1>
Go to your notification settings</a> and un-check the automatically watch repositories</strong> box.</p>
</p>
This will prevent you from automatically getting added to as a watcher to new repositories when you're given access to them. It's especially important to my GitHub workflow because I'm in several organizations that grant all members access to new repositories by default, and most of those repos are irrelevant to me.</p>
Assess the Damage</h1>
Head over to the watching repos list</a> to see how many repositories you're automatically getting notifications for. In my case, there are too many to sort through by hand, so the best solution is going to be to bulk unsubscribe.</p>
</p>
Declare Notification Bankrupcy</h1>
</p>
While you're thinning your watch list, head on over to the notifications list</a> and take action on any notifications that need it, or just use the magical Mark all as read</code> button in the upper right.</p>
It's important to get rid of all the spurious notifications generated by repos you shouldn't have been watching in the first place, so that in the future, having unread notifications will mean "something interesting happened" rather than "there's still background noise".</p>
</p>
Re-watch selectively</h1>
There are only about half a dozen repos where I need to take immediate action when an event such as a new PR occurrs.</p>
For each such repo, switch your status to "watching" on its home page.</p>
</p>
Remove those email filters</h1>
The whole point of this exercise was to extract signal from the noise, and to complete that goal, we have to make the signal visible somewhere. For gmail, the filter ignoring everything from github will be somewhere in the filters list</a>.</p>
Maintenance</h1>
To keep seeing only the relevant notifications, respond to irrelevant notification emails by un-watching the repo that they came from.</p>


Removing Blackarch
2015-04-08T00:00:00+00:00
I installed Blackarch</a> on top of my ordinary Arch install for the Oregon CTF</a> a few weeks ago.</p>
Now it's taking up a lot of space on my system and I no longer need it around.</p>
None of the Google queries I've tried have discussed how to get rid of it, and I've had to resort to reading the install script to figure out how to make its repositories go away. Here's what I learned.</p>
First, assuming that you used strap.sh</a> to install all of the blackarch packages like I did, you can uninstall them simply by chopping the package names out of the list of all blackarch packages and instructing Pacman to remove them:</p>
$ pacman -Sl | grep blackarch | cut -f 2 -d " " | xargs sudo pacman -Rn
</span></code></pre>

pacman -Sl</code> searches all of the package databases and lists their contents. The output will be of the form reponame packagename version</code>.</li>
grep blackarch</code> narrows that list to only those packages with blackarch in their titles</li>
cut -f 2 -d " "</code> says "I want only the second field, and fields are delineated by the space character". So far, we've selected the package name of each package whose listing contains the string "blackarch".</li>
xargs</code> executes the given command on each item in its standard in. The pipes (|</code>) each attach stdout of the preceding command to stdin of the following one. Xargs is necessary here because Pacman takes its arguments only on the command line, and does not gracefully handle taking args on stdin.</li>
pacman -Rn</code> says "Remove the given package, and don't worry about backing up its files". I passed the -n</code> flag because one of my main reasons for removing blackarch is to save space on my rather small SSD, so I don't want to keep around a bunch of backups for the configurations of programs I never ended up using.</li>
</ul>
Even after removing all Blackarch packages, the Blackarch repository entry remains in pacman.conf. To remove it, open /etc/pacman.conf</code> and remove the blackarch repo, which will probably be the final two lines of the file. Those lines will look something like this:</p>
[blackarch]                                                                     
</span>Server = http://mirror.team-cymru.org/blackarch/$repo/os/$arch  
</span></code></pre>
And now it's gone. No more packages to update, and no more repo to check every time updates get installed.</p>


SSHFS to public_html
2015-04-06T00:00:00+00:00
Today, someone asked the #osu-lug</code> channel how to mount their public_html</code> directory on OSU's shell server to a location on their local Linux machine using sshfs.</p>
So I basically googled a tutorial</a> and parroted it back at them. However, it would have been even faster to link them to a blog post, so here we go:</p>
What's SSHFS?</h1>
As the name suggests, SSHFS</a> is a tool for using the Secure Shell protocol (SSH</code>) to mount a remote file system (FS</code>). In other words, it lets you treat files on a remote server (such as shell.onid.oregonstate.edu</code>) as if they were on your local machine, so you can use your favorite IDE to work with them and then see your changes immediately appear on the remote server.</p>
Setting Up</h1>
First, install sshfs</code>. On Ubuntu, this looks like sudo apt-get install sshfs</code>. On Arch, yaourt -S sshfs</code>.</p>
Second, check that you're able to SSH to shell. username</code> will be your ONID username throughout this discussion. From a terminal:</p>
$ ssh username@shell.onid.oregonstate.edu
</span></code></pre>
If it accepts your ONID password and logs you in, congratulations! If you get an error, Google it and try the solutions, or check the ONID help docs</a>.</p>
Finally, you need an empty directory to which you wish to mount the remote directory. It's where your files will appear to show up. Let's give it a descriptive name:</p>
$ mkdir ~/shell_public_html
</span></code></pre>
Mount the remote directory</h1>
The syntax for specifying which directory on the remote host you want to mount is roughly the same as for scp</code>, so you can give the directory's path relative to your homedir. In other words, mounting your public_html</code> directory to the local directory shell_public_html</code> will look like this:</p>
$ sshfs username@shell.onid.oregonstate.edu:public_html shell_public_html
</span></code></pre>
And now you can edit files in shell_public_html</code> and their changes will be automatically written to your public_html</code> on shell.onid.oregonstate.edu</code>!</p>
To test changes, see whether the file you edited locally shows up in http://people.oregonstate.edu/~username/</code>.</p>
Other Neat Tricks</h1>
Once you've got that working, you might want to look into:</p>

Automatically mounting the directory at boot</li>
Handling disconnection more gracefully</li>
</ul>
This Ubuntu Forums thread</a> discusses some options to solve those problems.</p>


Optional Arguments in LaTeX Macros
2015-03-30T00:00:00+00:00
As I've mentioned before</a>, I use LaTeX to typeset my resume. I recently found a convenient workaround to handle formatting which differs based on whether or not a macro's argument is present.</p>
The solution to testing whether a given macro argument is empty is buried deep within the LaTeX community forums</a>:</p>
\newcommand{\maybe}[2]{
</span>    \ifx&#1&
</span>        Argument 1 was blank!
</span>    \else
</span>        Argument 1 was not blank.
</span>    \fi
</span>    Argument 2 was #2. 
</span>}
</span></code></pre>
The LaTeX wikibook goes into more detail on the ifx</a> command. It appears that &#1</code> is interpereted as a macro for purposes of equality comparison, then compared against the &</code> empty macro.</p>
The wikibook's TeX category</a> discusses a variety of other if</code> commands. Of particular interest is the ifnum</a> command, which tests whether a value is equal to a given integer.</p>


The Case of the Mixed-Case Tags
2015-03-30T00:00:00+00:00
I accidentally discovered the "feature" that Tinkerer tags are somewhat case-sensitive.</p>
Background</h1>
I sometimes write about LaTeX. On one post, I got lazy and typed the tag for it as latex</code>, while on another, I capitalized it correctly as LaTeX</code>.</p>
To make this post into an example of the behavior, I am tagging it EXAMPLE</code>. I've added the tag example</code> to my first post</a>.</p>
Behavior</h1>
Both the uppercase and lowercase versions of the tag appear in the tags listing in the right sidebar. However, clicking either capitalization will take you to http://edunham.net/tags/example.html</code>. That tag listing will only show the posts tagged with the lowercase</strong> spelling of the tag name being viewed.</p>
The Gotcha</h1>
If all the posts with a given tag use the same capitalization, they will all appear on the tag's page. To see this in action, look at the test tag</a> listing at http://edunham.net/tags/test.html</code>. This is the only post with that tag, so all posts tagged with any capitalization of "test" have the same tag, so all of them show up on that page.</p>
However, if I were to tag another post test</code>, this post would cease appearing on the test tag</a> page. This behavior is probably wrong, but what would one do about it -- preserve case into the tag display page URLs?</p>
In Conclusion</h1>
Be aware that Tinkerer modifies your tag names. Make sure you capitalize each tag the same way on every post, or the posts with a spelling which deviates from all lower-case may get lost.</p>


Fixing Mplayer
2015-03-15T00:00:00+00:00
When trying to fix some errors related to installing BlackArch</a> last week, I made a poorly thought out decision and deleted some kernel modules that I didn't think were necessary. This rendered my system impossible to boot or chroot into for a bit.</p>
The solution to most of that problem was to boot off of a livecd USB, then run the following commands:</p>
mount /dev/sda1 /mnt
</span>pacman -Syu -r /mnt/ linux
</span></code></pre>
At this point, it could not boot because it was missing /sbin/init</code>. This was solved by reinstalling systemd-sysvcompat</code>.</p>
However, even after uninstalling and reinstalling mplayer, I got the following error:</p>
mplayer: error while loading shared libraries: librtmp.so.1: cannot open shared object file: No such file or directory
</span></code></pre>
In my case, Google did not have a useful fix for the error on the first page.</p>
yaourt -Ss librtmp</code> reveals that there's a librtmp0</code> nominally available, although it was unable to build successfully on my system.</p>
The solution was to reinstall ``rtmpdump``</strong>. This restored the necessary kernel modules, and mplayer works now.</p>


hieroglyph2beamer with Pandoc
2015-03-05T00:00:00+00:00
I've played with getting Sphinx to generate PDFs before</a>, and while rst2pdf</code> generates a PDF with all the notes and pictures present, the results aren't as beautifully typeset as I've come to expect from LaTeX.</p>
This made me wonder whether any tool exists to convert Hieroglyph slides into Beamer</a> presentations.</p>
Note</p>
Skip to the end</a> if you just want the code for converting Hieroglyph sources to Beamer slides.</p>
Why not just write LaTeX?</h1>
I sometimes do write LaTeX directly, such as for my resume</a>. LaTeX is the right choice when a project requires:</p>

Extremely precise control of spacing, text size, and text position</li>
Output only to PDF</li>
Infrequent and relatively minor changes</li>
</ul>
However, the tradeoff is that one types a lot of boilerplate LaTeX code for a relatively small amount of output, and the resulting documents are relatively difficult to modify.</p>
I find that slides are a better fit with ReStructuredText's strengths:</p>


Minimal boilerplate</p>

Easy to read as plain text</li>
Fast to modify</li>
Styling is completely divorced from content (HTML vs CSS)</li>
</ul>
</li>

Supported out of the box almost everywhere I need it</p>

Toolchain fits my needs</li>
</ul>
</li>

Powerful enough to describe image scaling and tables</p>

This is my major complaint against Markdown</li>
</ul>
</li>
</ul>
So, I want to keep writing ReStructuredText, but I want the prettiness of Beamer slides. It's time to see whether a tool for this has already been built for me, or whether I'll have to make it myself.</p>
Install Pandoc</h1>
Pip only has pandoc up through 1.0.0a8, which raises an import error if it can't find a version of itself that starts with 1.12 / 1.13. So, I need system pandoc.</p>
Let's see if we can get a system pandoc of a more recent version:</p>
$ yaourt -S pandoc-static
</span></code></pre>
This takes quite a long time because it brings in a bunch of Haskell toolchain stuff, ghc</code> in particular. The amount of memory consumed by Yaourt's attempt to install Pandoc caused the rather disadvantaged laptop I'm currently using to freeze up, but cabal update; cabal install pandoc</code> worked fine.</p>
Make Slides</h1>
~/.cabal/bin/pandoc -t beamer index.rst -o test.pdf
</span></code></pre>
It generates pretty Beamer slides, just like that! If you want to get fancier with it, this blog post</a> is an interesting read.</p>
Comparison to Sphinx's make latexpdf</code></h1>

Pandoc builds slides. make latexpdf</code> doesn't.</li>
</ol>
2) make latexpdf</code> is incredibly loud. Pandoc is silent. Loud, you say? Running it with its default settings yields more lines of output than there were lines in the source it's comiling, and that's just silly.:</p>
$ wc -l index.rst
</span>1053
</span>$ make latexpdf | wc -l
</span>1429
</span></code></pre>
3) Pandoc is better at making the images the right size. With its default settings, each image fills its slide, which is how they look when built by Hieroglyph.</p>


Pandoc goes over twice as fast:</p>
$ time make latexpdf
</span>real    2m14.815s
</span>user    2m12.820s
</span>sys 0m0.783s
</span>
</span>time ~/.cabal/bin/pandoc -t beamer index.rst -o test.pdf
</span>real    0m57.835s
</span>user    0m57.523s
</span>sys 0m0.233s
</span></code></pre>
</li>
</ol>
Allow Page Breaks</h1>
My slides are somewhat atypical in that all the useful content is in the speaker notes, and the slides are filled by pictures for people to look at if they get bored listening to me.</p>
I typically make a PDF copy of my slides for distribution after a talk, so it's nice when readers can see my speaker notes. They're currently where all of my reference URLs and other bonus content reside. Notes are visible by default in Pandoc's beamer output, but they tend to be pushed off the slides by the large images.</p>
Beamer has an allowframebreaks</code> option, which makes sure all your content is visible by wrapping it to as many slides as needed. I learned how to inject a custom preamble from agoldst's lecture notes repo</a>.</p>
First, put the following lines into preamble.tex</code>:</p>
\let\oldframe\frame
</span>\renewcommand\frame[1][allowframebreaks]{\oldframe[#1]}
</span></code></pre>
Then tell Pandoc to include it in the "header"::</p>
$ ~/.cabal/bin/pandoc -t beamer -H preamble.tex -V fontsize=8pt index.rst -o test.pdf
</span></code></pre>
The Pipe Problem</h1>
Sphinx and Hieroglyph allow the use of pipes (|</code>) to force a blank line in html output, which I often use to align my images aesthetically in my slides. However, this use of pipes is neither mentioned in the rst spec</a> nor supported by Pandoc.</p>
Although the functionality of ignoring pipes could probably be implemented as a filter</a>, I decided to take the path of less Haskell-writing and leverage Pandoc's ability to behave like a nice Unix utility.:</p>
sed 's/^|$//' index.rst | ~/.cabal/bin/pandoc -t beamer -f rst -o test.pdf -H preamble.tex
</span></code></pre>
Note that in this command I also tell Pandoc the file type it's converting from, with the -f</code> option.</p>
Shrinking Figures A Bit</h1>
Note</p>
Pandoc tries to emit its output in the format specified by the extension of the output file that you give it. -o test.pdf</code> renders the Beamer slides as a pdf, whereas -o test.tex</code> simply produces LaTeX which could later be rendered into slides. This is helpful for debugging purposes.</p>
The output is getting prettier, but there are still blank slides before some of the images. It appears that the blank slides are only inserted before the images whose height is close to the full height of the slide.</p>
In the LaTeX source of the pandoc-generated beamer slides, figures look something like this:</p>
\begin{figure}[htbp]
</span>\centering
</span>\includegraphics{_drawn/hello.png}
</span>\caption{}  
</span>\end{figure}
</span></code></pre>
I'm using the preamble trick mentioned above to customize each figure. I found the parameters for scaling only those images which would otherwise be too large on stackoverflow</a>. Renewing the \includegraphics</code> command is tricky</a>, because it has an optional parameter, but it can be done with the letltxmacro</code> package. I added these lines to my preamble.tex</code>:</p>
\usepackage{letltxmacro}
</span>\LetLtxMacro{\OldIncludegraphics}{\includegraphics}
</span>\renewcommand{\includegraphics}[2][]{\OldIncludegraphics[width=0.5\textwidth,height=0.5\textheight,keepaspectratio,
</span>#1]{#2}}
</span></code></pre>
Now images are displayed as no larger than half the size of the total text area on a slide.</p>
</span></p>
All Together Now</h2>
The Makefile gets these lines. Remember to use hard tabs, not spaces, because it is a Makefile:</p>
pdf:                                                                            
</span>    sed "s/^|$$//" index.rst | ~/.cabal/bin/pandoc -t beamer -f rst -V fontsize=8pt -o $(BUILDDIR)/slides.pdf -H preamble.tex
</span>    @echo                                                                       
</span>    @echo "Build finished. The PDF is at $(BUILDDIR)/slides.pdf."               
</span></code></pre>
And the preamble.tex</code> gets the following:</p>
% Allow notes to wrap to additional slides.                                     
</span>\let\oldframe\frame                                                             
</span>\renewcommand\frame[1][allowframebreaks]{\oldframe[#1]}                         
</span>
</span>% Prevent spurious blank slides by shrinking images when needed.                
</span>\usepackage{letltxmacro}                                                        
</span>\LetLtxMacro{\OldIncludegraphics}{\includegraphics}                             
</span>\renewcommand{\includegraphics}[2][]{\OldIncludegraphics[width=0.5\textwidth,height=0.5\textheight,keepaspectratio,#1]{#2}}
</span></code></pre>
Now make pdf</code> turns the index.rst</code> of Hieroglyph slides into a relatively beautiful Beamer presentation!</p>


The Magic of Extundelete
2015-03-02T00:00:00+00:00
Last night, I was just falling asleep when my roommate knocked on my door with a Linux problem. Our CS480 assignment was due at midnight and she'd finished the work, then accidentally overwritten the most important file in her program with a malformed tar command.</p>
Seek Possible Backups</h1>
My first action was to go down the list of possible locations where a copy of the file might be:</p>

Did you back it up? (no)</li>
Did you email it to anyone? (no)</li>
Did you upload it to the school servers to test it? (no)</li>
Are you using revision control? (no)</li>
Are there any swap files from your editor? (no)</li>
Is the file in use by any program? (no)</li>
Is there a compiled file that we might be able to decompile into ugly but submit-able source? (no)</li>
</ul>
Out of familiar options, I had her email the professor before the deadline had technically passed to explain the situation. She then phoned her father, who's an engineer at Xerox and used to teach CS480. He ran her through the same questions that I had already asked, mirrored my decision to save the Git is your best friend</em> discussion for a more reasonable hour, and then suggested a tool called extundelete</code>.</p>
Delete vs Overwrite</h1>
To spoil the end of this saga, I found that it was easier to recover a deleted file than an overwritten one. Although the Ubuntu Forums have instructions</a> for recovering an overwritten file, they did not work for me. I suspect it may be because the file was overwritten with the output of tar</code>, rather than being overwritten with an empty file.</p>
Save the Inodes</h1>
Unix-style filesystems keep track of where files live on the disk using sequentially numbered datastructures called index nodes, or inodes</code>. When a file is deleted, the inode's contents aren't actually overwritten until some other program opens a file and needs that inode.</p>
In order to prevent anything from overwriting the inode of a deleted file, stop using the computer immediately after the accidental deletion</strong>.</p>
I'd expect an OS to clean up after itself when politely asked to shut down, and the artefacts I'd like to recover would be regarded as garbage by those cleanup routines. This meant the next step was to unplug the laptop's power supply and then pull out its battery</strong>.</p>
Finally, it is critically important that you do not boot from the drive with the deleted file until after successful file recovery</strong>.</p>
Boot, Carefully, from External Media</h1>
After powering down the laptop, I plugged in an Arch Linux livecd USB stick that I happened to have lying around. On a spare machine, I Googled which function keys would take me into BIOS settings, because the BIOS screen can flash past extremely fast on newer computers.</p>
I then entered the BIOS settings and instructed the laptop to have USB first in its boot priority order. To be super safe, I removed the internal HDD from the list of boot devices entirely.</p>
Unfortunately, my USB stick did not have extundelete</code>, trying to install it from the package manager onto the stick did not work (and probably isn't supposed to), and whatever C++ compiler it shipped with got really grumpy when I tried to build extundelete</code> on the USB stick from source. Life lesson: Roll your recovery CDs before, rather than after, burning them.</p>
However, I have the SSD from my laptop in a USB enclosure and have been booting an older box off of it until my new laptop arrives. I booted my roommate's laptop off of my SSD, and had my entire desktop environment, package manager, and other useful tools available.</p>
Note</p>
The bootloader on a drive accustomed to being in a laptop will default to booting the OS on /dev/sda1</code>. It is very important to edit this entry to have it boot from /dev/sdb1</code> (or the first letter after all the laptop's internal drives are accounted for), since internal drives are seen first and get earlier letters. Remember the part about do not boot from the drive with the deleted file until after successful file recovery</strong>?</p>
Find the Right Partition</h1>
The instructions</a> I was following said to mount the partition containing the overwritten file, so I mounted the sda</code> partitions one at a time until finding which one contained Linux (/dev/sda4</code>). As per the instructions, I removed the overwritten file and then unmounted the partition.</p>
Back Up?</h1>
The partition in question is 500GB, which is bigger than the largest spare HDD that I have lying around. If it was smaller, I would have very very carefully</strong> used dd</code> to clone the partition in case undeletion went wrong.</p>
Run Extundelete</h1>
Ok, by this time it was 1am and I'm still recovering from conference flu, so I did a stupid thing: I tried to run extundelete before unmounting the partition. It caused this error:</p>
The partition should be unmounted to undelete any files without further
</span>data loss.
</span>If the partition is not currently mounted, this message indicates
</span>it was improperly unmounted, and you should run fsck before continuing.
</span>If you decide to continue, extundelete may overwrite some of the deleted
</span>files and make recovering those files impossible.  You should unmount the
</span>file system and check it with fsck before using extundelete.
</span>Would you like to continue? (y/n)
</span></code></pre>
The correct answer is No</strong>. This error only occurs when, as it says, the partition either is mounted or was unmounted wrong.</p>
If the partition was unmounted when you ran extundelete</code> and you got that error, a fsck</code> might help. I'd recommend running fsck -Nn</code> for a dry run, to see what fsck</code> would do if you let it. fsck</code> is useful in cases of filesystem corruption, where a file exists on disk but does not have an inode.</p>
Another possible reason for getting that error would be if you were trying to run extundelete from the same partition as the lost file was on. In that case, you are a terrible person for ignoring literally all the instructions about running extundelete from a livecd, and do not boot from the drive with the deleted file until after successful file recovery</strong>.</p>
I unmounted the partition, and the error did not reappear.</p>
Undelete by File Path</h1>
You probably have to run these commands as root. I elevated to a root shell anyways, because at this point I'm playing with so much fire that avoiding the distraction of constantly typing sudo</code> is a net benefit:</p>
$ extundelete /dev/sda4 --restore-directory home/username/path/to/dir/
</span></code></pre>
Note that extundelete --help</code> explains how the path is relative to the root of the partition, and thus does not need a leading slash.</p>
The restore-directory</code> command didn't actually work (if it'd worked, a bunch of files would have appeared in RECOVERED_FILES/</code>) but it did print a long list of files and directories, their inodes, and whether they were deleted or not:</p>
File name                                       | Inode number | Deleted        
</span>status                                                                          
</span>.                                                 5243979                       
</span>..                                                5243970                       
</span>cs411                                             5375933                       
</span>cs480                                             5252709                       
</span>Essay 1.odt                                       5248187        Deleted        
</span></code></pre>
Digging Around by Inode</h1>
extundelete --help</code> reveals that I can use it to examine individual inodes. Since the missing file is in the cs480</code> directory, my next command examined its contents:</p>
$ extundelete /dev/sda4 --inode 5252709
</span></code></pre>
All this does is give me a list of the directory's contents, with their respective inodes and deletion status:</p>
...
</span>Milestone2_lexical_analyzer                       5375950                       
</span>Milestone4_Code_Generation_Constants              5377172                       
</span>Milestone1_gforth_basics                          5374187                       
</span>Milestone3_parser                                 5375948                       
</span>gforth_problem_6.txt                              5252921        Deleted  
</span>...
</span></code></pre>
The missing file was in the Milestone4</code> directory, so my next command examined it:</p>
$ extundelete /dev/sda4 --inode 5377172
</span></code></pre>
Now we're getting somewhere! The overwritten file was named code_generator.c</code>, and in the output of that last inode listing I have:</p>
code_generator.c                                  5378104        Deleted        
</span>code_generator.o                                  5378085        Deleted        
</span>code_generator.h                                  5378092                       
</span>.code_generator.c.swp                             5378066        Deleted        
</span>.code_generator.c.swx                             5378065        Deleted        
</span>code_generator.c~                                 5378102        Deleted
</span></code></pre>
Restore From Inodes</h1>
I restored each file separately, though looking at the help again I could easily have done them as a list:</p>
$ extundelete /dev/sda4 --restore-inode 5378104,5378066,5378065,5378102
</span></code></pre>
That's to grab the .c</code>, the .swp</code> and .swx</code>, and the .c~</code> files.</p>
Whenever a file is successfully undeleted, it gets saved as RECOVERED_FILES/file.xxxxxx</code> where xxxxxx</code> is its inode number.</p>
Results</h1>
Despite following the instructions</a> as correctly as I could at 1am, the recovered code_generator.c</code> file was still overwritten with the gibberish of the errant tar</code> command.</p>
The code_generator.c~</code> file contained a recent copy of the file which had been overwritten, so my roommate emailed it to the professor.</p>
Postscript: Swap File Recovery Attempt</h1>
But what if the code_generator.c~</code> file hadn't been mysteriously created by whatever utility spits out .c~</code> files?</p>
I tried to recover the swap files with vim -r code_generator.c</code>, and it detected both of them, but I got the error:</p>
E307: .code_generator.c.swx does not look like a Vim swap file
</span></code></pre>
And indeed, it is not a Vim swap file:</p>
$ file .code_generator.c.swx 
</span>.code_generator.c.swx: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped
</span></code></pre>
Something about these files has massively confused Linux. Let's see if they contain anything interesting:</p>
$ strings .code_generator.c.swx | less
</span></code></pre>
It looks a bit like output from running the code generator program interspersed with paths on the filesystem. Whatever this thing is, it's sure not a valid Vim swap file.</p>
To see what valid swap files look like, I forced Vim to generate one by opening a file, typing into it, waiting 4 seconds, then closing that terminal.</p>

The swap file is updated after typing 200 characters or when you have not typed anything for four seconds.</p>
-- (from the Vim docs</a>)</p>
</blockquote>
Unsurprisingly, it's a valid swap file:</p>
$ file .test.c.swp 
</span>.test.c.swp: Vim swap file, version 7.4
</span></code></pre>
and running strings</code> on it prints out the text that I had typed before killing Vim.</p>
The moral of this tangent is that it would not have been possible to recover the .c</code> files solely from their Vim swap files in this case.</p>


Code Coverage in the Monte Tests
2015-02-28T00:00:00+00:00
My schoolwork is particularly bad this weekend, so I'm procrastinating by learning to analyze the test coverage of a moderately complex Python codebase. Specifically, the reference implementation</a> of the Monte programming language</a>. This effort is hampered only slightly by the fact that I've never done much with code coverage tools before.</p>
Nedbat's tool</h1>
Some cursory searching and reading the resulting blogs points out that the Ned Batchelder's coverage</a> is the most-recommended Python code coverage analyser. You run it on your program; it spits out a report of any lines that weren't executed and optionally makes that report into pretty HTML.</p>
Trial --coverage</h1>
I then realized that since Monte's tests are run by Trial, it would probably make more sense to ask Trial to report the coverage statistics. Sure enough, there's a --coverage</code> option in trial --help</code>:</p>
--coverage           Generate coverage information in the coverage file in
</span>                     the directory specified by the temp-directory
</span>                     option.
</span></code></pre>
Wow, that's... almost helpful. This is the only reference to the temp-directory</code> option in the entire help, but maybe the output of running the command will tell me what temp-directory</code> is being used:</p>
$ trial --coverage monte.test.test_runtime    
</span>...
</span>PASSED (successes=231)
</span>Setting coverage directory to /full/path/to/pwd/_trial_temp/coverage.
</span>
</span>$ ls _trial_temp/coverage/ | wc -l
</span>175
</span></code></pre>
Well, it ran whatever coverage</code> function it's using on everything</strong>: the code I wanted coverage for, and all of its dependencies. But the .cover</code> filenames are formatted in a way that makes it easy to tell what they are:</p>
...
</span>monte.ast.cover
</span>...
</span>twisted.web.sux.cover
</span>...
</span></code></pre>
How to read .cover files</h1>
The .cover</code> files looked like scary nonsense at first when I opened them, because they had a bunch of seemingly random numbers and unfamiliar code. I stared at them for a while and realized that a .cover</code> file is exactly the source of the file it reports on, but with each line prefaced by the number of times it was run. (That's what I get for learning this stuff on parts of a codebase that I've barely touched!)</p>
The closest thing I could find to an explanation of the .cover</code> files was a twisted man page</a> in the Mac OSX 10.9 docs. I don't have a man page for Trial on my own system, perhaps because it's installed in a virtualenv rather than through my OS's package manager.</p>
The official Trial docs mention that the coverage tool exists, but provide no help on how to do things with its output.</p>
The moral of the story is that you should grep through the .cover</code> file for instances of >>>></code>, which point at lines that were never run.</p>
Getting Prettier Output</h1>
After reading the Twisted developers arguing about including coverage.py</a> and explaining its use on the mailing list</a>, I realized that Nedbat's coverage</a> is probably the "right answer" after all.</p>
The easiest way to run coverage</code> is to just wrap it around trial:</p>
$ coverage run `which trial` monte.test.test_runtime
</span></code></pre>
This initially has the same problem of testing all the imported files, whose coverage doesn't matter to me right now. Fortunately, coverage</code> has actual documentation on how to use it to omit files in certain directories:</p>
$ coverage run `which trial` monte.test.test_runtime 
</span>$ coverage report --omit=venv/lib/*
</span>$ coverage html -i --omit=venv/lib/*
</span>$ firefox htmlcov/index.html
</span></code></pre>
Note that the -i</code> flag is necessary because of an issue</a> where coverage</code> thinks it should have source for files that it actually shouldn't, in my case:</p>
$ coverage html --omit=v/lib/*
</span>No source for code: '/pymeta_generated_code/pymeta_grammar__CycleRenamer.py'
</span></code></pre>


Hardware Saga, part 1
2015-02-27T00:00:00+00:00
At around 1pm on 2/26/2015, my ThinkPad X230's screen died without warning. I put the laptop away to take a quiz in class, walked home, tried to boot, an it remained black. The LEDs and CPU fan went through their familiar boot process, but the screen remained stubbornly blank.</p>
Below the fold you'll find several pages of excrutiating detail on the situation, because I enjoy writing these things down.</p>
Troubleshooting</h1>
When the screen's power inverter breaks, images are faintly visible on the screen under bright light. However, I looked at it with my phone's flashlight while it booted and saw nothing.</p>
I also unplugged and re-seated the screen's cables, at both the motherboard and the LCD panel, in order to verify that they hadn't come loose if I had bumped the laptop wrong without noticing while carrying it home. Re-seating the cables did not change the screen's behavior; the machine has survived drops from a table to a hardwood floor without any detriment to its performance.</p>
This narrows it down to two components which could be at fault: the cables which run through the hinges and supply power to the screen might be damaged, or the GPU itself could have given out. I tried plugging my VGA port into an external monitor and the monitor showed nothing during boot, although this is not useful evidence since I never ran such a test while the machine was working correctly and thus don't know what should have happened.</p>
Ordering a Successor</h1>
Replacing the motherboard with another of the same quality would cost around $300, and when shopping for replacements, I discovered that I could get a slightly older machine with still-decent specs for less than half that price. Since I have a bit of a Thinkpad collection and have been envying the form factor of a coworker's X201 for quite some time, I ordered an X201 with an i5-m520. It'll ship with 2GB of RAM but I can put in the 8GB from my X230 without trouble.</p>
However, the new machine won't arrive till around March 9th, so I'm having fun persuading my old disk to boot on various systems until then.</p>
Brain Transplant: T60</h1>
I pulled out my X230's SSD and stuck it into the USB enclosure that I usually use for my backup disks. I plugged it into a T60, hit f12 during boot to select the USB HDD as a temporary boot device, and it got as far as the bootloader. However, my Arch kernel was incompatible with the T60's i616 CPU.</p>
Brain Transplant: T440S</h1>
I also have a T440s in my room. It's technically my work computer, so I generally avoid using it for non-work purposes, but "I have an assignment due and need a working laptop" is something of an exception.</p>
Plug in HDD, boot, select boot device, get bootloader. Hit enter to select booting Arch Linux... and a second later, I'm greeted by the Ubuntu login screen. Wat?</p>
Try again. Reboot, select boot device, get bootloader. This time, hit tab on Arch Linux to edit the options. It's trying to boot from /dev/sda1</code>. In all the cases like this that I've encountered, a laptop's internal drive is assigned to sda</code>, and external ones get sdb</code> and higher. Edit the bootloader item to sdb</code> instead of sda</code>, hit enter, and it works!</p>
Log into Arch, and everything works beautifully except the wireless. Am I missing drivers? Let's hope not; if they're missing, it'll be a royal pain to acquire them:</p>
sudo ip link set wlp3s0 up
</span></code></pre>
and there's wifi again!</p>
Brain Transplant: T61</h1>
The next day, I retrieved the hand-me-down T61 that I'd rescued when a roommate asked whether I wanted anything from a big stack of computers destined for recycling.</p>
Its battery is totally shot, and it's missing the hard drive enclosure, but in other regards it's a decently usable machine. It has a good keyboard, physical buttons for the trackpoint, and working wifi, so it wins most of my battle against technology already.</p>
Booting it has been distinctly interesting. Pull 320GB spinny-disk out of x100e, verify that the connections look the same, stuff it into the gaping hole where T61's hdd enclosure belongs. Try to boot. "Operating System Not Found". Okay then.</p>
Plug in bootable Arch USB. Reboot. Pick the x86_64</code> kernel; it boots. Mount partititions of the HDD; they're readable.</p>
Swap the spinny-disk for my X230's SSD. Try to boot. No "operating system not found" error this time; it doesn't see the disk at all. Maybe it's doing a spin check on boot, and the SSD says mu?</p>
Stick the SSD back in its USB enclosure, plug it in, reboot. It boots happily, and I'm typing this from my X230's arch install, booted by the T61, right now.</p>
Next steps will be to stick the X230's old spinny-disk in and see if it's a) recognized, b) bootable. Find disk that's ok to get wiped, and visible to the T61, and install to it. But that takes effort, and the system is usable for now.</p>
If It's Stupid But It Works...</h1>
The annoying part of the T61 setup is that the enclosure end of the USB cable is prone to getting bumped, which means that any process currently in memory keeps going but starting new processes becomes nearly impossible.</p>
My fix for this (it only has to last a week, while my X201 is shipping) is stupid, but it works.</p>
</p>
Packing tape!</p>


The SomeoneWhoCares Hosts File
2015-02-26T00:00:00+00:00
I've been having intermittent problems with Firefox freezing up, pegging a CPU, eating all my memory, and freezing my desktop environment over the past few weeks. The only apparent trend in their occurrence was that they happened while I was active on HabitRPG and other ridiculously Javascript-heavy sites.</p>
I disabled AdBlock Plus, and haven't had a similar freeze since then. I also haven't seen an increase in the amount of annoying ads, even when I visit questionable websites. This is because I took 30 seconds when first setting up my Arch installation to take an incredibly easy and effective action against ads.</p>
The way this works is based on the premise that most intrusive ads come from the same 10,000 or so domains. Wouldn't it be neat if, instead of spending a bunch of resources looking up the domain of an annoying ad and then loading it to barrage your screen with distracting and obnoxious content, your computer just recognized when a page was trying to load an ad and said "nope"?</p>
That's exactly what the SomeoneWhoCares hosts file</a> does. Since computers are not implicitly smart, they have to take dozens of tiny actions in order to load an ad or web page. One of those actions is "look up where to find it", using the Domain Name System (DNS).</p>
You can think of DNS as basically the phone book of the internet. Back when people used land-line phones, it was pretty common to keep a list of useful numbers pinned to the wall near the handset. It's faster to look up a number on a little note nearby than to dig through the phone book trying to find it. A Linux computer keeps a similar short-list of addresses it's expected to need in a file called /etc/hosts</code>.</p>
How does this help with removing ads? Well, back in the landline days, did you ever go through a bad breakup and scratch out your crazy ex's number on the list on the wall, writing "NO DO NOT CALL THEM" in its place? You can use /etc/hosts</code> to make your computer do basically the same thing. By putting advertisers' names in the hosts file with their numbers listed as 127.0.0.1</code>, you do the same thing: If your computer wants to talk to a site that only serves ads, it's told "just call yourself".</p>
If you were really enterprising, you could set up a server to offer pictures of kittens in reply to all queries to localhost, so that you'd get kittens instead of ads. However, I'm lazy, so I'm content to have 404 warnings instead of banner ads.</p>
If you want to use the hosts file too, it's super simple:</p>
$ wget someonewhocares.org/hosts/hosts
</span>$ sudo cat hosts >> /etc/hosts
</span>$ rm hosts
</span></code></pre>
The second line uses sudo because regular users don't have permission to change /etc/hosts. It's safer to concatenate the downloaded hosts file onto the end of the existing hosts file than to overwrite it, so that any hosts settings that you start out with will be preserved.</p>


Making a PDF of Hieroglyph slides
2015-02-24T00:00:00+00:00
I build slides for my presentations with Hieroglyph, which makes beautiful HTML presentations out of raw ReStructuredText. However, HTML slides with my speaker notes in a JavaScript console are not ideal for redistribution.</p>
Try Building via LaTeX</h1>
Note</p>
Skip down to the rst2pdf part if you want an easy and effective solution to this problem. I'm only writing up the LaTeX stuff in case it's useful to people Googling the errors that I hit.</p>
According to the Sphinx tutorial</a>, the recommended way to build PDFs of Sphinx documentation is through the LaTeX builder</a>. According to the docs, this should enable me to simply use make latexpdf</code>.</p>
I had installed texlive-core</code> to do basic PDF rendering, but it did not include certain required packages such as titlesec.sty</code>. At the Arch Wiki's suggestion</a>, I installed texlive-most</code>. The fonts took a long time to download, but it will prevent further problems with missing packages.</p>
With texlive-most</code> installed, it finds the required styles but has a problem with an errant Unicode character in the input file:</p>
! Package inputenc Error: Unicode char \u8: not set up for use with LaTeX.
</span></code></pre>
A thread on StackExchange</a> theorizes that the \u8</code> represents a "no-break space" character, so I think it's safe to strip it out entirely. A rather heavy-handed fix from elsewhere</a> is to forcibly convert the whole file from UTF-8 to ASCII:</p>
iconv -c -f utf-8 -t ascii index.rst > temp && mv temp index.rst 
</span></code></pre>
Note</p>
I previously had that command as iconv -c -f utf-8 -t ascii index.rst > index.rst</code> which actually deletes the entire contents of index.rst. No wonder the pdf wouldn't build :)</p>
After violently taking away its unicode, the file builds a pdf in _build/latex/</code>. But there are a bunch of gratuitous chapter titles, some pages are arbitrarily left blank, and the images' sizes seem totally random --some take up a full page; others are tiny.</p>
Rather than digging further into the LaTeX rabbit hole, I decided to see if there wasn't a more direct way to generate a pdf.</p>
Build Directly to PDF</h1>
According to the Sphinx docs</a>, there exists a builder which converts rst directly into pdf.</p>
To set it up, add the builder in conf.py</code> by adding this line to the end:</p>
extensions += ['rst2pdf.pdfbuilder']
</span></code></pre>
Also add the pdf target to the Makefile:</p>
pdf:                                                                            
</span>    $(SPHINXBUILD) -b pdf $(ALLSPHINXOPTS) $(BUILDDIR)/pdf                      
</span>    @echo                                                                       
</span>    @echo "Build finished. The PDF is in $(BUILDDIR)/pdf."
</span></code></pre>
Remember that Makefiles need hard tabs; a tab expanded into spaces will not be recognized by Make. You can set this behavior up automatically with Vim's filetype plugin, or just type :set noexpandtab</code> when editing Makefiles.</p>
Now make pdf</code> throws the error:</p>
WARNING: "pdf_documents" config value references unknown document contents
</span></code></pre>
This means that rst2pdf</code> wants some extra configuration information. The manual</a> doesn't discuss Sphinx-specific configuration, but a Sphinx issue</a> explains that you need one more line in conf.py</code>:</p>
pdf_documents = [('index', u'sample', u'Sample rst2pdf doc', u'Your Name'),]
</span></code></pre>
After adding the pdf_documents</code> configuration, make pdf</code> generates an oddly styled but entirely legible pdf copy of the presentation's content.</p>


Scale13x Retrospective
2015-02-23T00:00:00+00:00
This year I had the pleasure of attending and speaking at the Southern California Linux Expo. Here's what I observed about the location, the conference's organization, and my own talk.</p>
LA logistics</h1>
Gettting a room in the same hotel as the conference was, as always, an excellent choice. I booked about 3 weeks before the conference with no trouble, but a coworker who tried to book only a week before was redirected to the Mariott next door because the Hilton was full.</p>
The Los Angeles Airpot Hilton is in something of a food desert. Within walking distance there are a Burger King, a Carl's Jr, and a Denny's. The hotel contains a Starbucks, a sit-down restaraunt, and a bar, though the prices are 2 to 3 times what one would consider reasonable outside the combination of conference, hotel, and California. My hotel room contained a coffee maker but no fridge, so I was glad to have brought along non-perishable breakfast and snack foods. All of the fast food joints nearby make a pretty decent attempt at offering salads, non-deep-fried chicken, and other staples of my apathetically health-conscious dietary preferences.</p>
The Conference</h1>
The registration process was not immediately obvious, and the self-checkin kiosks demanded the exact email address and zip code with which one registered for the conference. The line for getting badges and bags after checkin seemed to go quite fast, though.</p>
The expo hall spilled into the smaller rooms across from the hotel's main ballroom as well as out into the hallways, which made it feel almost like two separate halls. The dense population and energy of the expo floor felt similar to OSCON, but SCALE's booths were predominantly actual open-source projects and companies which contribute back upstream, rather than big-name sponsors looking solely to hire. The larger booths, such as HP, were arranged to take several consecutive spaces in an aisle rather than to stand as their own little islands like OSCON's biggest sponsors tend to.</p>
The conference organizers were extremely friendly and helpful, which created a welcoming and friendly atmosphere. I felt that if an incident involving misconduct or bad judgement had happened to occur among attendees, the organizers would have been able to handle it immediately and effectively. There was almost always at least one organizer hanging out in the speaker lounge, often their affable publicist Larry Cafiero, to assist speakers and anwer any questions. This made it easy to get help with problems small enough not to warrant seeking out or interrupting a busier organizer, but large enough to be annoying if left unsolved.</p>
The rooms were often packed to capacity -- I think that there were more people in attendance than the venue was designed to comfortably hold. I doubt that all of the attendees would have fit into even the largest room at once; keynotes were standing room only and relatively early in the morning.</p>
Despite the room size constraints, the layout of the conference was generally good. All of the presentation rooms were centrally located on the second floor and the expo hall was on the ground floor, so it never took more than 2 or 3 minutes to get from one room to another.</p>
My Talk</h1>
I had completely rewritten my Human Hacking talk since I gave it at SeaGL, and ironically several of the strengths that I saw in its previous incarnation became weaknesses here. However, the weaknesses from SeaGL became strengths at SCALE as well.</p>
The good:</p>

Organization at the slide-order level was much better.</li>
Putting more work into my slides meant I had greater trust that they contained all of the important information, and caused me to take fewer tangents.</li>
Extra rehersals of the talk helped me excise most of the anecdotes, keeping only those that were directly relevant to the content. This slight narrowing of scope gave me more time to repeat key points.</li>
I made the audience laugh, and people remained engaged throughout the presentation.</li>
The little Q&A app</a> which Lucy Wyman helped me throw together the day before functioned flawlessly for my interactive demo. And the demo actually did what I expected this time, unlike the ones at SeaGL which got the opposite of the effects I'd hoped for.</li>
My laptop's VGA output Just Worked with the projectors. Nothing in the stack of web browser, window manager, operating system, hardware, cables, and projectors decided to spontaneoulsy quit while I was talking.</li>
</ul>
The bad:</p>

I spoke in the keynote room to an audience of about 40 people (surprisingly large for 3pm on the last day of the conference), and let myself get quite nervous. This caused me to regress back to using a variety of annoying speech habits which I've been making an effort to eliminate.</li>
I was unprepared for the wierd acoustics of the stage (another function of room size). The speakers were in the corners of the room and I was in the middle with a wireless mic, and I experienced perhaps half a second of delay between saying a word and hearing it over the sound system. Before my next talk I'm going to find a way to simulate that effect, and practice with it occurring, because I found it quite distracting and it's probably common in similar audio setups.</li>
Due to last-minute advice about potential network problems and admonitions to use a local copy of my slides, I presented off their second-to-last version since it was the most recent that I had built on my laptop. It had inconsistent capitalization in the slide titles and the wrong URL on the first and last slides.</li>
My transitions between individual slides, and organization of ideas within each slide's topic, were less polished than at SeaGL. This has taught me that the quality of my transitions depends on planning them ahead of time and leaving myself plenty of notes, no matter how often I practice a talk.</li>
</ul>
On the whole, I think that my talk was quite successful, and I get the impression (from body language and watching some people taking notes) that most members of the audience enjoyed at least some parts of the information-packed hour. I'm not particularly looking forward to watching the recordings, because I'm well aware of the verbal bad habits (such as filler words) that I engaged in, but it'll help me target my speaking practice to avoid such problems at future conferences.</p>


Automating Irssi
2015-02-16T00:00:00+00:00
I use Irssi as my IRC client because I don't need to write my own plugins for it, and its documentation is excellent even when you're not sure what questions to ask. If I needed plugins that haven't already been written by others, and didn't mind either reading the entire manual or constantly asking for help to solve simple problems, I would switch to Weechat.</p>
I've configured Irssi to automatically join the networks that I want, authenticate me to services, then join my preferred channels. The Irssi beginner docs</a> are a great reference for client commands to set these functions up, but if I was doing it over again, it would save a lot of keystrokes to simply hand-write the config file which those commands end up generating. Here's how you can do that.</p>
Directory structure</h1>
Irssi loads its configuration from the files in ~/.irssi/</code> by default, and I've never encountered a reason to put the configs elsewhere:</p>
$ tree .irssi
</span>.irssi/
</span>├── config
</span>├── default.theme
</span>├── sasl.auth
</span>└── scripts
</span>    └── autorun
</span>            └── cap_sasl.pl
</span></code></pre>
.irssi/scripts/autorun/cap_sasl.pl</h1>
This is how you automatically authenticate yourself to services before joining channels, so that you'll always have your cloak on.</p>
You can read Freenode's directions</a> on setting this up, or just download cap_sasl.pl</a> and put it into ~/.irssi/scripts/autorun/</code>.</p>
When cap_sasl.pl</code> runs, it will read sasl.auth</code> and authenticate you as specified in that file.</p>
The format of the file is networkname</code>, hard tab</code>, nick</code>, password</code>, hard tab</code>, PLAIN</code>.</p>
If my username/password combo to authenticate with Freenode's nickserv was hello/world, and on work IRC was username/mypass, the sasl.auth</code> file would look like this:</p>
freenode    hello world     PLAIN
</span>work        username mypass PLAIN
</span></code></pre>
.irssi/config</h1>
It's best to edit the config after running Irssi once, since it contains a bunch of defaults for aliases and display features which I won't discuss here.</p>
servers</h2>
Each server has an address, a nickname ("chatnet"), a port, SSL settings, and whether it should autoconnect. You should connect with SSL whenever possible</strong>, because encrypting all your communications with the server makes it safer to send your login credentials in plaintext.</p>
servers = (
</span>  {
</span>    address = "irc.freenode.net";
</span>    chatnet = "freenode";
</span>    port = "6697";
</span>    use_ssl = "yes";
</span>    ssl_verify = "no";
</span>    autoconnect = "yes";
</span>  },
</span>  {
</span>    address = "irc.workplace.com";
</span>    chatnet = "work";
</span>    port = "6697";
</span>    password = "Work Company's Super Seekrit Passphrase";
</span>    use_ssl = "yes";
</span>    ssl_verify = "no";
</span>    autoconnect = "yes";
</span>  }
</span>);
</span></code></pre>
You can add as many servers as you want, following that format.</p>
chatnets</h2>
Reassure Irssi that each network's nickname does belong to an IRC network:</p>
chatnets = {
</span>  freenode = { type = "IRC"; };
</span>  work = { type = "IRC"; };
</span>};
</span></code></pre>
Add a chatnet entry for each network that you refered to when configuring the servers settings.</p>
channels</h2>
Add one of these for each channel that your client should autojoin:</p>
channels = (
</span>  { name = "#osu-lug"; chatnet = "freenode"; autojoin = "yes"; },
</span>  { name = "#osuosl"; chatnet = "freenode"; autojoin = "yes"; },
</span>  { name = "#infra"; chatnet = "work"; autojoin = "yes"; },
</span>);
</span></code></pre>
aliases</h2>
You don't need to touch these to set up autoconnects. To learn more about them, check out this tutorial</a>.</p>
settings</h2>
The real_name</code> variable is where you set your display name, if Irssi was unable to find it in your GECOS information.</p>
hilights</h2>
They'll look like this:</p>
hilights = (
</span>  { text = "edunham"; nick = "yes"; word = "yes"; },
</span>  {
</span>    text = "batsignal";
</span>    color = "%G";
</span>    nick = "yes";
</span>    word = "yes";
</span>    channels = ( "#infra" );
</span>  }
</span>);
</span></code></pre>
This way, whenever anyone says my name in any channel, I'll get hilighted in the default color (purpleish). Whenever someone says "batsignal" in the infra channel, I'll get hilighted in green.</p>
More information about setting hilights with regexes, coloring, and other cool tricks can be found by grepping for hilight</code> in the manual</a>.</p>
ignores</h2>
Your ignores list specifies what message types ("levels") should be ignored in certain channels:</p>
ignores = (
</span>  { level = "JOINS PARTS QUITS NICKS"; channels = ( "#osuosl" )},
</span>  { level = "JOINS PARTS QUITS"; channels = ( "#infra" ) }
</span>);
</span></code></pre>
Section 2 of the manual</a> has more information on levels.</p>
I personally like to turn off joins and parts in larger or busier channels, but leave them on in smaller channels where it's important to notice when a new community member arrives.</p>
windows</h2>
This sets which buffer goes where when Irssi starts. Setting the layout like this is useful to me because I get accustomed to buffers being in particular places, and tend to memorize the numbers of important channels so I know immediately where I've been pinged:</p>
windows = {
</span>  1 = { immortal = "yes"; name = "(status)"; level = "ALL"; };
</span>  2 = {
</span>    items = (
</span>      { type = "CHANNEL";
</span>        chat_type = "IRC";
</span>        name = "#osu-lug";
</span>        tag = "freenode"; }
</span>    );
</span>  };
</span>  3 = {
</span>    items = (
</span>      { type = "CHANNEL";
</span>        chat_type = "IRC";
</span>        name = "#infra";
</span>        tag = "work"; }
</span>    );
</span>  };
</span>  4 = {
</span>    items = (
</span>      { type = "QUERY"; 
</span>        chat_type = "IRC"; 
</span>        name = "chanserv"; 
</span>        tag = "freenode"; }
</span>    );
</span>  };
</span>  5 = {
</span>    items = (
</span>      { type = "CHANNEL";
</span>        chat_type = "IRC";
</span>        name = "#osuosl";
</span>        tag = "freenode"; }
</span>    );
</span>  };
</span>};
</span></code></pre>
What next?</h1>
Read about starting Irssi automatically when your VPS boots</a>.</p>


Starting screen+irssi at boot
2015-02-15T00:00:00+00:00
Installing security updates isn't always enough to apply them; you have to reboot from time to time if you want the benefit of any kernel upgrades that don't use live patching</a>.</p>
There's also a small chance that my hosting provider might reboot my VPS. This has only happened once so far, as part of the Spectre/Meltdown mitigation in early 2018. After that reboot, my IRC setup came back to life without any problems as a result of the configuration that I documented in this post.</p>
In both cases, I would like my IRC client to get back online as soon as it can after reboot.</p>
The Easy Way</h1>
Create a script in your homedir for starting screen with your desired settings:</p>
$ cat ~/startscreen.sh
</span>#! /bin/bash
</span>screen -dmS irc bash -c '/usr/bin/irssi'
</span></code></pre>
The first line of the script tells your system to run it with bash. The screen command says:</p>

-dm</code> tells screen to start in detached mode.</li>
-S irc</code> means that the session will be named irc</code>, which is hard-coded into a couple of scripts I use to automatically connect to it.</li>
-c /usr/bin/irssi</code> tells screen to start irssi in the newly created session.</li>
</ul>
As root, add the following line to /etc/rc.local</code>, substituting the correct value for username</code>:</p>
su - username -c "/home/username/startscreen.sh"
</span></code></pre>
Leave exit 0</code> as the last line in the file.</p>
rc.local</code> is automatically run at the end of each multiuser runlevel, which makes it useful for starting programs at boot.</p>
Without Root</h1>
Let's say that you want to get the above behavior on a system to which you do not have root access.</p>
Hypothetically, one should be able to just drop the line @reboot /usr/bin/screen -dmS irc -c "/usr/bin/irssi"</code> or @reboot /home/username/startscreen.sh</code> into one's crontab (crontab -e</code>), where it would be executed at startup. This works for most scripts, but screen doesn't like to start when it's not attached to a terminal.</p>
The most common workaround for this, in an "if it's stupid but it works, it's not stupid"</em> kind of way, is to SSH to localhost and then start screen. Fortunately, a helpful user named znx typed up the instructions for that process here</a>. Here's a summary, if you're on a new install with no authorized keys set yet:</p>
$ ssh-keygen # accept all defaults
</span>$ cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys
</span>$ vim ~/.ssh/config
</span>    Host lo
</span>        HostKeyAlias localhost
</span>        HostName localhost
</span>        IdentityFile /home/username/.ssh/id_localhost
</span>        User username
</span>        Port 22
</span>$ ssh lo # accept the key this first time
</span>$ crontab -e # add following line and save
</span>    @reboot ssh lo /home/username/startscreen.sh
</span></code></pre>
With systemd</h1>
Check out mythmon's writeup</a> with how to make your own systemd services and configure them to run automatically.</p>
I haven't tried this yet myself, but I expect that any problems repeating it with screen would be most likely to happen when trying to start a detached screen without a terminal, and could be circumvented by the ssh-to-localhost method described above.</p>
What next?</h1>
Having Irssi running is cool, but only really useful if it's connected to the right newtorks and channels. The irssi docs</a> and freenode sasl guide</a> contain all the necessary information.</p>


Résumé Improvement with LaTeX Macros
2015-02-14T00:00:00+00:00
It's career fair season at OSU, which means that I'm taking my annual dive into LaTeX programming in order to improve my résumé.</p>
I'm discovering how powerful and easy to use the macros in LaTeX are. Let's talk about why you might want to convert your résumé to LaTeX, and how to do it easily.</p>
Why typeset your résumé in LaTeX?</h1>
1) Use version control</em>. This is essential for putting back sections that you removed earlier, and for undoing mistakes.</p>
2) Demonstrate technical competence</em>. I want to work at the kind of company that hires applicants who enjoy learning the best tool for an important job, and LaTeX is the best free tool for beautiful typesetting.</p>
3) More prettiness for less frustration</em>. I personally hate fighting with a word processor's GUI, trying to standardize various disparate sections and convince everything to align correctly. When your document's appearance is represented as code, it's also easier to copy and paste other people's fixes for common problems that you encounter.</p>
LaTeX Macros</h1>
If you've only used LaTeX to typeset math assignments and reports, its default formatting has probably worked fine for you.</p>
However, résumés are a special case that requires identical formatting for multiple items. In my own experimentation and experience, résumés look best when all the section names are formatted identically, and when all the items within each section match each other.</p>
Macros</a> are LaTeX's way of solving that standardization problem.</p>
As the WikiBooks article</a> explains, the format of a macro is simple. In the preamble (before \begin{document}</code>), you define the macro's name, number of arguments, and behavior:</p>
\newcommand{\example}[3] {
</span>    \emph{#1} was the first argument\\
</span>    % the double backslash forces a line break
</span>    \textbf{#2} was the second\\
</span>    ``#3'' was the third
</span>}
</span></code></pre>
Then in your document, you call the macro by passing it the number of arguments that were stated in the square brackets:</p>
\example{why}{hello}{there}
</span></code></pre>
The macro simply inserts its text, substituting in its first argument for #1</code>, second argument for #2</code>, and so forth.</p>
</p>
Remember the backslash before the macro name when declaring the newcommand</code>. Since the macro inserted the text as a paragraph, the first line defaults to being indented. To prevent default indentation behavior, put \setlength{\parindent}{0pt}</code> in your document's preamble.</p>
My Résumé Macros</h1>
The most important macros in my résumé are the one which formats a heading, and the one which formats an experience. I include the ulem package for convenient underlining in the titles:</p>
\documentclass{article}
</span>\usepackage[normalem]{ulem}
</span>
</span>\newcommand{\heading}[1]{
</span>    \section*{\uline{\hfill #1}}
</span>}
</span>
</span>\newcommand{\experience}[3]{
</span>    \item[{#1}, \emph{#2}]
</span>    \hfill #3
</span>}
</span></code></pre>
In the body of the document, I can call each macro wherever it's needed:</p>
\begin{document}
</span>\heading{Employment}
</span>\begin{description}
</span>\experience{Awesome Company}
</span>           {My Job Title}
</span>           {date - present}
</span>
</span>    I did a thing and improved the buzzword by 40\%.
</span>
</span>    Collaborated synergistically with diverse enthusiasm.
</span>
</span>\experience{Old Company}
</span>           {Just an Intern}
</span>           {date - date}
</span>
</span>    Wrote code and fixed bugs.
</span>\end{description}
</span>\heading{Education}
</span>\begin{description}
</span>\experience{Academic Institution}
</span>           {The Degree I Got}
</span>           {grad year}
</span>
</span>    I had a GPA!
</span>
</span>    I took classes in subjects!
</span>\end{description}
</span>\end{document}
</span></code></pre>
Note that the indentation is decorative, but the blank lines are necessary to prevent LaTeX from assuming everything should be one line and wrapping only when the line is full.</p>
With macros, LaTeX is actually easy to read! It becomes trivially easy to test changes -- "what if I italicized all the job titles?" can be tested with half a dozen keystrokes, rather than the clumsily wielded flamethrower of global find and replace.</p>
</p>
That's what the code in this post looks like when rendered. Lots of prettiness, not very much code. My own résumé</a> is only slightly more complex -- most of the extra code is dedicated to shrinking the margins and reducing the spacing between lines in order to fit its content correctly onto a single page. (A PDF of my résumé is available here</a>).</p>
More LaTeX Tricks</h1>


To make quotes show up correctly, use ```for opening-quotes and''\</code>` for closing-quotes.</p>
</li>

You can make C++ display more prettily, thanks to a macro from the FAQ</a>. Unfortunately, most of it has to be a single line or LaTeX will add extra spaces:</p>
\newcommand{\CPP}{C\hspace{-.05em}\raisebox{.4ex}{\tiny\bf +}\hspace{-.10em}\raisebox{.4ex}{\tiny\bf +}}
</span></code></pre>
</li>
</ul>


My Second Steps with Javascript
2015-02-13T00:00:00+00:00
There are many things to hate about Javascript. I'm not a fan of the language, and I've been known to laugh at people when they make unqualified claims that it's "good".</p>
However, I sometimes find myself wanting to build toys and share them with everyone who has a web browser. Compared to the security and scaling implications of making my poor little VPS run a bunch of other people's code for them, Javascript becomes the only viable option.</p>
Lazy Sysadmin Use Case</h1>
Sometimes I have an idea which would fit beautifully into a shell script, and is a useful tool or toy that I'd like to share with the world. Unfortunately, "the world" consists primarily of people who can't or won't run a shell script themselves. The lowest common demoninator in this target audience is the ability to use a web browser.</p>
One option would be to write the scripts in Bash or Python or PHP and set up my web server so that it runs the code for visitors to my site, with whatever inputs they give it. Unfortunately, from a security perspective, the problems with this idea are second only to handing over all your banking information to a disinherited royal heir. (Sure, some people do the latter for fun</a>, but they have a lot more free time than I do.)</p>
There are ways that I could have my server run other people's code in a "secure" way, but it's a lot easier to simply automate their browsers to run their calculations for them.</p>
First Project: Novel Word Calculator</h1>
For National Novel Writing Month</a>, crazy people like myself attempt to write 50,000 words in 30 days. We also write a lot of other words, on forums and IRC, to encourage one another and sometimes procrastinate on the 50,000. The NaNoWriMo site lets you track your word count over time, but I had some extra calculations that I found motivational and wanted to share.</p>
Novelists have some overlap with other types of nerd, but on the whole, I couldn't just hand out my custom python script</a> and say "run it!" unless I wanted to become tech support for hundreds of novices trying to install Python on Mac and Windows.</p>
So, to share it with the world (and procrastinate on that 50,000-word deadline looming in the distance), I ported it to javascript</a> and put it in a subdomain of my site</a>.</p>
Second Project: GitHub Stalking Tool</h1>
I wrote a cute but useless little article</a> preaching to the choir about how licenses are cool, and then wanted an easy way to check whether I was following my own advice.</p>
"GitHub has an API, right? JS can do stuff like make API calls, right? This shouldn't be too hard..."</p>
One week and many hours of confusion later, I have a piece of code</a> that works for the originally intended purpose. Learning experiences here have been that CSS is still hard, JS still tends to defy one's expectations, the API has a 60 request per hour rate limit unauthenticated, and One Does Not Simply OAuth. Or more accurately, OAuth requires a server-side program with an application secret, and everything gets exponentially more complicated from there.</p>
Next time I feel like experiencing many hours of frustration, I'll work through the process of adding OAuth (so that you can log into github and click a button to automatically add licenses to unlicensed repos, all through my site), then write up a tutorial. For now, though, it works okay</a>.</p>


Interview on opensource.com
2015-02-12T00:00:00+00:00
As part of my upcoming talk at SCALE</a>, I was interviewed on opensource.com</a>.</p>
Nitish Tiwari interviewed me by email about my talk, then edited my comments into what I feel like ended up as a very nice publication. I appreciate the CC-BY-SA license under which he released the article.</p>


Installing gavrasm on Arch
2015-02-11T00:00:00+00:00
Avra, the AVR assembler that I have been using for my ECE375 assignments, started throwing cryptic error messages such as "PRAGMA directives currently ignored</code>" from an include file which had previously been working fine.</p>
In order to sanity check whether the problem is in my code or the compiler, I installed gavrasm</code>, the assembler which the ECE375 lab website</a> recommends for Mac users.</p>
I wrote this up because I couldn't find a simple, no-extra-thinking-required walkthrough of how to build and install gavrasm</code>.</p>
Broken pkgbuild</h1>
gavrasm is available in the AUR, but its pkgbuild is broken:</p>
$ yaourt -S gavrasm
</span>
</span>==> Downloading gavrasm PKGBUILD from AUR...
</span>x PKGBUILD
</span>Comment by fatmike  (2011-09-30 09:26)
</span>Updated and moved fpc to makedepends.
</span>
</span>gavrasm 3.3-1  (Mon Jul 25 01:43:11 PDT 2011)
</span>( Unsupported package: Potentially dangerous ! )
</span>==> gavrasm dependencies:
</span> - fpc (already installed)
</span>
</span>
</span>==> Continue building gavrasm ? [Y/n]
</span>==> ---------------------------------
</span>==>
</span>==> Building and installing package
</span>==> ERROR: Missing package() function in
</span>/tmp/yaourt-tmp-username/aur-gavrasm/./PKGBUILD
</span>==> ERROR: Makepkg was unable to build gavrasm.
</span>==> Restart building gavrasm ? [y/N]
</span>==> --------------------------------
</span>==>
</span></code></pre>
Installing from Source</h1>
Install fpc, the Free Pascal Compiler:</p>
$ yaourt -S fpc
</span></code></pre>
Download and unzip the source</a>. Files will end up in a directory with a name like Sourcefiles_v3_4</code>, rather than a sensible name like gavrasm</code>, by default.</p>
Copy the language file to its correct location. If you want the English version, this is:</p>
$ cp gavrlang_en.pas gavrlang.pas
</span></code></pre>
If you forget this step, you'll hit an error when compiling:</p>
gavrline.pas(9,28) Fatal: Can't find unit gavrlang used by gavrline
</span></code></pre>
Once you've copied the language file, you're ready to compile:</p>
$ fpc gavrasm.pas
</span></code></pre>
In order to invoke gavrasm from the command line, copy the resulting executable into a directory on your $PATH</code>:</p>
$ sudo cp gavrasm /usr/bin/
</span></code></pre>
That's it! You can now compile assembly with $ gavrasm myfile.asm</code> anywhere on your system!</p>


Mailman and Multiple Addresses
2015-02-10T00:00:00+00:00
"Your message to the list requires moderator approval", replies Mailman when you try to post to your mailing list.</p>
"But I'm on</em> the list!" you complain, then waste a few minutes of your day finding the administrative password and releasing your message.</p>
If you're like me, you probably have several email addresses all forwarding into a single inbox.</p>
When a Mailman list is set to permit only members to post to it, your message will only get through if it happens to be sent from the address with which you're subscribed. Gmail, however, isn't quite smart enough to default to sending to a list from the same address at which you usually recieve that list's mail.</p>
There's an easy workaround if you're a list administrator, which is only slightly less convenient if you don't have admin permissions. The fix is to subscribe the address from which you prefer to send mail to the list, then change its subscription settings so that it's never sent any mail (otherwise you'd end up with multiple copies of each message!).</p>
If you're a list administrator, it takes 4 steps to add all of your alternate addresses. If you're not using admin privileges, it takes 3 steps for each alternate address you want to subscribe.</p>
List Administrator Technique</h1>
1) Navigate to the list's administrative interface and log in. If the list is listname@domain.tld</code>, the Mailman interface probably lives at lists.domain.tld/mailman/admin/listname</code>.</p>
2) Go to the "Mass Subscription" tab under "Membership Management" in the Configuration Categories menu. Or just build the URL of the form lists.domain.tld/mailman/admin/listname/members/add</code>.</p>
3) Set the radio buttons to choose Subscribe, No welcome message, and No notification of new subscription (to avoid spamming other list owners). Then enter the other addresses that you'd like to send mail from, one per line, in the box. Submit the changes.</p>
4) Now navigate to the "Membership List" link, or build the URL lists.domain.tld/mailman/admin/listname/members/list</code>. Find each of your alternate addresses, and check the box in the "nomail [reason]" column on their rows. Remember to leave one of your addresses' nomail box un-checked, or you won't get any mail from the list at all! Use the Submit your Changes button to save the configuration.</p>
Non-Admin Technique</h1>
1) On the list's info page (lists.domain.tld/mailman/listinfo/listname</code>), subscribe your alternate address to the list. (There's also a link in the footer of every email sent to the list.)</p>
2) On the list info page, enter your alternate address in the "Unsubscribe or Edit Options" box down at the bottom, then enter your list password when prompted.</p>
3) Scroll down to the "Mail delivery" setting, and switch the radio button to Disabled. If you choose "Set globally", delivery to your alternate address will be disabled for every subscription that you have in that Mailman instance.</p>


Automating Arch, I3, and Terminator to do the right thing at startup
2015-02-05T00:00:00+00:00
I'm currently on my second clean install of Arch Linux. On the whole I've been glad that I ditched most of my old configuration when I installed Arch to the new SSD I got for my laptop at the beginning of the school year. However, I'd been procrastinating on re-implementing a few things which worked last install without me really knowing which of my many attempts had fixed them. This time, I know what I'm doing and what questions to ask.</p>
Desired Behaviors</h1>

Start ssh-agent automatically at login</li>
Start X automatically at login</li>
Automatically connect to wifi if I'm at home</li>
Open a terminal in workspace 1 and Firefox in workspace 2</li>
</ul>
Preemptive Troubleshooting</h1>
Certain types of error when you're testing commands to auto-run at login will immediately log you out again, with no error message displayed. The easy way to fix these problems is to have at least one other user account on your system with sufficient permissions to edit your account's startup scripts. That way if you end up in such a failure loop, you can just login as them, alter the offending line, and immediately test logging in as yourself again.</p>
If you're testing automatic behaviors at login for the only account on your system and the root account doesn't have its own password and things break, the best ways I've found to get a bad line out of your config files are to either reboot into a recovery mode or boot any old distro off of a USB drive if you happen to have an old install disk sitting around.</p>
Starting programs at login</h1>
The wonderful Arch Wiki has an entire page on how to start X at login</a>.</p>
In summary, just add exec startx</code> to the end of ~/.bash_profile</code>.</p>
ssh-agent</code> is a special snowflake and causes things to fail silently if you try to invoke it along the lines of exec ssh-agent && exec startx</code>. However, the arch wiki</a> comes to the rescue again, citing a ssh-agent tutorial</a> that explains you can run ssh-agent</code> as a wrapper:</p>
exec ssh-agent startx
</span></code></pre>
(which of course goes as the last line of your ~/.bash_profile</code>).</p>
Automatically connecting to one wifi network</h1>
The Arch Wiki's page on netctl</a> has the relevant line buried, as usual, in a huge wall of otherwise-extraneous information.</p>
Note that I've been manually connecting to networks with wifi-menu</code> before, so I already have profiles with saved passwords for the networks to which I wish to autoconnect.</p>
If the SSID is My Network</code>, there's a file in /etc/netctl/</code> with a name something like wlp3s0-My Network</code>.</p>
To make it autoconnect, just give the command:</p>
sudo netctl enable wlp3s0-My\ Network
</span></code></pre>
Note that if the profile ever changes, such as for a change in password, you'll have to netctl reenable</code> it in order to automatically connect.</p>
Automatically connecting to the right network out of several</h1>
The netctl enable</code> stuff works great if the first enabled network is in range when the system boots, but fails and doesn't try others if it isn't. At this point I'm not sure whether it's supposed to be possible to autoconnect to multiple networks with netctl at all.</p>
When I asked some other Arch users how to get netctl to do the multiple networks thing, they reported having used network-manager instead:</p>
yaourt -S networkmanager
</span>yaourt -S network-manager-applet
</span>yaourt -S gnome-keyring
</span>sudo systemctl enable NetworkManager.service
</span>sudo systemctl start NetworkManager.service
</span></code></pre>
The wiki page</a> on networkmanager explains how to set up permissions so users without root can add networks, but that step solves a nonexistant problem for me since I'm the only person who should be changing network configuration on my laptop.</p>
Whenever I need to add a new network, I start nm-applet</code> through dmenu and add the network. When I'm at home or school, the networks I've already added can autoconnect and I don't have to have the nm-applet</code> icon cluttering up my toolbar.</p>
Automatically Starting Stuff in Terminator and I3</h1>
The stuff to start is a terminal with the command to ssh to my VPS and reconnect to my IRC screen, and Firefox.</p>
Since you can't start terminator with a bash alias in the --command</code> argument, and my irc alias is long and contains quotes itself, I've created a custom Terminator profile for launching irc. Either my Google-fu is weak or you can't have Terminator profiles inherit from one another, so I added a new [[irc]]</code> profile to my .config/terminator/config</code> file, with copies of the settings I care about from my default profile:</p>
[[irc]]                                                                       
</span>  scrollbar_position = hidden                                                 
</span>  use_system_font = False                                                     
</span>  cursor_shape = ibeam                                                        
</span>  background_image = None                                                     
</span>  show_titlebar = False                                                       
</span>  color_scheme = green_on_black                                               
</span>  font = Mono 10.5                                                            
</span>  use_custom_command = True                                                   
</span>  exit_action = hold                                                          
</span>  custom_command = ssh -t -p XXXX me@myvps.net "screen -dr irc"
</span></code></pre>
Note that the -t</code> flag to SSH forces it forces it to allocate a psuedo-TTY, preventing the Must be connected to a terminal</code> error which would otherwise occur.</p>
Adding the following lines to the start of ~/.i3/config</code> causes the desired behavior when i3 is started:</p>
exec --no-startup-id i3-msg 'workspace 2; exec firefox'
</span>exec --no-startup-id i3-msg 'workspace 1; exec terminator -p "irc"'
</span></code></pre>
Note that I exec firefox first and terminator second, so that the terminal which needs my VPS password to be entered in order to reconnect to irc is in the active workspace immediately after startup.</p>
As an added bonus, digging these commands out of my old config reminded me how to automatically set a background image: Just add the following to ~/.13/config</code>:</p>
exec feh --bg-scale ~/background.jpg
</span></code></pre>
Making Terminator pick the right font size</h1>
The default font size that Terminator has been using causes my screen to be a little under 160 characters wide. This causes the 80-character lines on which I standardize my writing to wrap annoyingly, and means I have to manually zoom out by one ctrl-minus</code> in each pane when I split my terminal.</p>
The fix is two lines in ~/.config/terminator/config</code>:</p>
[profiles]                                                                      
</span>  [[default]]                                                                   
</span>    ...        
</span>    use_system_font = False                                                     
</span>    font = Mono 10.5
</span></code></pre>
You have to manually override the default use_system_font=True</code> setting for any font changes in your terminator config to apply. After hearing good things</a> about the font Inconsolata, I gave it a try, but found that it looks unpleasantly blurry at the small sizes that I prefer to use. I'm sure there's a setting somewhere to fix that, but my needs are met equally well by switching back to the Mono font as they would be by shaving the font display yak.</p>
The Results</h2>
Based on IRC timestamps, it now takes me roughly 13 seconds and [ctrl+shift+e] + [laptop username] + [laptop password] + [vps password] of typing to kill X, log in, automatically connect to available wifi, and reconnect to IRC. This is an improvement over my previous process.</p>


Please License Your Code
2015-02-04T00:00:00+00:00
Code without a license isn't open source</a>. It isn't free software</a>, either.</p>
Posting your code publicly doesn't inherently apply a license to it.</p>
Adding a license really isn't hard -- it's just a label that clarifies the owner's intent to share it.</p>
Imagine you come into the school cafeteria and there's a plate of cookies just sitting on one of the tables. No sign on it, no indication of what they're for, but they're out there in public. Is it ok to take one?</p>
</p>
If you know the person who brought them, you can just ask if you can have one, and they'll probably say yes. If some stranger brought them, then maybe they're to share -- you can, and many people would, certainly take your chances and have one anyway.</p>
But what if they were put there by someone with malicious intentions and access to unpleasant chemicals, or even just baked by someone well-meaning but with extremely bad kitchen hygiene?</p>
Or maybe the art club has been making realistic cardboard cookies for an upcoming play, or the veterinary students decided to decorate some dog biscuits all festively, or... you get the idea.</p>
You can, and many people would, just take your chances and have a cookie. Most of the time, they're as wholesome and delicious as they look and were put there by a nice person with a clean kitchen who intended to put a sign on them but was just busy and forgot to, or assumed that everyone would know. But every once in a while, that isn't the case, and then the aftermath involves some awkward conversations or serious discomfort or even lawyers.</p>
Applying an appropriate license to your open source code is effectively just putting a label by that plate of tasty solutions you've brought to the internet, saying who they're for.</p>
Note</p>
I am not a lawyer. I am not your lawyer. Please do not try to eat software, regardless of how it's licensed.</p>


20 HabitRPG levels in 5 days
2015-02-02T00:00:00+00:00
I've been having fun with HabitRPG</a> lately, and discovered this One Neat Trick:</p>
Checklists.</p>
Checklists, at least the way I've been using them, are obscenely overpowered. I'll have a medium-difficulty to-do, break it down into a dozen sub-tasks (because that's what I usually do on to-do lists when I'm procrastinating), get them all checked off, and then reap a level and a half worth of XP</a>.</p>
According to that wiki page,</p>

Experience gain is determined by an algorithm from HabitRPG, which tracks how consistently a player completes a given task.</p>
</blockquote>
Cute theory. The thing is, I've never gotten more than maybe 20 XP from a habit or daily (the only types of tasks for which consistency makes sense as a metric), but I regularly get over 100 XP for the completion of a sufficiently detailed checklist.</p>
Annoyingly, the wiki doesn't point out where the relevant algorithm lives in the code. I've asked on IRC about how to find it, but for now, just try this:</p>

Make a to-do of a task that you'll have to do today.</li>
Add a checklist to the task, breaking it down into 8 or more necessary sub-items.</li>
Accomplish your task, check each item, then check the main checkbox beside its title.</li>
PROFIT!!</li>
</ol>
This is also really broken in the amount of damage it does against bosses. I was cleaning my room yesterday with the checklist method and ticked maybe 20 HabitRPG checkboxes (emptied and discarded about 4 largeish cardboard boxes and 2 garbage bags of stuff I no longer need, so I don't think I'm unfairly inflating points here), and look at this:</p>
</p>
That's a lot of damage to the boss. The calculator I used was this awesome site</a>, in case you haven't discovered it yet.</p>
I'm a rogue, with all my points in INT as per the guide</a> on the subject! I'm not supposed to be able to hit things very hard! Sure, I'm level 25 right now with a perfect day</a> bonus from yesterday, but my stats still aren't too great:</p>
</p>
tl;dr</strong>: I scored a bunch of points in a silly game and conclucluded that it must be broken.</p>


Gamifying Adulthood with HabitRPG
2015-02-01T00:00:00+00:00
After creating an account and abandoning it some time before December 2013 (since I have never yet subscribed and yet had a Trapper Santa</a> scroll in my inventory), I have returned to HabitRPG</a>. Here's a quick examination of why I think I left and then came back.</p>
Why Leave?</h1>
The short answer is that I never really "got into it". I didn't have a smartphone when I first created an account in the web UI, and it seemed at the time like using the app would be prerequisite to successful gameplay.</p>
When I first got my phone, I installed the app, but I hadn't touched the web UI in so long that I'd lost track of why</em> the game was fun or rewarding. The app's not super pretty, so when I wasn't motivated to touch it by an understanding of the bigger-picture gameplay, I didn't choose to.</p>
Finally, I started playing with the app at a time when I more or less already had my life under control. I didn't have any recurringly time-wasting habits cutting into productive work that I was late on, my daily schedule worked out quite well with room for everything I wanted to get done, and so forth. Such is the beauty of "grownup" life, with a 9-5 job and not much else going on. At that point, Ingress was my "I need to touch my phone a lot" game of choice, and satisfied my vague desire for quantified/gamified reality.</p>
Why return?</h1>
The game was brought to my attention in passing when I saw a friend playing it a couple months ago, and its usefulness clicked into place for me when I observed how much time I was spending on another online game called dominus</a>. It's an open-source real time strategy type game, and the realtime aspect of gameplay (as opposed to the immsersive, lose-hours-on-end nature of my other bad habits like Minecraft) made me realize how easily and unobtrusively I can fit a 5-minutes-at-a-time game into my workflow.</p>
Now, if only I could turn those little brain breaks into something useful. That's where HabitRPG fits in.</p>
What It Is:</h1>
You should really go do the tutorial and read the wiki</a> for any kind of a comprehensive overview. In a hideously undersized nutshell, HabitRPG is a game where you get items, gold, XP, and mana points by completeing tasks. Tasks give you rewards (or harm you when incomplete) differently based on their type:</p>


Habits</em> are behaviors that you want to get more consistent at. They can either be positive, negative, or have both positive and negative buttons. Consistently plus-ing a habit turns it slowly green then blue; consistently minus-ing it turns it orange then red. Habits stay in your habit list until you delete them, no matter how often they're clicked.</p>


Positive habits are "good" things and have only a +</code> button, which you click when you do them. For example, "eat a vegetable".</li>
Negative habits are things you're trying to avoid doing, and have only a -</code> button. For example, "drinking soda".</li>
Some habits have both +</code> and -</code> buttons. For example, you might put both buttons on "take the stairs instead of the elevator", and click on the +</code> each time you take the stairs and the -</code> each time you take the lift.</li>
</ul>
</blockquote>
</li>

Dailies</em> are things that you want to do once every day, or on certain days of the week. Various game mechanics hurt you when you fail to complete all your daily tasks, or give you advantages when you complete all of them (called getting a perfect day</em>). Dailies stick around until you delete them, and count the streak of how many times in a row you've gotten them.</p>
</li>

To-dos</em> are how you quantify all those one-off things that need to get done. For instance, "Attend CS class" might be a daily for Monday, Wednesday, and Friday, but "Do CS class assignment 1" would be a to-do.</p>
</li>
</ul>
What It's Not</h1>
HabitRPG is not a replacement for Google Calendar. Although recurring "daily" tasks can be set for certain days of the week, its deadlines feature just doesn't get in your face the way a calendar email and notification will.</p>
Although I'll discuss the ways that its game mechanics reduce the likelihood of "do it eventually" tasks getting totally forgotten, I don't find HabitRPG to match Google Calendar's ability to make me do an important, time-sensitive, one-off task like attending an appointment.</p>
It's also not your mom. It can't tell if you're lying to it, and makes no effort to detect or punish "cheating". While this game is a great way to channel a little bit of self-discipline into a lot of results, it cannot magically generate discipline and motivation for you if you start out with zero.</p>
Personal accountability through positive tasks</h1>
Part of the fun of this game is the challenge it presents in terms of learning to hack your own brain. One thing I've noticed is that I have a much easier time being honest on positive tasks than negative ones. For instance, a habit of "make a public commit on GitHub" is easy for me to recognize when I've fulfilled it, and it keeps the experience of using HabitRPG fun -- I'm motivated to open the site and record the action I took, because a good thing (XP and gold) will happen when I do.</p>
Conversely, negative tasks are harder to be accountable for because I know that nothing good will happen if I bother to go so far out of my way as to open a HabitRPG tab and wait for it to load, and maybe if I just go do something else I'll accidentally forget to record it, and... you get the idea.</p>
Good recurrence model</h1>
Daily tasks fill a niche that Google Calendar and paper checklists can't handle as well. There are many tasks that need to be done sometime</em> every day, but it doesn't matter exactly when. It's hard to add something to a calendar without picking a particular time slot in advance (then easy to put it off till a different day if that time slot gets taken by something higher-priority). Paper to-do lists are fine for one-off tasks without a set time, but remembering to check a list every day (motivated only by fear of failure</em>, rather than hope for reward</em>) requires more willpower than I usually have to spare.</p>
Partying</h1>
Getting together with friends to form a party and fight monsters is delightfully entertaining, and provides an element of reality to the game. Now, if I skip a daily task, it's not only hurting me but it's harming the cute little avatars of people I care about as well.</p>
It's also an incentive against cheating, since it'll be obvious to everyone you're with if you level too fast. (Though "too fast" is quite relative -- a min/maxed Rogue between levels 10 and 20, in the hands of a checklist fanatic, can gain 2 or 3 levels on a productive day.)</p>
Complexity</h1>
One trait of Minecraft which addicted me to the game was its complexity -- a player who's read the entire wiki and memorized a lot of recipes and facts about the game can do interesting tricks which amaze less educated players, and enjoy teaching their friends. I feel like so far, the subtleties of HabitRPG's game mechanics are hitting a similar sweet spot between being easy to start and difficult to master.</p>
Moderation</h1>
Sometimes I'll get going on a project or task which makes me feel productive and then repeat it, when another less-satisfying task is more important to do at that particular time. Habits in HabitRPG fight this tendency by implementing a rule of diminishing returns: The more often you've done a particular habit, the less gold and XP you'll get each time you engage in it.</p>
Notes-To-Self</h1>
I used to jot down ideas of things to try someday in a notebook; then I went through a phase of jotting them down in a git repository; then upon acquiring a smartphone I had moderate success with Google Keep. The problem with all of those media was that my cool ideas would get lost in them -- at times of boredom, I'd forget that my lists were out there.</p>
Now, I'm throwing such ideas onto my to-do list in HabitRPG and they gradually percolate up into being the least onerous challenge available.</p>
Since HabitRPG is a low-guilt site to check when I'm procrastinating on more useful things, it tend to remind me of my "that might be neat" ideas right when I have free cycles to actually do something about them. At the opposite extreme, Google Tasks puts a little list in my inbox and reminds me of things to do when I'm trying to compose email, and have the least free time right then.</p>
Emergent Task Weighting</h1>
I've noticed a tendency to throw the same unpleasant task into my to-do queue multiple times. or break it down into increasingly longer checklists in order to procrastinate on actually accomplishing it. In a classic to-do list this would just cause me to experience more guilt and fear about the task, but in HabitRPG, it slowly increases the amount of gold and XP that the task will be worth when I finally get it done and check off all of the associated items.</p>
Although some perfectionists might consider multiple occurrences of a task in the to-do list to be cheating, I find that letting them pile up is a powerful technique for convincing myself to finally get it over with and do the task.</p>
Improvements</h1>
It would be nice if the interface displayed the gold, XP, and mana values for each task's completion on the title of the task.</p>
The to-do list has a button for moving an item to the top, but no button for moving an item to the bottom. Since newly created to-do items default to being at the top of the list, it can get tedious trying to keep older and more important items near the surface.</p>
The interface which keeps track of daily tasks defaults to showing all tasks (even those which aren't due on the current day), so every time you reload the page you have to click "due" to see only the relevant items. It would be nice if the interface remembered whether you prefer seeing "due" or "all" items.</p>
The app lacks a few features which I find essential to normal gameplay, namely the ability to show only due dailies, and the ability to allocate points into various attributes when you level.</p>


The Trouble with Toctrees
2015-02-01T00:00:00+00:00
It's a couple weeks and nearly a dozen posts into this Tinkerer experiment, I'm mostly delighted with it. It's fulfilling its original promise of "write RST, push button, get pretty blog"... Mostly. There's one problem, though. I</a> constantly</a> forget</a> to add master.rst when committing.</p>
What happens when you forget to git add master.rst</code> before committing a new post? The post's identifier, which got added to the toctree in master.rst</code> when you said tinker --post</code>, doesn't end up in the master.rst</code> on the remote server! When you build a sphinx site, only those pages which exist in some toctree somewhere get links to them. This is the correct behavior, for if Sphinx tried to auto-detect new pages, it might not make the right guess about which toctree to include them in.</p>
However, the inconvenient consequence is that master.rst</code> has to be manually added to source control. I didn't touch it to create the post, so I forget that I have to touch it to successfully push my changes. I firmly believe that files created by my tools and their sources created by me are fundamentally different -- the former should never live in version control and the latter should (almost) always. master.rst</code>, with its changes coming from a side effect of my manually creating a file, has been in a nasty limbo between the two states.</p>
The solution is a quick and dirty Python script</a> which regenerates master.rst</code> whenever it's called. The call to this script adds a single line to the chain of tools which constitutes my "push button, get website update" command, and will save me many cumulative hours of annoyance and troubleshooting over the lifetime of my blog.</p>
Since automatic sorting would always put my pages into alphabetical order, I've solved the problem before it occurrs by renaming my pages into the form 01about.rst</code>, and configuring the script to sort them by casting the first 2 characters of the page name to an int.</p>
For my use case, remembering to preface my few pages' filenames with the place in the list where they belong is a small price to pay for privilege of not feeling stupid due to forgetting to add an unrelated file every time I push a blog update.</p>


Arbitrary Python versions on Flip
2015-01-31T00:00:00+00:00
A question in the #osu-lug</code> channel about running python 2.7 on a school server (which only has python 2.6.6) made me realize I should know how to do that but have never tried it. Stackoverflow</a> provides instructions for solving a similar problem, so I'm testing them out to make sure they work for Python 2.7.7 and Python 3.</p>
The Setup</h1>
At Oregon State University, students in the engineering department have access to a trio of servers which, for most purposes, are interchangeable. If you're running a screen session for IRC, it's important to log in to the same flip (flip1.engr.oregonstate.edu</code>, flip2.engr.oregonstate.edu</code>, or flip3.engr.oregonstate.edu</code>) every time because the session exists in that particular server's memory. If you only care about stuff stored to disk, you can just ssh to flip.engr.oregonstate.edu</code> to be round-robined onto an available server. Your NFS home directory is shared between all the flip servers, which is why the same files are also available whenever you log in with your engineering account to a Windows, Linux, or Mac computer in the university's computer.</p>
All 3 flips are identical; I happen to be on flip1 at the start of this tutorial:</p>
-bash-4.1$ hostname
</span>flip1.engr.oregonstate.edu
</span></code></pre>
It's running CentOS 6.6 and has Python 2.6.6 and Virtualenv 1.10.1 installed:</p>
$ python --version
</span>Python 2.6.6
</span>$ virtualenv --version
</span>1.10.1
</span>$ cat /etc/redhat-release
</span>CentOS release 6.6 (Final)
</span></code></pre>
tl;dr:</strong> Just ssh</code> to youronid@flip.engr.oregonstate.edu</code>. You only need to specify flip{1-3} if you're doing IRC stuff.</p>
Install Python locally</h1>
On Flip, run the following commands:</p>
wget http://www.python.org/ftp/python/2.7.7/Python-2.7.7.tgz
</span>tar -zxvf Python-2.7.7.tgz
</span>
</span># then you wait. it may take a few minutes.
</span>
</span>cd Python-2.7.7
</span>mkdir ~/python27
</span>
</span># all 3 of the following steps may take awhile.
</span>./configure --prefix=$HOME/python27
</span>make
</span>make install
</span></code></pre>
Use Python2.7.7</h1>
Instead of invoking python</code> and getting the old version from your path, specify which python you want to run your program with:</p>
~/python27/bin/python my_awesome_file.py
</span></code></pre>
Or Make a Virtualenv</h1>
$ virtualenv -p ~/python27/bin/python venv27
</span>$ source venv27/bin/activate
</span>
</span>(venv27)-bash-4.1$ python --version
</span>Python 2.7.7
</span></code></pre>
Errors</h1>
If you get a permission denied error, check that files actually ended up in your ~/python27 directory. If the directory is empty, run make; make install</code> in your Python-2.7.7 directory.</p>
Re-installing solved the failure-to-install issue for me.</p>
Trying it with Python 3, too</h2>
Repeating the same process with Python 3.4.2 yields the error:</p>
ImportError: No module named '_collections_abc'
</span>ERROR: The executable venv3/bin/python3 is not functioning
</span>ERROR: It thinks sys.prefix is '/nfs/stak/students/d/dunhame' (should be '/nfs/stak/students/d/dunhame/venv3')
</span>ERROR: virtualenv is not compatible with this system or executable
</span></code></pre>
So, Flip's virtualenv version is too old. Fortunately, Python3 shipped with pyvenv:</p>
$ python3/bin/pyvenv-3.4 venv3
</span>$ source venv3/bin/activate
</span>(venv3)$ python --version
</span>Python 3.4.2
</span></code></pre>
For anyone who wants to copy and paste, the commands for installing Python 3.4.2 are:</p>
wget https://www.python.org/ftp/python/3.4.2/Python-3.4.2.tgz
</span>tar -zxvf Python-3.4.2.tgz
</span>cd Python-3.4.2
</span>mkdir ~/python3
</span>./configure --prefix=$HOME/python3
</span>make
</span>make install
</span></code></pre>


Vim: Open file with cursor at the end
2015-01-29T00:00:00+00:00
As part of a recent quest to automate everything and learn more Vim tricks, I've been identifying patterns in my use of the editor and attempting to get them done with fewer keystrokes.</p>
The Problem</h1>
I have a file containing notes which I edit several times a day. This takes quite a few keystrokes:</p>
vim path/to/where/the/file/is/file.txt
</span><shift> G $
</span>i
</span></code></pre>
And then I'm ready to type. Wouldn't it be nice to have a short alias to do that for me instead?</p>
Choose Your Own Adventure</h1>
You can either keep reading to find out the neat but ultimately irrelevant stuff I learned along the way, or skip down to The Answer</a>.</p>
Techniques Available</h1>
There seem to be 4 options to accomplish this sort of task, 3 of which might be good:</p>

Modeline magic</a> can potentially introduce security vulnerabilities by allowing arbitrary code to be executed, but on the positive side, it allows arbitrary configurations to be set on a per-file basis. To enable modeline hacks, add set modeline</code> to your .vimrc</code> then put a line of the form # vim: set expandtab:</code> or similar in the file to which you want the command to apply.</li>
Autocommands</a> allow Vim to execute specific code when it sees a particular event in the file to which they apply. To be honest, I've never played with autocommands before, and left them till last because they seemed to have the steepest of all 3 formidable learning curves.</li>
Vim's command line options include -c {command}</code>, which runs the specified command after the first file has been read. Or --cmd {command}</code> executes it before processing the .vimrc</code> file. A shorter syntax for -c</code> is +{command}</code>.</li>
The final option, which is a Wrong Answer(TM) but seems awfully tempting after the previously described ones fail a lot, is to accomplish the same purpose through circuitous bash tricks. For instance if G</code> to jump to the end of the file fails when embedded in a command but :20</code> works to jump to line 20, I could use the output of wc -l</code> as a line number. Additionally, it might be easier to have Bash concatenate on some newlines than to convince Vim to insert them.</li>
</ul>
Trying Modelines</h1>
First, I added set modeline</code> to my .vimrc</code>.</p>
Then I added # vim: set noexpandtab:</code> as the first line in my file. This is a convenient test that my command syntax is right, since my .vimrc</code> defaults things to expandtab</code>. It works, in that the tab key inserts hard tabs when I hit it in that particular file now.</p>
Now, could a gesture possibly work? Switch it to # vim: G:</code>:</p>
Error detected while processing modelines:
</span>line    1:
</span>E518: Unknown option: G
</span>Press ENTER or type command to continue
</span></code></pre>
Unfortunately, the :normal</code> command which I discovered later does not work in modelines -- it seems that, true to their name, they really only do take mode settings as arguments.</p>
Trying command line syntax</h1>
I asked a few people how they solve this problem, and they suggested the vim +999999 file.txt</code> trick. Closer examination of the man page reveals that this works because when the specified line number is absent, it defaults to the last line of the file. Although with enough 9's it would work fine in practical applications, it seems icky to me because my notes file could hypothetically get very long. However, a quick test reveals that vim + file.txt</code> does the same thing!</p>
Chaining commands</h1>
Now I know that vim + file.txt</code> opens the file with the cursor at the first character of the final line. The man page contains a promising hint that I might be able to automate switching to insert mode as well:</p>
+{command}
</span>
</span>-c {command}
</span>    {command} will be executed after the  first  file  has
</span>    been read.  {command} is interpreted as an Ex command.
</span>    If the {command} contains spaces it must  be  enclosed
</span>    in  double  quotes  (this depends on the shell that is
</span>    used).  Example: Vim "+set si" main.c
</span>    Note: You can use up to 10 "+" or "-c" commands.
</span></code></pre>
I need fewer than 10 commands chained, so this might work. When looking up</a> how to express "enter insert mode" as a command line option, I discovered the :normal</code> command. It's the magic bullet for allowing gestures which you'd do in normal mode to be expressed on the command line. Let's try this:</p>
vim "+ normal G $" file.txt
</span></code></pre>
opens the file with the cursor at the last character of the last line! Success! However, "+ normal G $ i"</code> does not put one into insert mode after moving to the last character of the file. Time to chain commands!</p>
The Answer</h2>
vim "+normal G$" +startinsert file.txt
</span></code></pre>
Does precisely what I wanted all along!</p>
Its final form actually lives in my ~/.bashrc</code>:</p>
alias list='vim "+normal G$" +startinsert /absolute/path/to/list.txt'
</span></code></pre>
2 hours saving 10 seconds per file open twice a day is an egregious violation of the xkcd rule</a>, but it was fun and I learned a lot about Vim automation in the process.</p>
Update:</strong> Mythmon</a> pointed out that the command can be golfed into:</p>
vim '+ normal GA' foo.txt
</span></code></pre>
or to put yourself in insert mode in a new line at the end of the file:</p>
vim '+normal Go' foo.txt
</span></code></pre>


What Makes a Good Mailing List Post?
2015-01-28T00:00:00+00:00
As part of my student club officer duties, I send a lot of emails. One game that makes this chore less onerous is to try to optimize each email's quality. I do this by observing my own reaction to others' postings, and others' reaction to my posts. Here are a few trends I've noticed.</p>
Why the email?</h1>
What do you want your readers to do? Summarize the task in the email's topic. This shows respect for your readers' time -- you allow them to immediately discern whether the message might be relevant to them without investing the energy to open it.</p>
I doubt that I'm alone in feeling quite annoyed when an email with a vague title contains a request that's irrelevant to me. If the title had been more specific, I could have saved the time it took to open and read it.</p>
A few minutes invested in writing your message well will save time for tens or hundreds of list subscribers.</p>
Get to the point...</h1>
People skim their emails. Keep sentences short; complex ones only cause confusion. Long and chatty is great for personal conversations between friends, but frustrating and a waste of time for professional communication.</p>
...But include all the necessary information.</h1>
It disrespects your readers' time to make them look things up. "We'll meet at 6pm next Thursday (2/5/1015)" is far more useful than "we'll meet on Thursday". The latter forces the reader to go to your website and look up the time, and possibly go find a calendar to confirm the date as well.</p>
Know your audience...</h1>
If you're writing a technical message to a wide range of knowledge levels, it might be insulting to the intermediate and advanced viewers if you spell out every technical detail. However, it doesn't hurt to include enough detail to help a newer community member craft useful Google queries to get help.</p>
If you're asking readers to take an action, give them a concise explanation of why they should. Hint: "Because it's cool" might work if you have a lot of social status and a relatively simpleminded audience, but rarely convinces perceptive adults. However, if your audience thinks it's cool to write code and you say "Because you can write a lot of code if you take this action", they'll independently draw the conclusion that taking the action is cool too.</p>
...But don't expect too much.</h1>
Most people only look at one request per message. If you're requesting an action, make sure it sounds easy to accomplish, and you pre-emptively solve any problems they might run into along the way.</p>
For example, if your message's request is to have people add an event to their calendars, be sure you provide the date, start and end times, and location.</p>


Those funny characters in Vim
2015-01-27T00:00:00+00:00
</p>
I copied and pasted some of the lines from a PDF, and now I have a problem which is nearly impossible to Google.</p>
The first potential fix is to set the encoding so it tries to display the characters:</p>
:e ++enc=utf-8
</span></code></pre>
And it kind of works:</p>
</p>
Better. When I re-open the file with the encoding set correctly, my red bar in the 80th character column is one piece instead of being broken by the incorrect displays.</p>
Diamonds with question marks in them are equally difficult to search for, so I asked on IRC. My problem has 2 parts: how to find the un-display-able character I'm addressing, and how to refer to it in a regular expression.</p>
To find the problem character's value, place the cursor on it and use ga</code>. According to the vimdoc</a>, you can also use :ascii</code>, which might be easier to remember.</p>
</p>
This tells me that the problem character has hex value bf</code>.</p>
According to this helpful post</a>, which didn't show up until I googled "vim regex octal value of character", one can specify octal values in regex by prefacing them with \%x</code>. In my case, :%s/\%xbf/-/g</code> replaces all those questionmark symbols that used to be some kind of separators in the encoding from which I pasted them.</p>


Searching a FOSS project's history
2015-01-21T00:00:00+00:00
I'm curious about whether anyone has tried to build a predictive analytics plugin for Heka before. To find out, I'm going to stalk the project's entire recorded history. Since it's a relatively young project (only in its third year of having a public mailing list), the history is small enough for basic Linux command-line utilities to handle in a timely manner.</p>
Here are all the places one can look for project history, and how I used them.</p>
Find the Communication Channels</h1>
FOSS projects communicate on IRC, issue trackers, and mailing lists. Sometimes contributors also discuss things on the phone, via private messages, during video games, and in person. Although the latter type of interactions are rarely publicly documented, most of content that's important to a project is shared with all of its members, and thus easy for even a new contributor to find.</p>
When you want a particular piece of information, the question to ask is "if I was the person who created that information, where would I have put it?"</p>
Find the project's documentation on how to get involved and where to ask questions. On the front page of the Heka docs</a>, it suggests a mailing list, issue tracker, github project, and IRC channel.</p>
Searching the Mailing List Archives</h1>
Heka's mailing list archives</a> don't explicitly offer a search feature. Fortunately, the archives are not overwhelmingly large, so I can just download them and search them locally.</p>
Downloading the archives</h2>
Examining the page source reveals that the .txt.gz</code> archives have nicely standardized names, which allowed me to craft a slightly verbose wget command to retrieve them all:</p>
wget https://mail.mozilla.org/pipermail/heka/201{3,4,5}-{January,February,March,April,May,June,July,August,September,October,November,December}.txt.gz
</span></code></pre>
I suspect there's a recursive wget command which would grab all .txt.gz</code> links from a specified index page, but I didn't know it off the top of my head so I'll save the rabbit hole of crafting the shortest possible incantation for later. The command above was fast to research and write, and got all the tarballs I wanted, so it accomplished its purpose.</p>
Unzip</h2>
After that wget, I have a directory full of .txt.gz</code> files. I unzip them:</p>
gzip -d *
</span></code></pre>
Grep</h2>
Now it's easy to look for any keywords that might go with the information I'm seeking:</p>
grep -i "predict" *
</span></code></pre>
Searching the Source Code</h1>
Since the GitHub link was given in the heka docs</a>, it's trivial to clone a copy and search for keywords throughout the source and comments:</p>
git clone git@github.com:mozilla-services/heka.git
</span>cd heka
</span>git grep -i predict
</span></code></pre>
Searching the Commit History</h1>
The git grep</code> command used above is designed to search only in the tree and index, so Git's metadata is left out. If you do a regular recursive grep on a Git repo, you can get a bunch of redundant or spurious matches from Git's commit history.</p>
When it's time to intentionally search the commit history, Git has a tool for that too:</p>
git log -S"predict"
</span></code></pre>
The output of the git log</code> command will by default be a list of commits, with hash, author, date, and short message. For more detail on a commit, just git show</code> the hash:</p>
git show ea2b3c9f12f7f046be9b3bc133ee3eda90e16306
</span></code></pre>
Searching the Issue Tracker</h1>
Luckily, the terms I'm searching for are simple case-insensitive strings, so I can use the search box on the project's GitHub issue tracker</a>.</p>
If I needed to search with a regular expression, or simply wanted a local copy of the issue database, my options would be to use a pre-made GitHub backup tool</a> or the API</a>.</p>
Seek IRC history</h1>
Some channels are logged publicly. It's always worth a try to join the channel and examine the /topic</code> for any hints. IRC channel topics are also good places to find a community-specific pastebin or etherpad, both of which could potentially contain information relevant to your search.</p>
Also, Google.</h1>
In my case, Google helped find several repositories of Heka plugins, on which I repeated the issue tracker and source code search steps. Google is also the best way of finding a project's official blog, or simply individual blogs about the project, if those are available.</p>


ECE375: Using an Arduino Uno as a programmer
2015-01-19T00:00:00+00:00
I have an atmega128 development board</a> for the ECE375 class at Oregon State University. I believe the board is good, because it runs the test program that it came with when I picked it up from TekBots. I also have an Olimex AVR-ISP-MK2</a> programmer inherited from an ECE major friend, which I have come to conclude is bad, because despite testing and rebuilding all of the connections between it and my atmega128 board, despite passing avrdude all of the force and override-warnings flags at my disposal, it consistently refuses to program.</p>
Since my assignment is due tomorrow, I am configuring an Arduino Uno to stand in as a programmer. Here's how.</p>
I'm following the ArduinoISP</a> tutorial to turn an Uno left over from another project into a programmer.</p>
Add a capacitor</h1>
First, I scrounged a 10uF capacitor out of a failed attempt at building an audio amp. The capacitor goes on the Uno with its negative leg (the side with the stripe) connected to ground, and the other leg connected to the pin labeled reset.</p>
This is necessary to prevent spurious resets. If noise or interference on the board caused the reset line to fall below a certain minimum voltage (the logic low threshhold), the Uno would reset when the user didn't tell it to. The capacitor stores up electricity and can provide a tiny supply of power to keep the reset line high during fluctations, so that the board will only be reset if it's being actively driven low for more than just a moment.</p>
I asked a grad student in Electrical Engineering about the capacitor required by the tutorial, and he claimed that 10 uF is probably overkill -- a 0.1uF ceramic cap would do the job just as well. So if you're following this tutotial and only have a small capacitor available, try it anyways, and the worst that could happen without it is an occasional board reset during programming.</p>
Program the Arduino Uno</h1>
I installed the Arduino toolchain and IDE with yaourt -S arduino-beta</code>, since the regular arduino package was failing to download at the time I tried.</p>
I downloaded and unzipped the mega-isp firmware</a>, then opened it in the Arduino ide with sudo arduino ArduinoISP.pde</code>.</p>
When I attempted to compile and upload, it threw several errors:</p>
ArduinoISP.pde:36:14: error: expected unqualified-id before numeric constant
</span>/usr/share/arduino/hardware/arduino/avr/variants/standard/pins_arduino.h:41:22: note: in expansion of macro 'MOSI'
</span>static const uint8_t MOSI = 11;
</span>                      ^
</span>ArduinoISP.pde:35:14: error: expected unqualified-id before numeric constant
</span>/usr/share/arduino/hardware/arduino/avr/variants/standard/pins_arduino.h:42:22: note: in expansion of macro 'MISO'
</span>static const uint8_t MISO = 12;
</span>                      ^
</span>ArduinoISP.pde:34:13: error: expected unqualified-id before numeric constant
</span>/usr/share/arduino/hardware/arduino/avr/variants/standard/pins_arduino.h:43:22: note: in expansion of macro 'SCK'
</span>static const uint8_t SCK  = 13;
</span>                      ^
</span>Error compiling.
</span></code></pre>
Since MISO</code>, MOSI</code>, and SCK</code> were having a namespace conflict with a header file, the easiest fix was to replace their strings with MYMISO</code>, MYMOSI</code>, and MYSCK</code> throughout the ArduinoISP file. I'd guess they were added to the header files after the ArduinoISP software was published, since my version of the Arduino IDE is much more recent than the ISP's publication date of 2009.</p>
Note</p>
When saving these changes, the IDE asks if it's ok to change the extension from .pde</code> to .ino</code>. Just say yes; the file extension doesn't make any real difference.</p>
The next set of errors is:</p>
/usr/share/arduino/hardware/tools/avr/bin/avrdude: error while loading shared
</span>libraries: libtinfo.so.5: cannot open shared object file: No such file or
</span>directory
</span></code></pre>
A quick Google for the error has the Arch Wiki page</a> among the first hits, and the fix is straightforward:</p>
$ cd /usr/lib
</span>$ sudo ln -s libncurses.so.5 libtinfo.so.5
</span></code></pre>
After symlinking the libraries, my next error on attempting to program is:</p>
avrdude: ser_open(): can't open device "COM1": No such file or directory
</span>ioctl("TIOCMGET"): Inappropriate ioctl for device
</span></code></pre>
This would normally mean you get to mess around with groups</a> or just invoke the IDE as root. Since I'm already running it with sudo, it's a hint that Arch named the device something funny when it registered. In the Arduino IDE's menus, clicking Tools</code> -> Port</code> -> /dev/TTYACM0 (Arduino Uno)</code></p>
My next error was:</p>
avrdude: stk500_getsync(): not in sync: resp=0x00
</span></code></pre>
At the suggestion of this site</a>, I unplugged the capacitor from the RESET</code> pin and tried again. It uploaded without error.</p>
Wire the Uno up to the ATMEGA128</h1>
My ECE friend helped me read the ATMEGA board schematic and figure out what the pins in the 10-pin header do. Here's my ugly ASCII art of the header (note that the notch faces toward the inside of the board):</p>
-----------
</span>|a b c d e|
</span>|f g h i j|
</span>----___----
</span></code></pre>
Arduino Uno Pin</th> Name</th> Wire Color</th> ATMEGA Pin</th></tr></thead>

10</td> reset</td> gray</td> h</td></tr>
11</td> MOSI</td> white</td> f</td></tr>
12</td> MISO</td> black</td> j</td></tr>
13</td> SCK</td> brown</td> i</td></tr>
GND</td> GND</td> blue</td> b,c,d,e</td></tr>
+5V</td> +5V</td> purple</td> a</td></tr>
</tbody></table>
Surprisingly, the ATMEGA128 runs its test program quite happily when powered off of only the 5V line from the Uno. Here's a picture of the setup:</p>
</p>
Program the ATMEGA</h1>
A writeup</a> on the Arduino Playground provides the outline for my avrdude incantation. To find the actual port in use, I unplug and replug the Uno then examine the output of dmesg | tail</code> to see:</p>
[28403.507844] cdc_acm 3-1:1.0: ttyACM1: USB ACM device
</span></code></pre>
This means that the Uno is showing up in /dev/ttyACM1</code>.</p>
Note</p>
With the capacitor plugged in, I get the error:</p>
avrdude: stk500_recv(): programmer is not responding</p>
so I'm unplugging the cap for now and hoping that it works.</p>
For some code to test uploading with, I'm using Mythmon's Hello Dave</a> game.</p>
The installation command will look something like:</p>
sudo avrdude -p m128 -c avrisp -P /dev/ttyACM1 -U flash:w:hellodave.hex
</span>
</span>avrdude: AVR device initialized and ready to accept instructions
</span>
</span>Reading | ################################################## | 100% 0.01s
</span>
</span>avrdude: Device signature = 0x000000
</span>avrdude: Yikes!  Invalid device signature.
</span>        Double check connections and try again, or use -F to override
</span>        this check.
</span></code></pre>
When I pass the -F flag, it gets as far as a signature mismatch:</p>
sudo avrdude -p m128 -c avrisp -F -P /dev/ttyACM1 -U flash:w:hellodave.hex
</span>
</span>avrdude: AVR device initialized and ready to accept instructions
</span>
</span>Reading | ################################################## | 100% 0.01s
</span>
</span>avrdude: Device signature = 0x000000
</span>avrdude: Yikes!  Invalid device signature.
</span>avrdude: NOTE: FLASH memory has been specified, an erase cycle will be performed
</span>         To disable this feature, specify the -D option.
</span>avrdude: erasing chip
</span>avrdude: reading input file "hellodave.hex"
</span>avrdude: input file hellodave.hex auto detected as Intel Hex
</span>avrdude: writing flash (2468 bytes):
</span>
</span>Writing | ################################################## | 100% 0.32s
</span>
</span>avrdude: 2468 bytes of flash written
</span>avrdude: verifying flash memory against hellodave.hex:
</span>avrdude: load data flash data from input file hellodave.hex:
</span>avrdude: input file hellodave.hex auto detected as Intel Hex
</span>avrdude: input file hellodave.hex contains 2468 bytes
</span>avrdude: reading on-chip flash data:
</span>
</span>Reading | ################################################## | 100% 0.28s
</span>
</span>avrdude: verifying ...
</span>avrdude: verification error, first mismatch at byte 0x0080
</span>         0xff != 0x10
</span>avrdude: verification error; content mismatch
</span>
</span>avrdude: safemode: Fuses OK
</span>
</span>avrdude done.  Thank you.
</span></code></pre>


Don't rename Tinkerer posts
2015-01-15T00:00:00+00:00
Or if you must, at least do it correctly. Here's how.</p>
When you invoke tinker --post</code>, it creates a file in YYYY/MM/DD/postname.rst</code>, and also adds YYYY/MM/DD/postname to the toctree in master.rst</code>.</p>
I learned what happens when you rename things improperly because my very first post to this blog started out entitled "Blogging with Tinker". I then remembered that the framework is technically called tinkerer</code> even though one always invokes it as tinker</code>, so I changed the .rst file's name.</p>
If your sphinx-fu</a> is strong, the next part will be obvious. If you've finally carved out some time to play with your personal blog at a silly hour of the morning after a long day of schoolwork, it'll waste a lot of time and googling before you figure out why the post disappeared</strong>.</p>
When the name of the file does not precisely match the name in the toctree, you will not get a link to it in your sidebar. Sphinx does throw a helpful little warning:</p>
WARNING: document isn't included in any toctree
</span></code></pre>
but when that's right in the middle of a wall of 38 other warnings about how Sphinx isn't happy with Tinkerer's post templates being only templates and thus not used anywhere, it's not immediately obvious what changed since the build worked perfectly.</p>
Should Tinkerer have a --rename</code> option?</h1>
My first impression is no; the inconvenience of implementing and documenting --rename</code>, plus the clunky syntax of spelling out the full paths to the old and new posts and getting the dates right, seems to be worse than just mentioning it here.</p>
My Tinkerer post disappeared!</h1>
If your Tinkerer post is missing, make sure you didn't make the same mistake I did by forgetting to update the name in master.rst</code>. If you move the source file, you have to update the toctree too</strong>. Sphinx is just doing its job.</p>
Yes, this section is redundant, but I wanted to make sure this post includes the same phrases I googled when wondering what was going on. It's such an obvious error that I'm sure other people have made it, debugged it, gone "duh, that was obvious!", and then not written anything down.</p>


Floating-point Forth
2015-01-14T00:00:00+00:00
The first assignment</a> for CS480 (Translators) requests that we use Forth as a pocket calculator, rather than teaching the immensely powerful composition strategies for which it's valued in the real world.</p>
Since our first exposure to the language is a deep dive into the syntax of floating-point computation, no single tutorial on the web answers all the strange assortment of introductory and advanced questions that my classmates and I are running into.</p>
Here's what I've learned so far:</p>


Remember PEMDAS</a>. The quickest way to construct an expression tree for a given mathematical equation is to work from right to left in PEMDAS, then progress downward: The root of the tree is the first subtraction sign in the equation if there is one, otherwise the first addition sign, etc. Everything to the left of the sign which ended up as the root of the tree goes on its left side; everything to the right goes on the right. Then you build each little subtree, following the same rule.</p>
</li>

Deep within the introduction</a> of the book Starting Forth, it explains what the comments mean. If you learn nothing else from the introduction, this little section will make lots of other docs make sense:</p>

To communicate stack effects in a visual way, Forth programmers conventionally use a special stack notation in their glossaries or tables of words. We're introducing the stack notation now so that you'll have it under your belt when you begin the next chapter.</p>
Here is the basic form:</p>
( before -- after )
</span></code></pre>
The dash separates the things that should be on the stack (before you execute the word) from the things that will be left there afterwards. For example, here's the stack notation for the word .:</p>
.   ( n -- )
</span></code></pre>
(The letter "n" stands for "number.") This shows that . expects one number on the stack (before) and leaves no number on the stack (after).</p>
Here's the stack notation for the word +.:</p>
+   ( n1 n2 -- sum )
</span></code></pre>
</blockquote>
</li>

The floating point stack is separate from the default, or integer, stack. It has different arithmetic operators</a> than the integer stack -- typically same as the integer operator, but prefaced with an f.</p>
</li>
</ul>
F@      ( adr --)       ( f: -- x)
</span>F!      ( adr --)       ( f: x --)
</span>F+                      ( f: x y -- x+y)
</span>F-                      ( f: x y -- x-y)
</span>F*                      ( f: x y -- x*y)
</span>F/                      ( f: x y -- x/y)
</span>FEXP                    ( f: x -- e^x)
</span>FLN                     ( f: x -- ln[x])
</span>FSQRT                   ( f: x -- x^0.5)
</span></code></pre>


s>f</code> moves the top value of the integer stack onto the floating point stack. For this assignment, it won't make sense to ever move floating point values back to the integer stack.</p>
</li>

Declaring variables is pretty straightforward:</p>
variable a  (initialize an intger named a)
</span>23 a !      (assign the value 23 into a)
</span>a @         (put a's value onto the integer stack)
</span>.s          (hey look, the integer stack has 23 on it!)
</span></code></pre>
Remember that you have to preface everything with an f</code> to make it apply to the floating-point stack:</p>
fvariable z     (initialize a floating-point variable named z)
</span>69e z f!        (z gets the value 69)
</span>z f@            (put value of z onto floating-point stack)
</span>f.s             (Show the floating-point stack.)
</span></code></pre>
</li>

When an operator has both an integer and a float as its arguments, when do you put the int onto the float stack? According to a former instructor who used to teach this course, the correct solution is to insert a new node for the s>f</code> conversion between the parent and its integer child in the expression tree, before the post-order traversal. If you try to do the conversion after the traversal and it occurrs at the wrong time, you could end up accidentally swapping the order of the operator's children. This is harmless for addition and multiplication, but would break everything if it occurred with subtraction or division.</p>
</li>

There are two ways to do recursion in gforth. If you say recursive</code> after the name of the word you're creating, you are able to call the word by name in its definition. Alternately, you can say recurse</code> instead of the name of the word whenever you need to call it from within itself.</p>
Since both fibonacci and factorial are used in the homework, I'll use a function that sums from 0 to the argument as an example. In Python, it'd look like:</p>
def sum(i):
</span>    if (i == 0):
</span>        return 0
</span>    return i + sum(i-1)
</span></code></pre>
Using forth's recursive</code> command, it'll look like:</p>
: sum recursive
</span>dup 
</span>    ( top copy of the argument will get destroyed by comparison )
</span>0=  
</span>    ( take argument off the stack, replace it with true or false )
</span>if
</span>    ( we'll only take this branch if the argument was == 0 )
</span>    exit
</span>else
</span>    ( we have i on the stack, need to return i + the sum )
</span>    dup
</span>        ( extra copy, keep an i for adding later )
</span>    1-
</span>        ( decrement, now stack has i-1 on top to be the arg to next call )
</span>    sum
</span>        ( recursively call with that i-1 we made )
</span>    +
</span>        ( yay postfix! )
</span>endif ;
</span></code></pre>
If we don't use recursive</code>, it looks almost identical except for the actual recursive call:</p>
: othersum 
</span>dup 
</span>0=  
</span>if
</span>    exit
</span>else
</span>    dup
</span>    1-
</span>    recurse 
</span>        ( does exactly the same thing as the call to sum )
</span>    +
</span>endif ;
</span></code></pre>
So testing them out will look something like this:</p>
3 sum 
</span>. 6  ok
</span>4 sum 
</span>. 10  ok
</span>
</span>3 othersum 
</span>. 6  ok
</span>4 othersum 
</span>. 10  ok
</span></code></pre>
</li>
</ul>
Was this helpful? Did I miss anything? For feedback, go find my email address on my github</a> profile and send me your thoughts! (I might set up comments or a reasonably bot-resistant email address disclosure here someday, but for now, I hope the simple IQ test of clicking an extra link is enough to weed out spammers).</p>


Blogging with Tinkerer
2015-01-13T00:00:00+00:00
I had a wok</a> site here for a while, but I rarely (okay, never) updated it. My experience with blogging platforms has been limited to Wordpress (both self-hosted and on wordpress.com), Wok, Pelican, and an abomination of a Trac plugin that I'd prefer to forget.</p>
Here's how and why I am now trying Tinkerer</a>.</p>
Mythmon wrote Wok to meet his projects' needs, which have customizabity as a high priority and assume that the user is already comfortable with graphic design, CSS, and JavaScript. Since Wok's relatively small userbase means there's no standard template with the look that I had in mind for my personal site, I had some fun adventures in graphic design before getting frustrated and setting the project aside indefinitely.</p>
I've worked on other projects that use Wok, such as the OSU LUG</a> website, and found that it's easy to use as long as the artistic parts are handled by someone with more sophisticated aesthetic sensibilities than my own. Although I know an ugly website when I see it, my ideal user interface is a UNIX terminal, so I never managed to come up with a satisfying design for my personal site.</p>
I revisited my site recently and, in true sysadmin fashion, decided that the obvious way to solve the "human isn't happy" problem was to switch to a fancy new technology and see if that helped. Staticgen.com</a> has an exhaustive list of site generators, which one can sort by language and rating on GitHub.</p>
My criteria for a "better" site generator are as follows:</p>

Generate static sites</strong>. The purpose of my personal web site is just to disseminate a few pages of information, and the best way to accomplish that goal is a few static pages. I see no reason that my poor little DigitalOcean droplet should have to do more work than absolutely necessary for each visitor to my blog -- generating static pages only when they're changed is like caching a dynamic site, taken to the extreme. Plus, refusing to run any code input by my site's users (such as database queries, PHP scripts, etc.) on my server eliminates an entire category of security vulnerabilities.</li>
Take .rst input</strong>. Whenever I have a choice, I prefer to write presentations, documentation, and curriculum in reStructuredText</a>. In my experience, RST strikes a good balance between being supported by most documentation-generation systems and facilitating easy links, tables, and image embedding. Markdown is RST's main competitor and looks a bit simpler in plaintext, but if you want to embed an image or table, you're out of luck. It's possible</em> to get tables and images in markdown, if you don't mind writing raw HTML by hand, but it isn't pleasant</em>.</li>
Free and Open Source</strong>. Duh; if I improve it, I want to be able to share. If I love it and its maintainer gives up on it, I want to be able to keep using it anyway.</li>
Written in a language I know or want to learn</strong>. First, because I don't want to learn to set up the dependencies for some language that I never intend to use again. Second, because I'll inevitably end up getting an error message, or trying to hack in some new feature, and find myself digging around in the guts of my site generator. Althought the inner workings of these things are rarely pretty, it's less painful if I know or at least appreciate the language they're written in.</li>
Minimize irrelevant features for my use case</strong>. If a program's implementation and documentation are cluttered with features that I don't expect to ever want, the time I spend sorting support for those features from support for the parts I need will be wasted.</li>
Must ship with templates that I like</strong>. Yes, I'm lazy. After my misadventure trying to design a site theme from scratch, I'm rebounding toward the other extreme and trying to find something that Just Works how I want it to, right out of the box.</li>
Neutral or positive reputation among people I respect</strong>. I know, it's not cool</em> to admit that peer pressure exists, but it does. If everyone whose preferences I usually agree with says bad things about a particular blogging platform, I expect that it won't be very enjoyable for me to use either, and my time invested in learning it is likely to have been wasted. A popular and widely used SSG would be a safe bet, but I wouldn't necessarily learn anything new by deploying it. However, gaining experience with a promising but previously unheard-of tool is advantageous to my social groups, because next time someone asks for opinions, I'll have a non-redundant review to add.</li>
</ul>
Ultimately, the feature which set Tinkerer apart from the other SSGs which meet those criteria is that it feels like a thin layer on top of Sphinx, a versatile and delightful documentation tool with which I'm already very familiar. This has allowed me to leverage my existing knowledge and get a site up in only a couple of hours, rather than fighting with an unfamiliar design paradigm.</p>

edunham

Time Off, Time On

Collaboration

Assorted Git & Unix Tips

DIY Shoe Chains

Making Dice

Retroreflectors & Storing Ground Glass

DIY Thimble

Lumenator

Playing Dress-Up With Starlink

tree-style tab setup

top/htop in windows is ctrl+shift+esc

transcription with mplayer and i3

irssi and libera.chat

Assembly Lines

pulseaudio & volumio

Moving on from Mozilla

Offboarding

Git: moving a module into a monorepo

Finding a lost Minecraft base

Toy hypercube construction

Mathematical materials</h1> To construct a toy hypercube or any other graph, you need the graph. To make it from a single piece of string, every vertex should have an even number of edges.</p> Knowing the number of edges in the graph will be useful later, when marking the string.</p>

kubectl unable to recognize STDIN

What to bring to CCC Camp next time

More on Mentorship

Rustacean Hat Pattern

When searching an error fails

Running a Python3 script in the right place every time

CFP tricks 1

Tricks for choosing an audience</h1> Some strategies I've recommended in the past for dealing with this include looking at the conference's marketing materials to imagine who they would interest, and examining the abstracts of past years' talks.</p>

Skill Tree Balancing with a Job Move

Why an ops career

Thoughts on retiring from a team

Slacking from Irssi

This will work on all IRC clients, But...</h1> The good news is that once a Slack instance is exposing the IRC gateway, it looks to an IRC client just like a single standalone IRC server.</p>

Now Slack is almost IRC</h1> With the bridge set up, Slack behaves mostly like IRC. There remain some outstanding differences:</p>

Some northwest area tech conferences and their approximate dates

User is not authorized to perform iam:ChangePassword.

Better remote teaming with distributed standups

Problems with in-person standups</h1> When I've participated in stand-up meetings in person, I've noticed a few major flaws:</p>

Saying Ping

Resumes: 1 page or more?

"I existed!"</h1> I'm still surprised by how often people share awards and titles as "got X award", "was President of Organization". I hold the opinion that identical achievements sound much cooler with active rather than passive verbs: "Earned X award", "Led Organization", etc.</p>

Opinion: Levels of Safety Online

Salt: Successful ping but highstate says "minion did not return"

Advice on storing encryption keys

Tech Internship Hunting Ideas

Keep Following Up</h1> After your interview, write to whoever arranged it and thank the interviewers for their time. For bonus points, mention something that you talked about in the interview, or include the answer to a question that you didn't know off the top of your head at the time.</p>

Rust's Community Automation

Intro</h1> I'm a DevOps engineer at Mozilla Research and a member of the Rust Community subteam, but the conclusions and suggestions in this talk are my own observations and opinions.</p>

Automation</h1> To me, "automation" means "offloading peoples' work onto a system". This can be a computer system, but I think it can also mean changes to the social systems that guide peoples' behavior.</p> So, community automation is a combination of:</p>

Scoping the Problem</h1> While not all things can be automated and not all factors of the community are under the project leadership's control, it's not totally hopeless.</p> Choices made and automation deployed by project leaders can help control:</p>

Robots</h2> This lightning talk highlighted 4 categories of robots:</p> Maintaining code quality</li> Engaging in social pleasantries</li> Guiding new contributors</li>

Keep your tree green</h1> Saying "we can't break the tests any more" is a pretty significant cultural change, so be prepeared for some resistance. With that disclaimer, the path to following the Not Rocket Science Rule is pretty simple:</p>

Automating social pleasantries</h1> Have you ever submitted an issue or change request to a project, then not heard back for several months? It feels bad to be ignored, and the project loses out on potential contributors.</p>

Mathematical materials</h1>
To construct a toy hypercube or any other graph, you need the graph. To make it from a single piece of string, every vertex should have an even number of edges.</p>
Knowing the number of edges in the graph will be useful later, when marking the string.</p>

Tricks for choosing an audience</h1>
Some strategies I've recommended in the past for dealing with this include looking at the conference's marketing materials to imagine who they would interest, and examining the abstracts of past years' talks.</p>

"I existed!"</h1>
I'm still surprised by how often people share awards and titles as "got X award", "was President of Organization". I hold the opinion that identical achievements sound much cooler with active rather than passive verbs: "Earned X award", "Led Organization", etc.</p>

Keep Following Up</h1>
After your interview, write to whoever arranged it and thank the interviewers for their time. For bonus points, mention something that you talked about in the interview, or include the answer to a question that you didn't know off the top of your head at the time.</p>