The Unix guys showed us the way out, by (a) inventing the first non-assembler language really suitable for systems programming, and (b) proving it by writing an operating system in it. But they did something even more fundamental — they created the modern idea of software systems that are cleanly layered and built from replaceable parts, and of re-targetable development tools.
Behold, mimetypes.py, a gem from the Python standard library:
def guess_extension(self, type, strict=True):
"""Guess the extension for a file based on its MIME type.
Return value is a string giving a filename extension,
including the leading dot ('.'). The extension is not
guaranteed to have been associated with any particular data
stream, but would be mapped to the MIME type `type' by
guess_type(). If no extension can be guessed for `type', None
Optional `strict' argument when false adds a bunch of commonly found,
but non-standard types.
extensions = self.guess_all_extensions(type, strict)
if not extensions:
If there are multiple possibilities, return one at random. In this case, a Korn Shell script.
When I was 13 I became obsessed with programming. I was finally good enough at writing code to hack away like a madman. When I found a new programming problem I would work on it all night. I would enter the flow state. I wouldn't notice time passing. I'd dream about it and go right back in the morning. I was inventing problems at the same time I was solving them. Programming made me so happy (and still does today).
For the past 30 days I've had that same feeling, something I haven't experienced since I was 13 years old. I've been creating non-stop, obsessed with my latest experiments. I haven't been able to sleep, staying up until 4am focused on making one more smidgeon of progress. It's been a blast! But my time hasn't been spent programming. It hasn't been the occasional strategy game that has a similar effect (Civilization, Sim City, Starcraft, etc). What's made me feel like a kid again is my 3D printer and the promise of rapid prototyping.
This hub is a non-standard part. The geometry is totally different than what's out there. The chain never moves off the chain rings. All of the moving parts are hidden away. It looks awesome and works great (better than my old 12 speed bike). But the standard shifters for it look like this monstrosity:
There are twoother compatible, aftermarket shifters out there, but I don't like their look either.
The question was: What would it take to create my own?
Prototyping the shape
On January 18th I set out to make my own shifter a reality. The CAD software I decided to use was 123D Design from Autodesk (it's free!). I did some Alias back in school, so I'm not a total newbie, but look how horrible my first attempt at modeling the shifter was:
After a few days of experimentation with the CAD package I had a better design. What surprised me about CAD modeling is I'd enter the flow state just like I do when programming. I'd have to think hard about how the geometry moves and fits together, just like I simulate how a program will behave in my head, poking data in and out of memory. It made me ridiculously happy.
Meanwhile, I bought a 3D printer, the MakerBot Replicator 2X. I chose MakerBot because it's well supported. Here's a video of that thing in operation. The sound it makes is futuristic and haunting:
The CAD program exports an STL file (a mesh of triangles). The MakerWare software that came with my printer turns this into an execution plan (akin to Logo) that you can check for errors. The flat bottom parts are called "rafts", easily removable supports that prevent the ABS plastic from curling off the build plate.
Here it is printed into reality:
A few days later I prototyped how the pieces (screws, springs, ball bearings) would fit together, and had an assembly that could move:
What followed was a huge amount of trial and error to make the geometry shift the gears in precisely the right way. I also had to figure out how to make the holes more durable against the crushing impact of the indexed shifting springs. Here's the pack of duds I printed on my way to making it work. Each of these took about 75 minutes to print at 15% fill (meaning they are 85% hollow and chintzy):
But finally, miraculously, I made the shifter work. Here it is shifting the hub for the first time:
Iterating the design
After this things escalated quickly. I brought the prototype to the bike shop. We put it on a bike and found a design flaw: In high gears the knob would bang against the top tube when you turn the handlebars. It's finding problems like this that makes 3D printing amazing. I had been prototyping for weeks and thinking about it all the time, but I still overlooked simple constraints that become obvious once you actually try it out. Fixing those problems was fast and cheap.
I went home that night, spent a few hours rotating the geometry 90° forwards, and then did a 100% fill print to make it strong. The next morning I dropped the new parts off at the bike shop. They built the bike out. By the afternoon, all the pieces were in place: We had a fully built bicycle with the new shifter. The day was February 18th, exactly 30 days after I started the CAD drawings.
I rode the bike home that day and have been riding it a bunch since. I've also improved the design further, shrinking the base and knob to make the whole thing more ergonomic. What's crazy is how fast you can iterate. This design I thought of at 10am, had it in CAD by 10:30, had a print by 12:30, had it on the bike by 1pm. Being able to move this quickly reminds me of the liberating feeling of continuous deployment for building software. Having a 3D printer makes me feel like I can create anything. I'm no longer afraid of the physical constraints of designing real objects.
Now I'm looking for a machine shop to turn this part into a reality. ABS plastic is fine for a demo, but for durability and precision having it CNC'ed out of aluminum would be best. I want to anodize it in cool colors. I have no idea how much this will cost or how much time it will take. I'll write up that experience in another installment of this story as soon as it's done.
Added documentation to my Cohort Visualizer tool to explain what the calculations mean, reproduced here. This shit is more complicated than I remember:
A "bar" is every cohort for a particular day. A "bar segment" is one part of the bar for a day in single color, corresponding to a particular "cohort state" (like "Made two posts" above), which is usually some level of progression in the funnel.
∑↑ / ∑↕ "Percentage here and up"
Sum the bar your mouse is over vertically upward, including the bar segment your mouse is on top of. Then divide by the total sum for that bar vertically. Answers: "On this day, what percentage of users are beyond and including this cohort state in the funnel?" For the bottom bar segment this will be 100%.
∑↓ / ∑↕ "Percentage here and down"
Sum the bar your mouse is over vertically downward, including the bar segment your mouse is on top of. Then divide by the total sum for that bar vertically. Answers: "On this day, what percentage of users are before and including this cohort state in the funnel?" For the top bar segment this will be 100%.
X / ∑← "Percentage here of cumulative past sum"
Sum all bar segments from the cohort state you have your mouse over going back in time to the left, including the one your mouse is over. Divide the bar segment you have your mouse over by that sum. Answers: "What percentage of users does the highlighted bar segment represent as part of the whole past for that cohort state, including this day?"
X / ∑→ "Percentage here of cumulative future sum"
Sum all bar segments from the cohort state you have your mouse over going forward in time to the right, including the one your mouse is over. Divide the bar segment you have your mouse over by that sum. Answers: "What percentage of users does the highlighted bar segment represent as part of the whole future for that cohort state, including this day?"
X / ∑↔ "Percentage here of cumulative sum over time"
Sum all bar segments for the cohort state you have your mouse over for all days. Divide the bar segment you have your mouse over by that sum. Answers: "What percentage of users does the highlighted bar segment represent over all time for that cohort state?"
∑← / ∑↔ "Contribution of past to cumulative sum over time"
Sum all bar segments from the cohort state you have your mouse over going back in time to the left, including the one your mouse is over. Answers: "What percentage of users over all time got into the highlighted cohort state before and including the highlighted day?" For the last day this will be 100%.
∑→ / ∑↔ "Contribution of future to cumulative sum over time"
Sum all bar segments from the cohort state you have your mouse over going forward in time to the right, including the one your mouse is over. Answers: "What percentage of users over all time got into the highlighted cohort state after and including the highlighted day?" For the first day this will be 100%.
X / Max ↔ "Percentage of maximum single day ever"
Find the biggest bar segment for the cohort state you have your mouse over for all time. Divide the bar segment you have your mouse over by the the biggest amount. Answers: "How big is this day for users to get into the highlighted cohort state compared to all other days ever?" The biggest day will be 100%.
1 - X / Max ↔ "Delta from maximum single day ever"
Find the biggest bar segment for the cohort state you have your mouse over for all time. Divide the bar segment you have your mouse over by the the biggest amount. Subtract that from 100%. Answers: "How much bigger is the biggest day ever for this cohort state compared to this day?" The biggest day will be 0%.
Intersectionality (or Intersectionalism) is the study of intersections between different disenfranchised groups or groups of minorities; specifically, the study of the interactions of multiple systems of oppression or discrimination. The term is particularly prevalent in black feminism, which argues that the experience of being a black female cannot simply be understood in terms of being black, and of being female, considered independently, but must include the interactions, which frequently reinforce each other.
The context was considering the interactions between being female and being a software engineer.
A blind item is a news story, usually gossip, in which the details of the matter are reported while the identities of the people involved are not revealed. The invention of the blind item is credited to William d'Alton Mann (1829 – 1920), publisher of Town Topics, who often used it for blackmail.
I hope no one missed Peter Norvig's solution to "xkcd 1313: Regex Golf". It's awesome. Reminds me of his classic spell checker tutorial. If you ever wondered why people love Python both of these examples should make it clear.
I've been playing around with 123D Design recently. It's a free (!) CAD package by Autodesk that lets you build models for 3D printing. I haven't done any 3D modeling since I used Alias back in school, but fortunately it's like riding a bicycle. To be efficient in a CAD package like this you need pan, zoom, and rotate all on the mouse.
The problem: Out of the box Mac OS X has no middle click! On a trackpad you have zoom with two-finger scroll up/down, rotate with two-finger right-click drag, but there's no middle click. With the Magic Mouse I use you have zoom with one-finger scroll up/down, rotate with right-click drag, but there's no middle click. So you can't pan! Oy.
The best solution I've found is BetterTouchTool. I mapped trackpad three-click drag to the middle mouse button. I mapped two-click drag on the Magic Mouse to the middle mouse button. And it works in 10.9.1 Mavericks.
Steve Blank shares some thoughts on how continuous deployment negatively affects building products. I agree with the issues he's identified, but I think they're growing pains. Users will become accustomed to the new model.
Recently a friend asked if Objectives and Key Results (OKRs) actually work. Presumably their startup is considering using them. Google is well-known for using OKRs (video and helpful summary). I guess I've formed an opinion about OKRs from the experience of using them in different ways on multiple teams over 8 years.
Here's my reply to that email with some edits for clarity:
Overall yes, OKRs work. Transparency is good. OKRs are a good way for team leads to express vision to a whole group and make it clear what your combined goals are.
The real purpose of OKRs, in my view, is to make it so the members of your team can anticipate what to work on next, or what is the highest priority, or why you're making the choices you're making. You never want someone to ask "what should I be doing next?" or "is A more important than B?". That should always be obvious based on the vision you've agreed to as a team. If it's not clear that's your fault not your teammate's.
I find OKRs on smaller teams lead to micromanaging (especially personal OKRs). OKRs are stupid when the key results are "launched" or "project does not fail". If you have sales numbers then it's easier (Objective: Grow revenue by 50%; Key Results: Sign up 200 Llama video producers, Increase viewership by 10x).
I believe quarterly goals are too slow. On my team we do 6 week milestones. We have a list of "must have", "would like", and "nice to have" goals for each period. The lists are primarily key results because objectives are obvious. Each milestone is the exhaustive list of everything we should be working on as a team, including stretch goals. If it's not on the list, you probably shouldn't be working on it (unless it's a bug, interrupt, 20%, prototyping, etc).
We don't expect to finish everything. That lets us reset our trajectory as we go along. Any faster than 6 weeks seems too often because shit just takes about that much time to actually do. Any slower is too infrequent and people end up adrift or over-designing things.
We've tried to grade the milestones like OKRs. They end up being 50-60% complete (as OKRs often are). So we usually shoot for getting ~2/3rds done. We're often pretty close, so that part of the exercise is less important, but it helps transparency for the team to understand our velocity.
As my friend summed it up:
OKRs are an effective tool for communicating priorities to teams.
Be wary of using them when things are too early to measure effectively.
6 week timeframes work better for smaller teams in my experience.
The difference between "key results" and "objectives" can be confusing.
The relationship between lead engineers and product managers is unique. The goal of your collaboration is to logically determine the best outcome for your users that balances all of the tradeoffs (time, features, complexity, cost, etc). It's hard to do. It's easy for conversations to get heated and personal. But if that happens you (and/or your counterpart) are doing it wrong.
I've had many conflicts with product managers, especially when I was leading my first programming projects. Most of these were my fault due to my inexperience and love of confrontation. But many conflicts were also caused by the product managers I collaborated with who were similarly inexperienced. I believe preventing these conflicts is something everyone needs to learn regardless of their role.
The lesson is this: Discussion between product and engineering is not a debate, it's a dialectic. That's a classical concept defined by Wikipedia this way:
The dialectical method is discourse between two or more people holding different points of view about a subject, who wish to establish the truth of the matter guided by reasoned arguments.
In classical philosophy, dialectic is a form of reasoning based upon dialogue of arguments and counter-arguments, advocating propositions (theses) and counter-propositions (antitheses). The outcome of such a dialectic might be the refutation of a relevant proposition, or of a synthesis, or a combination of the opposing assertions, or a qualitative improvement of the dialogue.
Contrast that to debate:
In its original meaning, the term dialectics is not synonymous with the term debate. While in theory debaters are not necessarily emotionally invested in their point of view, in practice debaters frequently display an emotional commitment that may cloud rational judgement. Debates are won through a combination of persuading the opponent; proving one's argument correct; or proving the opponent's argument incorrect.
Dialectics are logical, not persuasive through rhetoric. In a dialectic there is no loser. In a dialectic you can play devil's advocate without seeming overly negative. It's weird compared to everyday communication, but it works well for the goal of making users happy.
So remember: Next time you find yourself in a debate, choose to have a dialectic instead. It's not easy, but it's always worth it.
Statistics Done Wrong is a guide to the most popular statistical errors and slip-ups committed by scientists every day, in the lab and in peer-reviewed journals. Many of the errors are prevalent in vast swathes of the published literature, casting doubt on the findings of thousands of papers. Statistics Done Wrong assumes no prior knowledge of statistics, so you can read it before your first statistics course or after thirty years of scientific practice.
For better or worse, though, the IETF does not get to define the world that our protocols are used in. We cannot require that companies don’t monitor employee behaviour, or that countries don’t watch and curb what people do. We cannot and should not try to force people — whether they be browser users, web site operators, or the network administrators in between them — to do things against their will, because in reality we can’t; all we’d do is encourage people to stay on HTTP/1.1, or develop yet other competing protocols, or subvert the existing protocols in yet new and more damaging ways.
What we can do, however, is provide the flexibility and precision in the protocol to allow this discussion, this push and pull amongst the stakeholders — end users, web servers and the networks in between them — to play out.
He refers to this contention as "the Tussle" from this paper (2005). I can't believe I've never heard of this!
This paper explores one important reality that surrounds the Internet today: different stakeholders that are part of the Internet milieu have interests that may be adverse to each other, and these parties each vie to favor their particular interests. We call this process "the tussle." Our position is that accommodating this tussle is crucial to the evolution of the network’s technical architecture.