"Google Abandons Open Standards for Instant Messaging"

Sigh.

I went to AAPOR for the first time this year, a conference covering public opinion polling and survey research. It's hardcore. Using "data" as a singular noun there is gauche. My goal was to see the panel session on Pew's use of Google Surveys. I had a great time and would go again.

A highlight was when I used triangulation to correct the data in an economist's presentation (never done that before!). It made me realize that beyond academia, entrepreneurs and startups should also be using triangulated research to validate their product plans and business models.


What is triangulation?

Triangulation is when you ask the same question many different ways and compare the results. You'll see agreement or disagreement between the questions. If they don't agree, something is happening that you don't understand. This lets you self-validate or corroborate your findings. Think of it like running an A/B test on survey question correctness, except that you want zero separation. It's similar to a fundamental part of the scientific method.

To show what I mean, I ran four different survey questions about cat ownership in the US. Here are the results (after some simple math):

#	Question	Cat ownership	Dog ownership	Pet ownership
1.	What kind of pets do you have in your household?	23.9% (+2.2/-2.0)	41.0% (+2.4/-2.4)	~52%
2.	How many cats do you have in your household?	28.2% (+2.3/-2.2)	-	>= ~28%
3.	Do you have one or more cats in your household?	24.0% (+2.1/-2.0)	-	>= ~24%
4.	Are cats or dogs not present in your household? Or do you have both types of pet?	30.5% (+2.4/-2.3)	44.0% (+2.5/-2.5)	~58%

The results converge on the same numbers (cat ownership between 22% and 28%) and agree with each other. Most differences are within the margin of error. The min/max span is 4 percentage points. The numbers also agree with data from the humane society and the AVMA sourcebook. I'm confident I know how many people online have cats.


What happens without triangulation

At AAPOR, one of the researchers from NORC / University Chicago presented test results on how representative Google Surveys are. Their original, less accurate finding was that Google Survey data does not closely agree with benchmarks for telephone ownership. We ran a follow-up survey to find out why. The problem was modal bias: Asking a question over the phone introduces errors that are different than the errors from a microsurvey.

By tweaking the question slightly we were able to reproduce the Pew Internet data within 3 percentage points (our results are here in Q1/2; Q3/4 demonstrate the modal bias).

The NORC folks were happy to hear our data was better than they thought. Had they triangulated their results themselves, they would have seen disagreement and known that something else (modal bias) was happening.


Why not triangulate?

Surprisingly, nobody I asked about triangulation at AAPOR had employed it in their own research. Maybe I missed somebody, but it makes sense:

1. Most polling that exists today is extremely rigorous and proven.
2. But it's slipping away because:
   
   • It's illegal to robo-call cellphones.
   • Phone response rates are very low.
   • Your current location and cell phone area code are often unrelated.
3. This makes traditional market research and opinion polling expensive and introduces bias.

So it's plausible that traditional researchers don't triangulate because they can't afford to. And why would they triangulate if the existing measures and techniques work well? The problem is when old measures are applied to new situations, like the NORC example above.


Now it's easy

With new methods it's cheap to do triangulation. I've seen startups triangulating decisions using Google Surveys and the results are great. I presented one such case in London, recently. Anyone can do it.

So: Whenever you make an important decision about a product, business, or research you should triangulate the data used in your conclusion. Try multiple approaches and find agreement between many measures of the same idea. This will give you confidence in your conclusions. It will provide defense against detractors. It will bring consensus to your team.

And you'll know that you're right.

“When you wake up in the morning, Pooh," said Piglet at last, "what's the first thing you say to yourself?"

"What's for breakfast?" said Pooh. "What do you say, Piglet?"

"I say, I wonder what's going to happen exciting today?" said Piglet.

Pooh nodded thoughtfully. "It's the same thing," he said.

I enjoyed this slide deck from Nick Galbreath, especially slide 25, which states the following hypothesis:

• It is impossible to simulate the production environment in development, either due to operational differences or data differences.
• No amount of QA or Security Testing can prove you don't have bugs, vulnerabilities, or cause severe operational problems.
• You have bugs and vulnerabilities, right now, in your application.

And the conclusion is that the only solution is continuous deployment. Indeed! I'm happy to see this viewpoint taking hold.

Here are the slides:

The Isley Brothers – Footsteps In The Dark - Part 1 & 2

The Google Ventures Startup Lab posted the video of my talk about Cohort Analysis. The slides are here.


Enjoy!

I've talked about my preference for first world goods previously. The gist of it is, I want to buy things made by people who have the same rights and freedoms that I do. I've managed to find great first world sneakers, shoes, tees, hoodies, shirts, and more. The missing bit has been a decent pair of jeans.

There's a place in SF called Self Edge that, for no real reason, I always assumed was snooty and bullshit. They also have a store in NYC, and I stopped in with a friend on a recent trip, putting my skepticism aside. That day, Kiya Babzani, the store's founder, happened to be in the NYC shop. We talked for a while and it turns out he's a rad guy. It made me realize that Self Edge isn't snooty, it's border-line Otaku. What they sell is mostly made in Japan, where the car/motorcycle/rockabilly subculture has co-opted classic American manufacturing and brought it back to life.

Which brings us to the pair of pants I bought. Check out the well-executed, overly descriptive, not-quite-but-still-is-Engrish on this tag (e.g., "vintage sleek"):



Anyways, these pants are great. They're sturdy enough that they identify them by weight more than anything. I figure they'll last five years, probably more, justifying the cost to me (it helps that the Yen is super weak right now). Contrary to popular belief, you're supposed to wash them frequently.

If you're still skeptical, try reading this two part interview with the founder. I enjoy things that people are passionate about, like bikes, to the point of being obsessive. The Flat Head and Self Edge pass the test. I stand corrected. And now I have a nice pair of first world pants.

I actually sit down and open my postal mail maybe four times a year. Wish I could stop it.

The past few summers I've attempted the near-impossible in San Francisco: Growing tomatoes. Every day it's some combination of cold, foggy, and windy. But I've managed.

What's surprising about growing tomatoes is how something so small can grow so big. I start with a tiny sapling in a thin plastic pot. I move it to a larger pot and then water it every couple of days. By the end of a week its mass has doubled. At a month it's over a foot high and blossoming. After two months the first tomatoes are growing fast and the plant is chest-high.



Each day I water the plant it looks the same as the day before. I can't notice daily changes because the differences are subtle. I went away for a week and when I got back the plant looked enormous compared to before. When I hadn't seen the incremental changes, the plant's growth was astonishing. Growing tomatoes has made me see the value solely in time passing.

How would I be if I spent a tiny bit of time cultivating myself every day? It wouldn't seem like much to me, since I'd witness the small differences. It'd be hard to stick with it. But after two months or a year, I may look back and realize how far I've come. I think this is how I developed as a programmer. I wonder how else I could improve this way.

Bad marketing email. Not even trying to be experimental about it. No call to action. Too much text.

From this article: "The people you want to hire aren’t applying and interviewing, they’re running their own companies." Or at Google, Facebook, Twitter, etc.

All useful code must be rewritten at least twice.

"Today we can officially confirm the release date for the Mac version of SimCity is June 11". Sweet. Goodbye summer!

John Carmack's magical sqrt(). Came up in conversation again tonight. It's amazing!

The feeling when the domain you want for your project is available.

I dislike dubstep because it's so desperate for attention.

Woah, end of an era.

subprocess.CalledProcessError: Command '['java', '-version']' returned non-zero exit status 1

I don't even have it installed.

The other day I saw a tweet go by that made me think twice:



The idea that this issue is big enough to win the popular vote for President seemed questionable.

I wondered:

• Who would want to use electronics during takeoff/landing?
• How many people actually own a smartphone or tablet at all?
• And how many people fly on an airplane regularly anyways?

If you are reading this, chances are you fall in these categories and so do your peers. But the real answers are surprising.


The data

Baseline for perspective: 81% of US adults use the Internet. 67% of them are on Facebook. (Pew)



87% of Americans have a cellphone, but only 45% have a smartphone. 31% have a tablet. (Pew)



The kicker: 56% of US Internet users have not been on an airplane this past year. 18% never have. (Google Surveys)



Let that sink in

• Only 43% of the US Internet population has been on an airplane in the past 12 months
• More people have used Facebook than have flown on an airplane this year
• More people own a smartphone than have flown on an airplane this year

So don't assume that everyone's experience is just like yours. As we see in the data, outside of your circle the world is very different.

I don't think we should feel ashamed for living in a bubble and having privileges. Products like Dopplr and TripIt are very valuable to their niche of frequent travelers. As long as we acknowledge how lucky we are, it's fine. The problem is when people don't realize they're a niche and assume everyone thinks like they do.


In the end

Perhaps Ben's tweet was deadpan, a hilarious critique of Senator Claire McCaskill, Democrat of Missouri, who really should be focusing on the welfare of the majority of Americans, not the privileged few.


(Discuss on YC HN)

HN is so far gone now that someone felt compelled to put "Time for a technical article" in their submission title.

Shit happens

Had a nightmare about the shape of our funnel. Not based in reality, just exposes my anxieties.

I hope the big winner in this is Mozilla. Hooray diversity! Boo monoculture!

Chromium has forked WebKit. An important nut:

"Nevertheless , we believe that having multiple rendering engines—similar to having multiple browsers—will spur innovation and over time improve the health of the entire open web ecosystem."

Missed this one!

So Tufte decided it was time to out himself as an ex-phone phreaker. He wrote Bowen an email. If AT&T could decline to ruin Tufte's life, maybe JSTOR could find the courage to make sure that Swartz's abilities would not be wasted. Tufte was phreaking before Captain Crunch, before Steves Wozniak and Jobs. Those guys turned out to have lots more to contribute.

JSTOR did the right thing. Not only did it tell the Federal prosecutors that it had come to a satisfactory arrangement with Swartz, but it took significant steps to advance Swartz's (and JSTOR's!) agenda af making information more accessible to everyone.

Best one I've seen all day: Google Chrome’s Broken Image Icon Heals Broken Hearts

Good summary of an interview with Ben Kamens about How Khan Academy Scaled To 6 Million Users A Month On Google App Engine.

From here:

In response to a complaint we received under the US Digital Millennium Copyright Act, we have removed 2 result(s) from this page. If you wish, you may read the DMCA complaint that caused the removal(s) at ChillingEffects.org.

Searches related to da39a3ee5e6b4b0d3255bfef95601890afd80709

Problem is: That is the SHA-1 hash of the empty string:

>>> import hashlib
>>> hashlib.sha1('').hexdigest()
'da39a3ee5e6b4b0d3255bfef95601890afd80709'

And of course the complaint is about files that are porn.

One of the hallmarks of a good design or redesign is you don't like it initially. Good design pushes you outside your comfort zone. Good design is not what you had before or what you're used to. Good design takes away things that were inessential, things you may have liked or found useful.

When you find yourself arguing or complaining about design, stop and think. Hold your judgement. Wait it out. Try the new design for a while and see how it actually feels. The way you rationally think about design changes is not how you will actually interact with the new design. Your opinion is valid but probably irrelevant.

I've seen reactions like this recently with a product I work on and many others that I use. People deal with it like the five stages of grief (and please, I don't mean to trivialize real grieving):

1. Denial: "This must be temporary, this redesign can't be permanent."
2. Anger: "This design is terrible, why would they do this to us?"
3. Bargaining: "Just give me an option to use the old design please!"
4. Depression: "Ugh-- now I need to find a replacement to use instead."
5. Acceptance: "Well, I guess I'll give it a shot and see how it goes."

When you get to the other side of this you almost always prefer the new design. Because the new design is better! The designer knew what they were doing. Surprise, surprise! They were willing to take risks and deal with your bullshit complaining because long-term they knew what was best.

So please: Trust designers, have an open mind, stop complaining.

"Making things -- writing, software or blog posts -- is hard." A note on positivity by Evan.

Proud today. We're celebrating a year of Google Consumer Surveys, a product I helped create. For a year we have:

• Provided free content for users
• Earned millions in revenue for publishers
• Enabled businesses large and small to make data-driven decisions

Everybody wins. See more in this interview with Paul on Forbes.

Looking forward to 2013 and beyond!

Here's the video from my talk at Mozilla SF yesterday (slides):



Thanks everyone for coming! Great turnout for the Continuous Delivery SF meet-up.

Tonight! My talk on continuous deployment and perceptual diffs at Mozilla SF. Streaming live at ~7pm.

You probably can't use continuous deployment for spacecraft.

PubSubHubbub on NewsBlur

Giving it a try. Right now it's slow, trying to chew through 1000 feeds.

A nice message I received on reddit today:

Hey, I stumbled on /r/hackerethic from checking my referrer logs for an article you submitted. The contents seem very in line with what I want to read, and so I wondered about submitting a thing or two — can/should this be done? It looks like some kind of gated community, only its also public

It's totally public! Please join us. Spread the word. We're just very new.

It's embarrassing when a & slips in there.

Chris is doing the unsung heroes of Reader right now.

Generative logos are cool.

Come to my talk on Continuous Deployment! It's at Mozilla SF on Tuesday, March 26th at 6:30pm. Thanks to Sauce Labs for hosting!

From More innovation means less control. Is that bad? Emphasis added:

The conventional wisdom, which is conventional but not wisdom, says that RSS is obsolete because now we have Twitter and other social things. Techcrunch even said "In essence, Twitter is a big RSS reader."

In fact, Twitter is not a big RSS reader. RSS is something you control, and Twitter is something other people control. (Even if you dedicate a Twitter account exclusively to the same sources of content you had in Google Reader, the viewing options, functionality and everything about Twitter is controlled by Twitter.) That both give you streams of content is a superficial similarity. Fundamentally, they are opposites.

What Google Reader and RSS fans fear is not the loss of a good service and a great format. They fear the loss of control. They fear a future in which decisions about what they see, watch, read and listen to are determined by secret algorithms and the whims of the social media masses.

TIL: David Beazley created SWIG! He's made such an amazing set of contributions. #PyCon

That awkward moment when you meet the person who's maintaining the code you wrote 7 years ago.

Looking forward to #PyCon sessions:
Use STM to remove the GIL. X-ray profiling w/frames. SQLAlchemy in depth.

Just arrived at #PyCon! Where the snakes at?!

fontrum n.

Feeling embarrassment for someone that doesn't have enough common sense to feel the embarrassment that they should be feeling for themselves for their actions.

Watching movies like Meet the parents can be a fontrum. It goes so bad for Greg that you feel uncomfortable.

Not a proud day.