I'm Brett Slatkin and this is where I write about programming and related topics. You can contact me here or view my projects.

26 April 2014

Go's power is in emergent behavior

I just got back from Gophercon and had a great time. This was the first Go language conference ever. It was a single track of speakers and 700 people packed into a single ballroom in downtown Denver. Everyone said they felt like we were part of something special (but perhaps that's just the lack of oxygen talking – small room, high altitude).

Anyways, one of my favorite sessions was Q&A with the Go team. Rob, Andrew, Brad, and Robert took questions from the audience for an hour. By far the best question was "What was the first thing you saw in Go that surprised you?"

Andrew and Rob's favorite was the http.HandlerFunc type (I'll explain it below). That example is great, but what surprised me the most was Robert Griesemer's example of opaque types, which takes the wonder of http.HandlerFunc to the next level.

What I love about these examples is they help you internalize the subtlety of Go.

Let me explain in two parts:

1. Why is http.HandlerFunc surprising?

Background: In Go, implementing interfaces is implied, not explicit. You don't declare a super-class or "implements". Instead you just make your new type define all of the methods of an interface. Once you do that, your type "is" that interface. Then you can pass values of your concrete type to functions that expect the interface type and know nothing about your type.

Here's a simple example:

type Awesome interface {
        IsAwesome() bool

type MyType struct {
        howAwesome int

func (m *MyType) IsAwesome() bool {
        return m.howAwesome > 0

func PrintItOut(a Awesome) {
        if a.IsAwesome() {
                fmt.Printf("%#v is awesome\n", a)
        } else {
                fmt.Printf("%#v is not awesome\n", a)

func main() {

The output is:

&main.MyType{howAwesome:0} is not awesome
&main.MyType{howAwesome:10} is awesome

Nowhere in this code did we ever declare a relationship between MyType and Awesome. Yet PrintItOut successfully operates on a MyType instance even though it only declares that it requires an Awesome interface. That's how Go's implicit interfaces work.

Back to Andrew and Rob's surprising example: Go defines this standard interface for serving HTTP requests:

type Handler interface {
        ServeHTTP(ResponseWriter, *Request)

Making your own web server with a single handler is easy:

type MyHandler struct {}

func (h *MyHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintf(w, "Easiest webserver ever")

func main() {
        http.Handle("/", &MyHandler{})
        log.Fatal(http.ListenAndServe(":8080", nil))

But you may have noticed this example isn't ideal. We had to define the MyHandler type (an empty struct) so we could define the ServerHTTP method on it, thus satisfying the http.Handler interface. How would you make this simpler in Go?

The solution is what surprised Andrew and Rob:

type HandlerFunc func(ResponseWriter, *Request)

func (f HandlerFunc) ServeHTTP(w ResponseWriter, r *Request) {
        f(w, r)

That's it: a tiny adaptor that makes a bare function satisfy an interface. How is that possible?

Go has first-class functions, meaning functions can be used in code like any other type in the language. That means you can define new types from function signatures. All types can define methods, allowing them to implicitly satisfy interfaces (described above). Thus, even new types derived from function signatures can have methods. In this case, the http.HandlerFunc type satisfies http.Handler.ServeHTTP. This causes all bare functions with that signature to be easily used by the http package:

func MyHandler(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintf(w, "Easiest webserver ever")

func main() {
        http.Handle("/", http.HandlerFunc(MyHandler))  // Cool!
        log.Fatal(http.ListenAndServe(":8080", nil))

The http.HandlerFunc(MyHandler) part of this isn't a constructor or function call. It's a simple conversion between assignable types for the sake of the compiler – it's free. Wow!

What's so surprising about this example is it makes something powerful by bringing two simple ideas together: first-class functions and implicit interfaces. Each of those features is completely separate in the language. However, when they combine something magical happens: What was hard becomes easy.

2. How do you implement opaque types in Go?

Refresher: An opaque type is a datatype provided by an API where the implementation details are hidden from the API consumer. Opaque types are used to provide strict interfaces between components, especially around sensitive structures like file handles. One very common opaque type system you may have used is Apple's Core Foundation library.

Concretely, why would I want to hide an implementation? Say I want to carry around important state, like an open socket, with an instance of my type. But I want to make sure my API clients don't mess around, like write to the socket directly. Perhaps the reason is today I'm using TCP/IP but next week I'm switching to UDP. My API clients shouldn't know. I should be able to switch and provide the same functionality with just a recompile. That's the whole point of abstraction.

Making the socket a "private" field in my type achieves this. Unlike C++ and Java, which have explicit keywords for visibility, Go controls visibility through capitalization. If an identifier starts with an upper-case letter, other packages can see it and mess with it. If it starts with a lower-case letter then only your package can touch it:

package peanut

type MyType struct {       // Public to consumers
        doNotTouch Conn    // Private to peanut
        PrettyName string  // Public to consumers

This visibility strategy works great for structs, but what about interfaces?

Imagine my API package provides a common interface to many implementations (like a database driver):

package peanut

type MyType interface {
        OpenConnection() error
        DoQuery() error
        CloseConnection() error
        WriteRow() error

The problem is anyone might implement this interface and do something wrong, like incorrectly manage state transitions. To have a real opaque type with a strict interface, I need to prevent anyone else from ever implementing my interface.

That sounds like a contradiction: How can you have a public interface that can't be implemented?

The answer is what surprised Robert.

Let's define a new opaque interface type in Go and a function that consumes it:

package opaque

type Opaque interface {
        GetNumber() int

func DoSomethingWithOpaque(o Opaque) string {
        return fmt.Sprintf("Hello opaque #%d", o.GetNumber())

We can then define a concrete type that implements the interface in the same package:

type internalBasicType int

func (m *internalBasicType) GetNumber() int {
        return int(*m)

func NewBasic(number int) Opaque {
        o := internalBasicType(number)
        return &o

We can exercise this code simply:

func main() {
        o := opaque.NewBasic(15)
        s := opaque.DoSomethingWithOpaque(o)
        fmt.Println("Doing something with an opaque I get:", s)

The output is:

Doing something with an opaque I get: Hello opaque #15

Unfortunately, the Opaque interface isn't a private implementation at all. Anyone could come along and implement it themselves. But what if we modify our original interface definition to be this instead:

type Opaque interface {                   // Public
        GetNumber() int                   // Public
        implementsOpaque()                // Private

Now we have a public interface, Opaque, but it contains a private method implementsOpaque. Within my package I am free to define implementsOpaque because it's visible, so I can make the internalBasicType conform to the Opaque interface with one line:

func (m *internalBasicType) implementsOpaque() {}

How does this work? Because of the lower-case letter of implementsOpaque it is not visible outside of my package. But the rest of the interface is. Thus, API consumers can see and use my public interface, but they can never implement their own. The implementsOpaque method simply doesn't exist in the exported interface, it can't be defined, thus external types can never satisfy the interface implicitly. Wowza!


What this all means is complex, useful behaviors naturally emerge from the interplay between simpler language features. These aren't pre-planned outcomes. They are the byproduct of good foundational design. And there are many more examples of this beyond http.HandlerFunc and opaque types. It's these subtleties that are the true power of Go.

(PS: See all the examples including negative cases in this repo)
© 2009-2024 Brett Slatkin