The goal was not to create a new programming language, but to provide a better way to write software. Specifically, the Go team wanted to address the complexities that arise when building modern server software, such as:<ul><li>controlling dependencies</li><li>programming with large teams with changing personnel</li><li>ease of maintainability</li><li>effective use of multicore CPUs and networking</li><li>efficient testing</li></ul>

Why Go was created

<ul><li>Portability. It allowed programmers to work on whatever platform they liked, and compile for whatever platform was required.</li><li>Gofmt. It showed that autoformatting could be done well and it could save valuable time by not arguing over spaces and newlines.</li><li>Strong standard library. It kept the community working with the same toolkit and prevented variant libraries from arising.</li><li>Concurrency. It helped popularize concurrency as a way to structure server software by making concurrency a first-class component of the language.</li><li>Fast builds 😁</li></ul>

What Go got right

computerscience

Computer Science

softwareengineering

Software Engineering

INTRODUCTION

Hello.

Let me start by thanking Katie and Chewy for the giving me the honor of presenting the closing talk for the conference. And apologize for reading this from my script but I want to get the words right.

Today is November 10, 2023, the 14th anniversary of the launch of Go as an open source project.

That day, at 3pm California time if memory serves, Ken Thompson, Robert Griesemer, Russ Cox, Ian Taylor, Adam Langley, Jini Kim and I watched expectantly as the site went live and the world learned what we had been up to.

Fourteen years later, there is much to look back on. Today I'd like  to take the opportunity to talk about some of the larger lessons  learned since that day. Even the most successful projects have  things that, upon reflection, could have been done better. And of  course, things that in hindsight seem to have been key to their  success.

Up front I must make clear that I am speaking only for myself, not  for the Go team and not for Google.   Go was and still is a huge effort by a dedicated team and a huge community, so if you agree with anything I say, thank them. If you disagree, blame me but please keep it to yourself 😃.

Given the title of this talk, many people might expect I'm going  to be analyzing good and bad things in the language. Of course I'll do some of that, but much more besides, for several reasons.

First, what's good and bad in a programming language is largely a  matter of opinion rather than fact, despite the certainty with which many people argue about even the most trivial features of Go or any other language.

Also, there has already been plenty of discussion about things such as where the newlines go, how nil works, using upper case for export, garbage collection, error handling, and so on.   There are certainly things to say there, but little that hasn't already been said.

But the real reason I'm going to talk about more than the language is that that's not what the whole project was about. Our original goal was not to create a new programming language, it was to create a better way to write software. We had issues with the languages we were using — everyone does, whatever the language — but the fundamental problems we had were not central to the features of those languages, but rather to the process that had been created for using them to build software at Google.

The creation of a new language provided a new path to explore other ideas, but it was only an enabler, not the real point. If it didn't take 45 minutes to build the binary I was working on at the time, Go would not have happened, but those 45 minutes were not because the compiler was slow, because it wasn't, or because the language it was written in was bad, because it wasn't. The slowness arose from other factors.

And those factors were what we wanted to address: The complexities of building modern server software: controlling dependencies, programming with large teams with changing personnel, ease of maintainability, efficient testing, effective use of multicore CPUs and networking, and so on.

In short, Go is not just a programming language. Of course it is a  programming language, that's its definition, but its purpose was to help provide a better way to develop high-quality software, at least compared to our environment 14 plus years ago.

And that's still what it's about today. Go is a project to make  building production software easier and more productive.

A few weeks back, when starting to prepare this talk, I had a title  but little else.  To get me going, I asked people on Mastodon for  input. A fair few responded, and I noticed a trend in the replies:  people thought the things we got wrong were all in the language,  but those we got right were in the larger story, the stuff around  the language like gofmt and deployment and testing. I find that  encouraging, actually. What we were trying to do seems to have had an effect.

But it's worth admitting that we didn't make clear early on what  the true goals were. Perhaps we felt they were self-evident.   To  address that shortcoming, in 2013 I gave a talk at the SPLASH  conference entitled,  Go at Google: Language Design in the Service of Software Engineering.

That talk and associated blog post are perhaps the best explanation of why Go happened.

Today's talk is something of a follow-on to the SPLASH talk,  looking back on the lessons learned once we got past building  the language and could apply ourselves to the bigger picture more broadly.

And so... some lessons.

First, of course, we have:

The Gopher

It may seem an odd place to start, but the Go gopher is one of the  earliest factors in Go's success.    We knew long before the launch  that we wanted a mascot to adorn the schwag - every project needs schwag - and Renee French offered to create one for us. We got that part absolutely right.

Here is a picture of the very first instance of the gopher plushie.

And here is a picture of the gopher with the less successful first prototype.

The Gopher is a mascot who serves as a badge of honor, even an  identifier for Go programmers everywhere.   At this moment you are in a conference, one of many, called GopherCon.   Having a recognizable, funny creature ready to share the message from day one was vital to Go's growth.   Its goofy yet intelligent demeanor — he can build anything!

— sets the tone for the community's engagement with the  project, one of technical excellence allied with real fun. Most  important, the gopher serves as a banner for the community, a flag to rally around, especially in the early days when Go was still  an upstart in the programming world.

Here's a picture of gophers attending a conference in Paris some  years back. Look how excited they are!

All that said, releasing the Gopher design under a Creative Commons Attribution license was perhaps not the best choice.  On the one hand, it encouraged people to remix him in fun ways, which in turn helped foster community spirit.

Renee created a "model sheet" to help artists work with him while  keeping him true to his spirit.

Some artists had fun playing with these characteristics and making their own versions of him; Renee and my favorites are the ones by the Japanese designer @tottie:

and game programmer @tenntenn:

But the "attribution" part of the license often resulted in frustrating arguments, or in false credit given to Renee for creations that were not hers and not in the spirit of the original. And, to be honest, the attribution was often honored only reluctantly or not at all. For instance, I doubt @tenntenn was compensated or even acknowledged for this use of his gopher illustration.

So if we were doing it over, we'd think hard about the best way to  make sure the mascot stays true to his ideals. It's a hard problem,  maintaining a mascot, and the solution remains elusive.

But on to more technical things.

Done Right

Here is a list of things that I think we got objectively right,  especially in retrospect.   Not every language project has done these things, but each was crucial to the ultimate success of Go. I'll try to be brief, because they will all be familiar topics.

1. Specification. We started with a formal specification. Not only  does that lock down behavior when writing a compiler, it enables  multiple implementations to coexist and agree on that behavior. A compiler alone is not a specification. What do you test the compiler against?

Oh and by the way, the first draft of the specification was written  here, on the 18th floor of a building on Darling Harbour in  Sydney. We are celebrating Go's birthday in Go's home town.

2. Multiple implementations. There are multiple compilers, all  implementing the same spec. Having a spec makes this much easier to achieve.

Ian Taylor surprised us when he sent mail one day informing us that, having read our draft spec, he'd written a compiler himself.

Subject: A gcc frontend for Go

From: Ian Lance Taylor

Date: Sat, Jun 7, 2008 at 7:06 PM

To: Robert Griesemer, Rob Pike, Ken Thompson

One of my office-mates pointed me at http://.../go_lang.html .  It

seems like an interesting language, and I threw together a gcc

frontend for it.  It's missing a lot of features, of course, but it

does compile the prime sieve code on the web page.

That was mind-blowing, but many more have followed, all made possible by the existence of a formal specification.

Having multiple compilers helped us refine the language and polish the specification, as well as providing an alternative environment for others less enamored with our Plan-9-like way of doing business.

(More about that later.)

Today there are lots of compatible implementations, and that's great.

3. Portability. We made cross-compilation trivial, which allowed

programmers to work on whatever platform they liked, and ship to whatever platform was required. This may be easier with Go than with any other language.  It's easy to think of the compiler as

native to the machine it runs on, but it has no reason to be.

Breaking that assumption is powerful and was news to many developers.

4. Compatibility. We worked hard to get the language in shape for

version 1.0, and then locked it down with a compatibility guarantee. Given what a dramatic, documented effect that made on Go's uptake, I find it puzzling that most other projects have resisted doing this.  Yes, there's a cost in maintaining strong compatibility, but it blocks feature- itis and, in a world in which almost nothing else is stable, it's delightful not to have to worry about new Go releases breaking your project.

5. The library. Although it grew somewhat as an accident, as there was no other place to install Go code at the beginning, the existence of a solid, well-made library with most of what one needed to write 21st century server code was a major asset. It kept the community all working with the same toolkit until we had experience enough to understand what else should be made available. This worked out really well and helped prevent variant libraries from arising, helping unify the community.

6. Tools. We made sure the language was easy to parse, which enabled tool-building. At first we thought we'd need an IDE for Go, but easy tooling meant that, in time, the IDEs would come to Go. And along with gopls, they have, and they're awesome.

We also provided a set of ancillary tools with the compiler, such  as automated testing, coverage, and code vetting. And of course the go command, which integrated the whole build process and is all many projects need to build and maintain their Go code.

Also, it didn't hurt that Go acquired a reputation for fast builds.

7. Gofmt. I pull gofmt out as a separate item from tools because  it is the tool that made a mark not only on Go, but on the programming community at large. Before Robert wrote gofmt (which, by the way, he insisted on doing from the very beginning), automated formatters were not high quality and therefore mostly unused.

Gofmt showed it could be done well, and today pretty much every language worth using has a standard formatter. The time saved by not arguing over spaces and newlines is worth all the time spent defining a standard format and writing this rather difficult piece of code to automate it.

Also, gofmt made countless other tools possible, such as simplifiers, analyzers and even the code coverage tool. Because gofmt's guts became a library anyone could use, you could parse a program, edit the AST, and just print byte-perfect output ready for humans to use as well as machines.

Thanks, Robert.

Enough with the congratulations, though. On to some more contentious topics.

Concurrency

Concurrency is contentious? Well, it certainly was in 2002, the  year I joined Google. John Ousterhout had famously written that  threads were bad, and many people agreed with him because  they seemed to be very hard to use.

Google software avoided them almost always, pretty much banning them outright, and the engineers doing the banning cited Ousterhout. This bothered me. I'd been doing concurrency-like things, sometimes without even realizing it, since the 1970s and it seemed powerful to me. But upon reflection it became clear that Ousterhout was making two mistakes. First, he was generalizing beyond the domain he was interested in using threads for, and second, he was mostly complaining about using them through with clumsy low-level packages like pthread, and not about the fundamental idea.

It's a mistake common to engineers everywhere to confuse the solution and the problem like this. Sometimes the proposed solution is harder than the problem it addresses, and it can be hard to see there is an easier path. But I digress.

I knew from experience that there were nicer ways to use threads,  or whatever we choose to call them, and even gave a pre-Go talk  about them.

But I wasn't alone in knowing this; a number of other languages,  papers, and even books had been written about concurrent programming that showed it could be done well. It just hadn't caught on as a mainstream idea yet, and Go was born partly to address that. In that legendary 45-minute build I was trying to add a thread to a non-threaded binary, and it was frustratingly hard because we were using the wrong tools.

Looking back, I think it's fair to say that Go had a significant  role in convincing the programming world that concurrency was a powerful tool, especially in the multicore networked world, and that it could be done better than with pthreads. Nowadays most mainstream languages have good support for concurrency.

Also, Go's version of concurrency was somewhat novel, at least in the line of languages that led to it, by making goroutines unflavored.  No coroutines, no tasks, no threads, no names, just goroutines. We invented the word "goroutine" because no existing term fit. And to this day I wish the Unix spell command would learn it.

As an aside, because I am often asked about it, let me speak for a  minute about async/await.   It saddens me a bit that the async/await model with its associated style is the way many languages have chosen to support concurrency, but it is definitely a huge improvement over pthreads.

Compared to goroutines, channels, and select, async/await is easier and smaller for language implementers to build or to retrofit into existing platforms. But it pushes some of the complexity back on the programmer, often resulting in what Bob Nystrom has famously called "colored functions".

Reflections on Go's 14th Anniversary - Rob Pike

Why Go was created

What Go got right

Other curated ideas on this topic: