Over the last fifteen years, GitHub has become an indispensable part of the world of coding. The platform, launched in 2007, is now used by over 100 million developers to collaborate, track changes, and store their code.
Acquired by Microsoft in 2018 for $7.5 billion, GitHub is now developing a reputation for something else: radically changing the way coding works. The catalyst is Copilot, an AI tool that generates code as easily as OpenAI’s ChatGPT produces sonnets. (In fact, Copilot is powered by OpenAI’s Codex, a large language model trained on code scraped from the Internet.)
By simply pressing the tab key, a developer using Copilot can finish a line, generate blocks of code, or even write entire programs. According to GitHub, over 10,000 organizations, ranging from Coca-Cola to Airbnb, have signed up for Copilot’s enterprise version, and more than 30,000 employees at Microsoft itself now regularly code with assistance from Copilot.
“Sooner than later, 80% of the code is going to be written by Copilot. And that doesn’t mean the developer is going to be replaced.”
Recently, Freethink spoke with Thomas Dohmke, GitHub’s CEO, to learn more about how Copilot promises to refashion programming as a profession, and the questions AI-powered development raises about the future of innovation itself. We also talked about why coding with Copilot is so much fun, how AI is going to change the way we learn, and whether Copilot can fix banks that are still running COBOL on mainframes.
This interview has been edited for length and clarity.
Freethink: What changes do you see coming down the pike, in terms of the nature of programming? Is software development going to remain the domain of experts? Or do you think that AI-fueled, no-code development will become more widespread — to the point that what it means to be a software engineer looks different?
Thomas Dohmke: I think that systems thinking — understanding the complexity of software and being able to take a really big problem, big challenge, big new feature and decomposing it into small problems — will play an ever-increasing role.
We went from punch cards to hundreds to thousands to millions and probably now trillions — if not whatever the next thing is after trillions — of lines of code. And so we, as developers, need to be able to manage these large complex systems that solve large, complex problems. And you need to be able to break them down into small building blocks.
The skills of the developer will be to figure out, “How small do I have to go until I can leverage AI to synthesize that code for me?” At least for the foreseeable future, we’re still going to have to review all that source code — understand what it does, and do security reviews, compliance review, see if you’re having a prompt injection, where somebody tries to inject harmful code into the code base.
The developer will still be the expert, understanding the code and whether what was synthesized by the AI actually matches the developer’s intent. That’s probably going to shift over the next decade — how much are we writing ourselves and how much is just synthesized. But I don’t think we are remotely close to where everything will be just no-code development.
“The number of distractions doesn’t go down, but you can leverage the creative time better.”
Freethink: What are you seeing in how the day-to-day work of programmers is changing thanks to Copilot?
Dohmke: Developers don’t actually spend most of their time coding these days — between two and four hours per day is when a developer writes code. The rest of the day they do other things, like stand-up meetings, looking at crash reports. (Or maybe somebody in the App Store reviews is telling you that the latest version is broken, and you can’t even reproduce the bug.)
With Copilot, if you only have two to four hours a day to actually code, you can use that time better. You can use that time to stay in the flow, to get the job done and enjoy doing it.
The number of distractions doesn’t go down — whether it’s Slack, or Twitter, or your cell phone and your WhatsApp messages. Those don’t go away, but you can leverage the creative time better, and you can kind of zone into the problem, because if you’re not constantly switching out of the editor to something else, you also have fewer triggers to get distracted. So we see that people are staying in the zone more often.
“Copilot brings the fun back, it brings the creativity back, it brings the flow back.”
Freethink: How has your own process of writing software changed? I don’t know how much time you’re able to devote to coding these days, but I’m really curious about how your own coding process has evolved thanks to Copilot.
Dohmke: It’s mostly hobby projects for me. But the first thing that comes to my mind: Copilot brings back the fun. Oftentimes, there’s lots of distractions, and there’s lots of other things you could do. The weekend — our time away from work — is, in a way, a competition around attention span: Do you want to go for a walk? Or do you want to build some Lego sets? Or do you want to code?
When I find an hour to write software, Copilot brings the fun back, it brings the creativity back. It brings the flow back, because you don’t have to constantly context-switch between the editor and the browser and search engines, and things like Stack Overflow or GitHub itself, to find those three lines of code that you need to — I don’t know, make a rounded corner on your iPhone app that you’re building.
And so I think it brings back this feeling of “I’m building my big lightbulb idea” that I had in the moment, in that hour or two I have on a Sunday morning before other things get priority.
“Startups understand that even a 20-25% productivity gain may decide their survival.”
Freethink: How do you think your own experience as a founder would have been different if you’d had access to a tool like Copilot when you were creating your first startup?
Dohmke: I think it would have made my life much easier. As a small startup founder, you can move really fast because that’s the only way to survive. So then having a tool available that keeps you in the flow, a tool that lets you get to a prototype, get to a preview, and then, ultimately, the product-market-fit is incredibly empowering.
I talk with a lot of startups, and I’m an angel investor in a few, and they’re all using Copilot because they understand that even a 20-25% productivity gain may decide their survival.
“The response was mind blowing. People told us that they can’t develop without Copilot anymore.”
Freethink: I’m curious what data you’ve gathered — either anecdotally or more formally — about how Copilot is changing how GitHub and Microsoft’s own programmers work.
Dohmke: We started working on Copilot in August 2020, and then rolled out the first internal preview in early 2021 to about 250 GitHub employees, or “Hubbers,” and the response that we got was mind blowing. People were excited. People told us that they can’t develop without Copilot anymore.
We measured net promoter score, NPS [ed. note: a measure of user’s satisfaction], which goes from minus 100 to plus 100. And we had outcomes of +73 across our population of 250 testers. We never see those numbers in any other change to the development environment. Typically, developers are like, “Don’t touch my system, I know what I’m doing, Thomas — why are you introducing a new tool?”
We saw tremendous positive feedback from our team, and the retention numbers, how many people kept using it after the first day or the first week, were incredibly high — higher than any other product that we built. You expect a certain drop off — you launch a new feature within the company, and everybody’s excited on day one, and then by day three, everybody keeps moving back to their old habits. And we haven’t seen that with Copilot, whether it’s within GitHub, within Microsoft, or really across our customers.
“Almost half the code on average is written by Copilot. That alone is mind boggling.”
Freethink: Do you have any facts or figures about how the broader community of developers on GitHub is using Copilot, and what effect it’s having on their productivity?
Dohmke: When customers use Copilot, we track how often they press the tab key and how much code is written by Copilot in those files. And that number is now an average 46% — almost half the code on average is written by Copilot in those files where it’s enabled. That alone is mind boggling.
We started working on Copilot three years ago; it went into preview almost two years ago. That number has been steadily going up ever since we launched. Sooner than later, 80% of the code is going to be written by Copilot. And that doesn’t mean, as we discussed, that the developer is going to be replaced. That means that the developer has more time to focus on the 20% that they’re writing.
When you use Copilot, your style changes, because you become more explicit. You write code comments and maybe a little example. When you write a function with an input and an output, just by providing example inputs and outputs, Copilot “gets an idea” of what you wanted to do and uses that as additional context to synthesize the code.
“The group with Copilot was 55% faster. That is going to change the industry.”
We actually know from a quantitative survey with 2,000 developers how they feel about using Copilot, and 75% of those developers say they’re more fulfilled. We hear from them that they require less mental capacity to get the job done. They’re spending less time on writing the boilerplate that we always have to do as developers.
We also did a qualitative study, where we gave 50 developers access to Copilot and had about 50 developers who didn’t have Copilot — the control group, if you will — and we asked them to build a web server. The group with Copilot was 55% faster. That is going to change the industry.
Not only were they 55% faster, they actually had a higher success rate: 78% compared with 70%.
“I think the way we learn will completely shift.”
Freethink: What effect is Copilot going to have on the development of future programmers? There’s this tension when you’re learning to program, between getting frustrated with things that seem like they should be easy, and the fact that by doing them repeatedly, you eventually gain a deeper understanding of how it all works. I’m curious what effect a tool like Copilot might have for, say, a high school student who’s learning how to do a bubble sort and can just press tab and have the code appear, without ever needing to do it from scratch first.
Dohmke: I think the way we learn will completely shift, and the way we will teach in school will also completely shift.
I think we will focus less on knowing who was the president in 1845 and focus more on how we learn. The teacher becomes more of a coach that helps kids at all ages to leverage technology to explore these topics by themselves and at their own speed. I think this was going to be incredibly democratizing for kids, specifically on software development.
It’s easy to forget here in the United States that the most common language in coding is English. If you grow up here, you probably have an easier time understanding English than if you grow up in Brazil, China, or Germany. That alone will be liberating for kids because they can just query an AI agent, like Copilot, in their native language. Copilot will understand those languages and still synthesize the code in the programming language that the student is learning, but the students can interact and ask questions in their own language.
I think knowledge will matter less and knowing how to reason, knowing how to explore content — knowing how to figure out whether the content provided by the AI is actually the right answer is going to be crucial. Teachers will work with kids to learn how to do that. And the students will have to figure out, is what the AI is giving me actually the answer to the problem that I’m solving, or is the model hallucinating?
“Kids can just query an AI agent, like Copilot, in their native language.”
Freethink: At least to my mind, it seems in whatever discipline you’re picking, whether it’s medicine or computer science, in order to do that kind of higher-level reasoning you have to start with some level of content knowledge.
Dohmke: If you use Copilot, you will find out relatively quickly that you still have to have this baseline knowledge. You can’t get away without it. You can go to ChatGPT or Copilot and ask it “how to build GitHub,” and it will give you a very abstract answer.
You have to have the ability to say, “OK, now how do I decompose this really big problem into small problems?” And to do that, you have to have the understanding of what’s the database, what’s a virtual machine, what’s a container? You have to know all these things in computer science to actually get to the point where the AI can solve your question.
“We keep fighting for the rights of open-source developers because we believe that open source is fundamental to achieve human progress.”
Teachers, if you go back to the school example, are going to immediately find that out. If a student just tries to cheat their way into solving their homework, they will not only need to provide the answer, they will need to show their work. It’s like math class: show your work of how you used Copilot to build something.
The other thing is, no software project is the same, and there’s a big difference between building a point-of-sale system for a supermarket or building an iPhone app that runs on each individual device, but might still have 100 million users. You still need to learn about these concepts and how to solve these problems. And I think that’s a much more useful skill for us as humans than knowing all the details of the programming language.
Freethink: As someone who’s been very strongly in favor of open source, what role do you see open source playing here?
Dohmke: Open source is going to play a fundamental role in the evolution of Copilot. The model that OpenAI provides for Copilot was trained on open data on the Internet. Not only source code, but also all kinds of other documents. That allows Copilot to not only speak programming languages but also human languages. Open source provides the data sets for training these models.
We have now seen a number of open-source large language models. A lot of them are actually hosted on GitHub, which excites us! GitHub has for the last 15 years embraced the open-source ecosystem we provide. Basically all our features were free to open-source projects, and we want to keep supporting this ecosystem. We keep fighting for the rights of open-source developers because we believe that open source is fundamental to achieve human progress.
The collaboration model that comes with open source is probably the most border- and boundary-less collaboration model that exists: developers across many countries work with each other and don’t care about all the other conflicts that we have on this planet.
“Languages go out of vogue. But they also never go out of service.”
More often than not, when you get a pull request, you don’t know who that is, what their title is, what they’ve been working on, whether they went to university or not. You don’t care. I don’t think if you want to contribute code back to the Linux kernel anybody will ask you, “Do you actually have a proper education and five years of work experience?” That’s not how open-source collaboration works.
At GitHub, this is what we have embraced and will always embrace as part of our mission to empower and enable open-source developers. I think large language models will play a huge part and will help us together accelerate progress.
Freethink: What role do you think Copilot might play in developing new programming languages? Languages rise in favor, go out of vogue — I’m curious what effect Copilot might have on that.
Dohmke: You said languages go out of vogue. I think that’s right. But they also never go out of service. The most common example here is from banks and financial service institutions that are still running COBOL on mainframes. And it’s not just one bank somewhere, it’s any bank I’ve spoken to in any country around the world that still runs COBOL on mainframes.
It’s a problem, because the people that wrote COBOL in the ‘60s are, luckily, all retired by now and no longer have to maintain that code for the banks. Copilot will play a role in helping companies to modernize their code base — and it will do so in many ways.
For one, Copilot can explain the legacy code to the developer that now has to own it. You can highlight the code and say, “Explain this code to me.” And once you have an explanation, you can actually use the explanation and the code to then generate test cases for that code. The next logical step is that you leverage the AI in an iterative process with the “pilot,” the developer, to rewrite everything in Python. And because you generated the tests and you generated the description — neither of which likely existed in the ‘60s; a lot of that code is undocumented — you are able to transform that old code base into a new code base.
“We have a product called Copilot Voice that helps you to code with your voice, instead of your hands.”
We are all fighting the tech debt of not only the past generation of developers but of our own six-month-old code base — most developers, whatever they wrote six months ago, when they look at this now, they say, “I would do it much better now than six months ago.” We will see more iteration and hopefully less baggage that we’re carrying along with us.
I think another way is that it will be just easier to learn coding, and programming languages themselves will probably shift to be more human. Instead of writing a lot of code, I might just write a lot of comments and then synthesize the rest of the code. And those comments might be in German, Brazilian Portuguese — really almost any language will be understood by these models for the purpose of synthesizing code.
The last one is that we have a preview of a product called Copilot Voice that helps you to code with your voice, instead of your hands — that will give access to folks that have disabilities who cannot type all day because they have repetitive strain injury.
Freethink: When I was teaching high school students programming, the biggest jump for them was understanding what is different about coding language versus natural language. What frustrated them was how computers do exactly what you tell them — like an annoying sibling who will do the obviously wrong thing, and then say “Well, technically, you said this.” With tools like Copilot, do you think there’s going to be a convergence, where coding becomes more similar to the way we communicate person-to-person?
Dohmke: I think you can see that already. When you use these chat agents, they have not only the sentence you just typed, they have the context of the previous conversation. For Copilot Chat, it actually knows what’s in your editor — not only in the file you have open, but actually in adjacent tabs, and in your debugger. In the future, they’re probably going to have the whole repository or file folder available to them as context.
Today, you can already write an HTML file — a web page — and as you write it, with the help of Copilot, you can say, “Make this text bold.” Or, “Center this area on the page.” And it then shows you how to do that, and it removes that frustration in the past when we thought the computer doesn’t understand what I want. Well, now it does.
“One of the best moments when you interact with Copilot is that you can tell it that something’s wrong.”
Just recently at a conference, I built a Snake game as a Copilot demo. I said, “Make the game prettier,” and it just output the style sheets. I never said I want a style sheet. I never even prompted it with the concept of HTML or CSS. It figured that out itself by having the context of what I was trying to achieve.
I think one of the best moments when you interact with Copilot is that you can tell it that something’s wrong. You can actually say, “This is not what I was looking for,” or “This doesn’t work.”
And then it says, “Oh, I apologize. Here’s another thing you can try.” So it has this interactive mode, and you don’t have to know the exact syntax for the programming language. You don’t have to know what you would have to know to solve that problem. You can explore it and get to the solution yourself.
“Even though they were only trained on existing art, they can generate a whole new category of art.”
Freethink: As I understand it, Copilot is built on OpenAI’s Codex, which is sort of a snapshot from the past, of code prior to 2021. I don’t know if there are plans to connect Copilot to the Internet, so it gets updated more frequently, but I’m curious if you see any potential loss that might come from not having to write boilerplate so often. When you have to get in the weeds all the time, one person in 100,000 might eventually be like, “Wow, here’s a way to do this much better.” And then that solution propagates throughout the developer community. If folks are just asking Copilot for the best solutions based on yesterday’s code, how do you see innovation itself changing?
Dohmke: The thing to consider first is that large language models do not have a library of code snippets stored in them. They’re neural networks that have billions of parameters that describe how to predict the probability of something. In terms of language models, that something is the next word. In terms of diffusion models, it’s how to get from noise to a picture.
If you take those diffusion models as a visual example, they clearly can create new things that didn’t exist before. Even though they were only trained on existing art, they can generate a whole new category of art. You can go to Stable Diffusion and say, “Render the skyline of New York as it would be painted by a French Impressionist.” And then it actually renders you today’s skyline of New York as if it were painted by Monet or some other French impressionist.
There’s no picture in the training set that actually looks like that. The picture generated itself based on your prompt and ultimately out of noise.
I think the same is true for language models. They can generate completely new text, completely new code that was never there before, and as such, they might, in partnership with the developer, actually create innovation and create new algorithms or new code. But, as you can infer from the term “Copilot,” we believe there’s a pilot involved that works with the Copilot iteratively to get to this innovation.
We’d love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at [email protected].
Leave a Reply