GitHub Copilot – A new tool which writes its own code. Is the future here?

What is GitHub Copilot?

Well, as we all know GitHub Copilot was created by the Codex, the new AI system powered by OpenAI. GitHub understands context more than code assistants. So whatever you have provided eg, code, docstring, comment, etc GitHub copilot uses the context and synthesizes code to match.

As per GitHub,

Skip the docs and stop searching for examples. 
GitHub Copilot helps you stay focused right in your editor.

Through more innovation comes easy-peasy tips and techniques. To help programmers write better code, OpenAI and Microsoft have come together to release a technical preview of GitHub Copilot. As of January 2020, Github reported 190 million repositories and 28 million (including) public repositories, making it the world’s largest host of source code. The GitHub tool is acquainted with specific functions and features that help developers quickly recover and discover alternative ways for problem-solving, writing tests, and new API exploration.

Let’s understand how it functions

Automatic pilot to make your coding easy and less lonely

What is Copilot based on?

Copilot works on OpenAI codex, an AI system trained on a dataset made up of available sizable chunks of public source code. Copilot works with a broad set of frameworks and languages. For languages like Python, Ruby, JavaScript, and Go the technical preview is ideal.

Copilot isn’t the first of its kind. Tabine and Kite are two popular tools that cover the same functionality. Why does the Copilot stand out? It is powered by Codex and a descendant of GPT-3 and thus it provides a deeper understanding of the context as compared to other assistants. Codex is similar to GPT-3. It is trained on a huge amount of publicly available coding data from GitHub repositories.

How good is it?

Developers at GitHub are conducting tests and experiments to measure tool accuracy. It is tested on a set of Python functions that have good test convergence in open-source repositories.

All function bodies were deleted and the only context was provided, which includes function name and docstrings. The copilot filled it correctly 43% of the time and the accuracy level increased to 57% after 10 attempts.

Code suggestion by GitHub Copilot for Ruby

This is an exceptional achievement. Copilot truly made a remarkable feat since the tool generated working code that can be used in projects. This will be used by millions once it goes public and will significantly speed up the development.

To get the most out of it, it is suggested to divide code into small functions and provide meaningful names, parameters, and docstrings.

The above example proves it truly is your pair programmer – who comes up with best engineering practices and in return learns from code to improve suggestions.

Works with multiple languages

The best feature of GitHub Copilot is that it works with a broad set of frameworks and languages, and a technical preview

works well with Python, JavaScript, Ruby, and Go. As per the tech preview page, Github Copilot is currently available as a Visual Studio Code extension. It works where Visual Studio code works, and it is fast enough to use.

Copilot seems a potentially fantastic learning tool for developers of all abilities. It removes barriers to entry.

Code suggestion by GitHub Copilot for Python

What is the code quality generated by Copilot?

AI is still improving!! There is a long way to go. Though Copilot is correct almost half the time, the creators comment on its output by saying we should monitor it.

“GitHub Copilot tries to understand your intent and generate the best code it can, but the code it suggests may not always work or even make sense.” —— GitHub Copilot

The suggestion quality depends on existing code, and Copilot uses the current file as a context. As it cannot test its code, it does not run or compile. It’s like using the tool at your own risk. It might give you old or depreciated versions of frameworks and libraries.

Issues are raised around Copilot’s training set as it contains code written by millions. The main question is does it ever repeat the code from the training data? And the answer is yes, of course it does. As per observation, the chances of the leak by the tool is 0.1%

With time, how will GitHub Copilot evolve better?

Neither Codex nor Copilot is perfect. Both tools work fine on given testimonies of those who pre-tested it. As humans write the code, Copilot’s code needs to be tested, reviewed, and vetted.

GitHub developers have created a safety mechanism to provide the best experience to the user but systems being artificially monitored sometimes produce undesired outputs including biased, discriminatory, abusive, or offensive outputs.

Copilot code may not be compiled as the code output has not been tested. So, we can say it is an early version of a new category of tools. It has flaws and is under process, and very few developers have access to it.

GitHub Copilot does not test the code it suggests and thus code may not even compile or run. GitHub Copilot can hold very limited context so even single source files last longer than a few hundred lines that are clipped and immediate preceding context is used. It suggests old and depreciated uses of libraries and languages. Use it at your own risk and use the code anywhere. Still testing is going on.

More and more developers will start using it and it will slowly and gradually evolve and with insights, it will come up with more sorted solutions.

For JavaScript

Stochastic parrot?

Timnit Gebru and other authors coined the term for an AI system as ‘Stochastic parrot’ that is it reproduces what it learns during the training period.

GitHub Copilot is a code synthesizer and is not a search engine. However, the vast category of code it suggests is unique and has never been used before. Still, the code duplication can’t be overruled. The team noticed that 0.1 percent of time suggestions may contain verbatim snippets of code from the training set. This happens when developers have not provided sufficient context or when there is a universal solution to a problem. A few users pointed out that since Copilot uses public code, it should be considered as ‘open-source code laundering.’

Are people criticizing GitHub?

The announcement of GitHub has brought a revolutionary kick on social media. The initial reaction is largely positive, and many people are calling the coding

assistant a game-changer.

It is interesting to see that from a slight chance of the tool you can generate an exact copy of some code from the training set. To make sure it does not turn into a problem, GitHub is building an origin tracker to print out occurrences of verbatim code.

Ownership of code created with copilot is a source of discussion on social media. GitHub assigned the rights to the person writing codes since GitHub is a tool, like a compiler or a pen. GitHub does not take responsibility for the code written with copilot- thus it does not exempt developers from testing and reviewing code.

Testing and reviewing are heavily advised as there is a lot of public code in the world with insecure coding patterns, bugs, or references to outdated APIs or idioms. It helps a bit with repetitive tasks and makes things easier for new developers. Microsoft claims it works with multiple languages- especially Python, JavaScript, Ruby, and Go.

More Developer Productivity, Better Code Quality

In copilot technology, developer productivity is very important and can be achieved by internal users. Better code quality is also an important goal as the pool of developers is broadening. Newbies can learn how to code, and existing ones can learn new languages faster.

As per Holger Mueller – The world needs more developers for coding. The concept of developer velocity is now putting an end to the era of “one developer and his toolchain” with the infusion of AI into the process. In the initial step, they may be skeptical but developers embrace any help from the vendors like now they will get from GitHub to increase their productivity.

Well, Mr Holger, is Copilot ground-breaking or potentially game-changing?

The answer is “no.” As per Mueller, ML is out there and needs to be applied as often as possible. A copilot is important for testing software. The tech preview page insightfully notes, “Tests are the backbone of any robust software engineering project.” Import a unit test package and let GitHub Copilot suggest a test that matches your code implementation.

Is GitHub Copilot a threat to your job?

GitHub Copilot is an AI pair programmer. It would be best if you were more innovative by creating great behavioral models that are far better than structured data implementation, as most of the copilot belongs to the code smell area. Very soon, transformers like copilot will replace lazy programmers. Well, this is quite controversial as AI is still in the development stage, and every day, something new is added. Thinking about how AI can displace a large portion of developers and how copilot can be more augmentative than disruptive to the developer ecosystem? Software development is entering into the third wave of productivity change. The first was the creation of tools like compilers, debuggers, garbage collectors, and languages that made developers more productive. The second was open source with a global community of developers, and the third revolution was the use of AI in coding. The problem-solving attitude and time taken may change with this new introduction. But in the end, there will always be problems for humans to solve. So, for developers, it is safe for the immediate future. Let us know your views, and share your thoughts in the comment section.

Related Blogs

The Python FAQ- Quick Answers to Common Python Development Company Questions

AI Writing Poems: Building LSTM model using PyTorch

Frequently Asked Questions