⚡️TLDR;
GitHub Copilot suggests code based on the coding tabs you have open. It’s context aware but it’s not actually reading your entire code base yet.
🎩 Limits of LLM Magic
LLM means Large Language Model. It basically means there’s machine learning that read a ton of text (or code) and can guess what the next line of text should be.
From the article:
GitHub Copilot can process about 6,000 characters at a time
This means copilot can only read 6000 characters to guess how to auto-complete your code. So it can’t read your entire code base (yet)
🤔 How does it choose those magical 6,000 characters?
Here’s how a prompt is created:
Algorithms first select relevant code snippets or comments from your current file(s). Other file tabs are included
These snippets and comments are then prioritized, filtered, and assembled into a prompt that asks the LLM to give a coding suggestion
🔧 High bar or low bar for suggestions?
Low-bar. The more people accept a suggestion, the more data the Copilot LLM has on making a better suggestion.
💡 Is it getting suggestions from anywhere else?
It may seem obvious now but suggestions used to be generated by everything before your cursor. That means none of your code after the cursor would be used as input for coding suggestions.
Now copilot adds code after your cursor and gave it a fancy name called Fill-In-The-Middle
🏁 API calls are slow. How did copilot make them fast?
The magic behind LLM is Vector Databases. This makes generative AI output fast enough that auto-suggestions feel seamless.
Faster API’s means faster developers. Developers who use github copilot are 55% faster than developers who don’t.
📖 Give me more system design resources!
If you found this article helpful, you’ll enjoy amazingly detailed system designs from ByteByteGo’s System design Interview.
They’re more detailed and catered to interviews. Some are byte-sized, some are not. But it’s all amazing.
💰 HELP WANTED
This newsletter has grown to 8500+ → 9600 AMAZING READERS. It’s grown to a scale that a single person can’t maintain all of it on their own.
If you’re interested in being a byte-sized design writer, apply here!
📝 Official Article
(Links to official article and sources are available to paid subscribers. They help maintain and support this newsletter!)
Keep reading with a 7-day free trial
Subscribe to Byte-Sized Design to keep reading this post and get 7 days of free access to the full post archives.