convedo Digital Experts
part of convedo
What is Codex?
What is Codex?
OpenAI Codex is a descendant of GPT-3 which stands for Generative Pre-trained Transformer 3. GPT-3 is a massive deep learning language model, trained using internet data to generate any type of text. Developed by OpenAI, it requires a small amount of input text to produce large volumes of relevant and sophisticated machine-generated text. As of early 2021, GPT-3 is the largest neural network ever produced and, as a result, it is better than any prior model for producing text that is convincing enough to seem like a human could have written it. In addition to generating text, GPT-3 can also create workable code using only a few snippets of example code text.
Codex is a new AI system created by OpenAI and is a version of GPT-3 that has been finetuned for programming tasks. OpenAI Codex has broad knowledge of how people use code and is significantly more capable than GPT-3 in code generation, partly because it was trained on billions of lines of source code from publicly available sources. OpenAI Codex has much of the natural language understanding of GPT-3 but it can produce working code in any piece of software with an API using simple commands in English as input. As a general-purpose programming model, it can be applied to essentially any programming task. The Codex model available via the API is most capable in Python but is also “proficient” in JavaScript, Go, Perl, PHP, Ruby, Swift, TypeScript, Shell, and others.
Use cases
Codex is designed to speed up the work of professional programmers, as well as help amateurs get started coding. For example, codex can be used to perform tedious tasks such as rendering web pages, launching web servers, and sending emails. Alternatively, it may be used to build simple websites and rudimentary games using natural language, as well as translate between different programming languages and tackle data science queries i.e. plotting charts in Python’s Matplotlib library. These are not complicated tasks, but they’re tedious and error-prone processes, and they usually require browsing programming forums, so, having an AI assistant writing this kind of code for you can save valuable time.
Short example
The example above shows how well the model was able to understand what the user wanted from their instructions and translate this into the code that generated the landing page. Codex empowers computers to better understand people’s intent, which can enable everyone to do more with computers.
Key features of Codex
- Its interpretation of natural language commands is creative and precise. It can determine exactly what the user wants even when the input text is informal, ambiguous or the user includes metaphors.
- The model can self-reference. It can store in memory (within the same session) previous commands and use them adequately. As a result, the user can refer to things they previously defined using the word “it” and the model will understand what item they are referring to. This demonstrates a certain understanding of syntax and context.
- When using adjectives to describe the relative size of an input object, Codex understands the meaning of the adjective and also has a good sense of the relative nature of it. The example below shows how Codex changes the size of the boulder to make it small relative to the size of the human.
Limitations
- Codex is sample-inefficient, in the sense that even inexperienced programmers can be expected to solve a larger fraction of problems despite having seen far fewer than the model was trained on.
- When exploring prompts on which Codex is likely to fail or display counter-intuitive behaviour, it was discovered that Codex can recommend syntactically incorrect or undefined code, and can invoke functions, variables and attributes that are undefined or outside the scope of a codebase.
- Like other large language models, Codex generates responses as similar as possible to its training data, leading to unclear code that looks good on inspection but does something undesirable. As an example, OpenAI found that Codex can be prompted to generate racist and otherwise harmful outputs as code.
Summary
- Codex provides a new way to interact with existing software. By improving human-computer communication, Codex enables computers to understand what we are asking and translate this into code.
- Codex allows for higher interpretability. Although it is trained as a black box, Codex allows us to see the code it writes from our prompts. We don’t simply receive the output from the computer, but also what Codex interpreted.
- Instead of programmers having to understand how to write the code that solves a problem, Codex enables programmers to simply need to understand how to solve the problem, which is generally considered the most interesting part of programming.
What Is Important To You?
Would you like to exchange ideas with us on the subject of digital transformation and process automation without obligation? Let’s talk!
