GitHub Models: An Overview of AI Prototyping

Welcome to the GitHub Models series! This series will guide you through using GitHub Models to enhance development workflows by integrating AI into projects. We'll explore key features, learn to prototype AI models, integrate them into workflows, and cover best practices. By the end, you’ll be ready to leverage GitHub Models effectively!

Prerequisites

Before diving into the GitHub Models series, it's helpful to have a basic understanding of the following concepts:

Familiarity with GitHub: Basic knowledge of how to navigate GitHub, create repositories
Git commands: General understanding of common git commands
AI and Machine Learning: A general grasp of what AI and machine learning are, including key terms and concepts like models, training, etc.
Optional but nice to have: Knowledge of common models like Llama 3.1, GPT-4o, etc.

What Are GitHub Models?

Anyone can be AI engineers. Everyone should have access to develop and test AI models. That is what I believe in today’s era of AI-driven application development.

GitHub Models is a suite of capabilities provided by GitHub that enable developers to prototype, integrate, and utilize AI models effectively. In addition to these AI capabilities, developers can streamline their workflows, enhance collaboration, and ultimately create more efficient and innovative solutions - all right in GitHub!

I believe the introduction of GitHub Models represents a significant shift in how developers can approach software development, especially in the realm of AI. That is why I’m super excited to be writing this series.

The Benefits of Using GitHub Models

Faster Prototyping

With GitHub Models, developers can quickly experiment with different models. They can quickly dive into exploring various types of model. This rapid prototyping allows teams and individual hobbyists to test and iterate on their ideas more effectively.

It is also super easy to spin up an environment in GitHub Codespaces after exploring the models in the playground, making it super easy to get your hands dirty with the code and make various projects.

For a guide on using Codespaces, feel free to check out this article.

Easy Access to Multiple Models

GitHub Models allows any developer to quickly learn and explore. With the rise of more models available, developers can stay updated using GitHub Models' easy-to-use and accessible playground. This allows them to continuously test and compare the pros and cons of different models, all within GitHub.

Additionally, GitHub Models make advanced AI capabilities accessible to developers of all skill levels. Even those with limited experience in AI can utilize these models to enhance their projects, lowering the barrier to entry for adopting AI technologies.

Scalability & Automation

As projects grow in complexity, GitHub Models can scale alongside them. Developers can easily integrate more sophisticated models into their workflows, adapting to changing project requirements without significant overhead.

Ever heard of MLOps with GitHub Actions? GitHub Models can easily be integrated with GitHub Actions, allowing developers to automate tests, prompt evaluations and any actions in the MLOps pipeline.

Need a beginner-friendly tutorial on MLOps with GitHub Actions? Check this series out!

Or check out an in-depth GitHub Actions 101 series!

Getting Started with Prototyping AI Models

Now that we’ve established the value of GitHub Models and why we should use it, let’s dive into how we can get started with prototyping AI models.

Join the waitlist

At the time of writing this article, GitHub Models is currently in limited public beta. If you would like full access, you would need to join the waitlist.

Access GitHub Models Playground

To access GitHub Models, you’ll need a GitHub account. If you don’t already have one, visit GitHub's website to create an account.

Then, head over to GitHub Marketplace to view the library of models available.

Now, if you select a model, let’s say “OpenAI GPT-4o”, you will navigate to a page where you can see description and details about the model. In the top right corner, click on Playground to launch the model’s playground.

Exploring the Playground

This is my favourite feature of GitHub Models. You get to adjust parameters of the model such as:

Adding a system prompt
Adjusting max tokens, temperature (randomness), top P and stop strings

It is so easy to experiment and test the model thanks to this feature! And best of all, it’s all in GitHub! There’s no need for you to spend time to spin up and test this model in another environment.

And let’s say while experimenting with the model’s parameters, you want to see the code - well that’s super easy too! Simply click on the ‘Code’ tab as shown in the screenshot below.

You can change the programming language on the top-right and the SDK you are using to access the model. For this model, we can use either OpenAI SDK or Azure AI Interference SDK. This makes it very convenient for when you need to use the code to build your own AI projects!

Note that all models in GitHub Models can use Azure AI Interference SDK. Other SDK options would depend on the model.

Example of code using Azure AI Interference SDK:

Test and Compare Models

Now that we are familiar with the playground, let us learn how to test and compare models. As developers, we know that each model has their own unique strengths and use cases. So it is important that we can test and compare models fast to build applications that suit our use cases.

In GitHub Models, you can compare up to 2 different models at a time. To do that, go back to the Chat tab and click on ‘+ Compare’ as shown below.

Now you will see the 2 models side by side. Select the 2nd model you would like to compare with in the dropdown menu.

For this example, let’s see how well the “Phi-3-mini-instruct (4k)” model can perform for the same prompt and parameters. As you can see, “OpenAI GPT-4o” is the winner for this use case.

As written in GitHub’s documentation, the playground is rate limited. To view the details, read the Rate Limit documentation.

Prototyping Models in Codespaces

Let’s say you’ve compared a few more models and decided to go with “OpenAI GPT-4o” model. You want to tweak some code to make a quick prototype, you can easily create a new Codespaces environment by clicking on the Get API key button.

A popup window with some getting started instruction will show. Skim through it and click ‘Run codespace’. An environment should spin up immediately and you can test code with no hassle to configure your API credentials since Codespaces already handles that for you.

Now you are ready to test and tweak the code samples for your project! Continuing our FAQ bot example, I created a file called nutrition_bot.js and add the same system prompt as we see from the playground.

Then, in the terminal, I run:

node samples/js/openai/nutrition_bot.js

If everything runs perfectly, you should see that the terminal output is as expected!

Conclusion

Thanks for reading! In this first part of our series on GitHub Models, we’ve learned some key features of GitHub Models and how GitHub Models can transform your experience with experimenting and prototyping models.

In the next part, we will delve deeper into integrating AI models into our development workflows, providing practical examples and insights to help us make the most of these powerful tools. If you have any questions, do let me know in the comments below. If this article is helpful, be sure to leave a like and share the article around! Stay tuned for more! Cheers!

Let's Connect!

References

Image credits:
https://github.blog/wp-content/uploads/2024/07/six-models.png?w=1024