Amazon Bedrock Breakdown

Last week Amazon launched its Bedrock service to GA. Bedrock is a fully managed service that makes it easier to deploy foundational models (FMs), like those from Anthropic, Stability AI, Cohere, and Amazon itself.

Amazon is playing catchup to OpenAI, who is the clear leader in this space right now. Even though they are launching a year after ChatGPT took the world by storm, there’s still plenty of opportunity for Amazon and its partners.

In this article I’ll breakdown what Bedrock is, its key features, and what this means in the competitive landscape.

Bedrock Basics: What is it?

Amazon Bedrock simplifies building apps with generative AI features, such as LLMs, image generators, and agents. It’s basically one API connected to many foundation models (FMs).

Think of it as a Lego set, but for generative AI. The pieces? High-performing Foundation Models (FMs) from big AI players – AI21 Labs, Anthropic (Claude), and Stability AI, to name just a few (minus OpenAI, of course).

It’s essentially AWS saying, "Hey, why pick one model when you can have all the models?" It’s a very AWS thing to do.

I think this is a smart move by Amazon. At the very least, it lets developers evaluate multiple models via one API. And, if they have some need for multiple models, they get that out of the box.

Don’t forget the Llamas

Earlier this year, Meta open-sourced Llama 2, it’s rival to OpenAI’s GPT-4. While Llama 2 is not on par with GPT-4, it is perhaps the most powerful open source LLM available, backed by a company who is both capable and motivated to compete well in this space.

There are two versions of Llama 2 coming to Bedrock. Both the 13 billion and 70 billion parameter models will be available. Both will have a max token size of 4k, and support English only. These models are intended to be used as chat bot assistants.

Here are some additional highlights about Llama 2:

• Extensive Training: Llama 2’s training on 2 trillion tokens underlines its proficiency and adaptability in understanding and generating human-like responses.
• Safety Assurance: A commendable 1000+ hours spent on red-teaming and annotation fortify its performance while safeguarding interactions.
• Scalability: With models ranging from 7B to 70B parameters, Llama 2 caters to a plethora of applications, scalable to diverse use-cases and organizational needs.

A blog post by Amazon does indicate that fine-tuning will be available for Llama 2.

Amazon Titan

Titan is Amazon’s foundational model, used for text generation, summarization, semantic search, and retrieval augmented generation (RAG). Titan is currently is the most feature rich model available on the platform, as its the only model that supports embeddings and fine tuning at the moment.

Titan comes in three variations:

• Titan Text Express
  • Purpose: Serves as a comprehensive large language model.
  • Balance: Offers a balance between cost and performance.
  • Languages: Supports numerous languages.
  • Applications: Catering to a variety of applications.
• Titan Text Lite
  • Purpose: Provides an affordable, compact model variant.
  • Cost-Effective: A more budget-friendly option compared to other versions.
  • Languages: Exclusively supports English.
  • Use Case: Tailored for basic tasks and fine-tuning efforts.
• Titan Embeddings
  • Purpose: Translates text into numerical representations.
  • Use Case: Especially beneficial for text retrieval, semantic similarity, and clustering.

These models, each with their specific focuses and use cases, collectively make Amazon Titan a versatile tool, opening up an array of possibilities for various text-related applications across different domains.

I’d imagine that Amazon will continue to invest a lot in Titan, trying to bring it up to par with other models. This is one of those “frenemie” things, where they are competing with the partner models available on Bedrock.

Let’s see how well they do with this!

Embeddings Support

Amazon Bedrock currently has limited support for embeddings. The only model is suported currently is its Titan Embeddings G1 model. For those of you scratching your heads, embeddings are a way to represent textual information in the form of vectors. They’re used to by applications to have a LLM respond to queries with information that’s not in its training set. For example, if you have a doc that you’d like to do Q&A with, you need to use embeddings with your model.

Hopefully Amazon expands embeddings support to other models in the short-term. Most applications built with LLMs will likely need them in some capacity.

Fine Tuning Support

Bedrock offers the ability to fine-tune models. This can help improve model performance. Fine tuning is more about customizing how the LLM responds to queries, vs adding data to the model. Again, however, fine tuning is limited to Amazon’s Titan models for now. Let’s see if the other models open this feature up.

It’s important to note that fine tuning does result in higher costs. From the online pricing examples, this can be much more expensive. Customizing your model means you'll need to get provisioned throughput for optimal performance. Check out the finer details on Provisioned throughput.

Provisioned Throughput Pricing

Bedrock Agents

Developers who build applications with LLMs often need to augment their apps with additional automation. For example, if you want up-to-date web information, you may want an agent that performs a search on Google and then adds the response as context to your LLM of choice.

Open-source tools like LangChain have support for these types of models. And, so does Bedrock. The Bedrock docs have this to say about agents:

“Agents for Amazon Bedrock enables developers to configure an agent to complete actions based on organization data and user input. Agents orchestrate interactions between foundation models, data sources, software applications, and user conversations, and automatically call APIs to take actions. Developers can easily integrate the agents and accelerate delivery of generative AI applications saving weeks of development effort.”

This is a really handy feature that I’m sure will make it easier for developers to build apps around the models on Bedrock.

What this Means

Clearly Amazon sees LLMs as being a very important product category. The success of ChatGPT made that painfully obvious. Like other tech companies, Amazon was caught off guard and has had to make up ground quickly.

For AWS customers, Bedrock is a step in the right direction. Making many models available in one place, with an established billing system, will make it easier for developers to evaluate and integrate LLMs.

While it’s great they launched Bedrock, it is an early version of the service and they’ll need to keep moving fast to catch up to OpenAI.

In Conclusion

Amazon Bedrock's entry might be the fresh gust of wind the AWS’ generative AI sail needed. With its offerings and integrations, AWS is not just playing the game; it's looking to change it. For software developers like us, it's a call to action – to dive in, explore, and perhaps, shape the future of generative AI.

Feeling intrigued? Don't stop here. Dive deep into our courses to get hands-on experience with generative AI tools and technologies. Remember, at Discover GenAI, we've got your back every step of the way!