(Caption: A sonnet situated on a bedrock connecting knowledge graphs. Image source: generated by DALL-E.)
On 2024-03-04, Anthropic released the Anthropic Claude 3 model family. On the same day, Amazon also announced Amazon Bedrock adds Claude 3 Anthropic AI models.
This article will provide a simple guide to getting started with Claude 3 on Amazon Bedrock and exploring its capabilities. After playing around with it, you might feel more inclined to integrate these LLM models into your workflow to some extent.
Quick glossary:
- Anthropic: A company focused on AI safety and research.
- Anthropic Claude 3: A family of models including several different sizes suitable for various application scenarios.
- Amazon Bedrock: An AI one-stop shopping portal provided by Amazon that offers the simplest way to start using, building, and scaling generative AI applications, incorporating responsible AI.
Anthropic Claude 3
The Claude 3 model family includes three tiers:
- Claude 3 Opus ($15 | $75)
- Claude 3 Sonnet ($3 | $15)
- Claude 3 Haiku ($0.25 | $1.25)
Cost: [Input $/million tokens | Output $/million tokens]
(Image source: Introducing the next generation of Claude \ Anthropic)
Like products across various industries worldwide, Claude 3 has its own strengths and preferred application scenarios. In addition to the comparison charts released by the official, Claude 3 has been enhanced in analysis and prediction, generating detailed content, and generating code. It has also specifically focused on non-English languages, with the official highlighting Spanish, Japanese, and French. This likely reflects market demand direction, suggesting that markets using these languages may have higher market potential (e.g., Spanish is the second most used language in the business world after English, and Japan has a relatively concentrated market share in the Asian B2B market). When planning your AI products, consider this.
(Image source: Introducing the next generation of Claude \ Anthropic)
There have also been improvements in visual capabilities. Sometimes I wonder if all models currently have about a 50% understanding of general charts, perhaps because the world is full of incorrectly drawn charts that have been fed into training. According to the official comparison chart, more rigorously drawn scientific charts seem to have higher processing capabilities.
(Image source: Introducing the next generation of Claude \ Anthropic)
Currently, the Claude 3 model family offers a 200K context window for use, which appears to be a soft limit, with the option to request expansion to 1M.
(Image source: Introducing the next generation of Claude \ Anthropic)
Amazon Bedrock
Next, let’s dive into Amazon Bedrock to experience Claude 3 Sonnet.
Amazon Bedrock can save a significant amount of time in initial research, offering a one-stop-shop experience (or a one-stop way to spend your money XDD), suitable for those who want to quickly evaluate without managing deployment and focus solely on applications.
From the top right corner of the AWS official website, there’s a button that leads directly to the AWS Management Console interface. After entering your account, password, and two-factor authentication, use the search function at the top and type in “bedrock” to switch to the Amazon Bedrock operation screen.
At the time of publishing this article, the AWS regions that support Amazon Bedrock include the following five, remember to switch to one of them before continuing:
- Europe (Frankfurt)
- US West (Oregon)
- Asia Pacific (Tokyo)
- Asia Pacific (Singapore)
- US East (N. Virginia)
The first time you use Amazon Bedrock, the AWS Management Console will prompt you for permission settings. If you do not have access to Amazon Bedrock, you can ask your AWS Account administrator to assign Amazon Bedrock-related permissions to your IAM account according to the IAM documentation on the screen.
Model Access
Once you have the permissions, the next step is to apply for the usage license for each model you intend to use.
Amazon Bedrock makes this process very convenient and simple. Everything is centralized in one portal, allowing you to submit your application all at once. The review process is fast, usually taking just a few minutes, and you can start using the service immediately.
(Although Amazon Bedrock makes entering the world of AI easier by integrating a single window that leads directly to the Marketplace of various model providers, it’s advisable to take some time to read and digest the terms for product management and business operations. Make sure to check the restrictions of each provider and model to ensure they won’t impact the future operation of your application when it goes live.)
To apply for a model, simply click on “Manage model access” in the top right corner, and then select the models you wish to use. Some model providers may require you to provide use cases or scenarios for review, so input your information accordingly. The models available to different accounts can vary, and this can change over time. Be sure to consult the documentation or contact the model providers for the latest information.
After a few minutes, when you check back, you should have received authorization, indicated by the “Access granted” message.
Playground
Next, everyone can use this Chat Playground environment as a chat setting to pose questions from various domains for experimentation.
For example, I wanted to experiment with Claude 3’s understanding of AWS architecture diagrams, so I took out the most basic diagram from my previous Laravel on Amazon ECS Workshop to see if Claude 3 could comprehend it.
Below is the image I uploaded to Claude 3, along with the Prompt:
Hi there, you are a seasoned AWS solution architect familiar with various AWS specialties and Amazon ECS container services, and proficient in both Traditional Chinese and English. Except for code discussions, please respond in both Traditional Chinese and English. Please explain the meaning of this architecture diagram step by step to me.
(Prompt in Chinese, but you will see the response in English below)
After clicking the “Run” button, below is the result returned to me by Claude 3:
好的,我會用繁體中文和英文為您解說這張架構圖的意義。
This architecture diagram depicts a setup for running Laravel on AWS ECS (Elastic Container Service). It showcases a highly available and scalable infrastructure spanning across two Availability Zones within an AWS Region.
這張架構圖展示了在 AWS ECS (彈性容器服務) 上運行 Laravel 的設置。它描繪了一個高可用和可擴展的基礎設施,橫跨 AWS 區域 (Region) 內的兩個可用區 (Availability Zone)。
In the public subnet of each Availability Zone, there is an Application Load Balancer acting as the entry point for incoming traffic from the internet.
在每個可用區的公有子網路中,都有一個應用程式負載平衡器 (Application Load Balancer),作為來自網際網路的入站流量的進入點。
The load balancer distributes traffic across multiple instances of the Fargate task, which run in private subnets across both Availability Zones for high availability.
負載平衡器會將流量分配到兩個可用區的私有子網路中運行的多個 Fargate 任務實例,以實現高可用性。
Fargate is the serverless compute engine for ECS, allowing you to run containers without managing servers or clusters.
Fargate 是 ECS 的無服務器計算引擎,允許您運行容器而無需管理伺服器或叢集。
The private subnets are connected to the internet through a NAT Gateway in the public subnet of Availability Zone 1.
私有子網路通過可用區 1 的公有子網路中的 NAT 網關連接到互聯網。
An ECR (Elastic Container Registry) is used to store and deploy the Docker container images for the Laravel application.
ECR (彈性容器登錄) 用於存儲和部署 Laravel 應用程式的 Docker 容器映像。
The isolated subnets are likely used for data storage or other stateful resources that don’t need internet access.
隔離子網路可能用於不需要互聯網訪問的數據存儲或其他有狀態資源。
Overall, this architecture provides a scalable, highly available, and secure way to run Laravel applications on AWS ECS using Fargate, load balancing, and multiple Availability Zones.
總的來說,這種架構提供了一種可擴展、高可用和安全的方式在 AWS ECS 上使用 Fargate、負載平衡和多個可用區來運行 Laravel 應用程式。
In the Model metrics area below, you can see some reference numbers to keep track of the token usage for inputs and outputs, along with the corresponding costs.
Bottom Line
That’s a wrap for today’s simple experiment. You should now have a grasp on how to interact with Claude 3 in your own Chat Playground. Feel free to try out any questions you have or situations you want to resolve. Share and exchange your experimental results with each other.
If there’s a chance in the future, we’ll experiment with more complex scenarios. For instance, after understanding an architecture diagram, we might ask Claude 3 to directly generate AWS CDK code for us, check deployment, and so on.
Everyone is welcome to leave comments or mention me on Twitter to share your experimental results :)
Notes:
- AWS CDK is a set of Infrastructure as Code (IaC) tools that allows you to deploy simple or complex cloud infrastructure through coding. This lets your cloud infrastructure be version-controlled, tracked, and compliant just like your application code.
- The container Docker image used in the workshop is dwchiang/nginx-php-fpm | Docker Hub, maintained by me and a good friend. If you’re also a PHP enthusiast, feel free to try it out. We have specially packaged an arm64 version that runs on AWS Graviton.