Revamp Sagemaker doc #1706
Open
Revamp Sagemaker doc #1706
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
- Update the GitHub Actions - Move the `docs/sagemaker` docs into the `source` directory - Add `docs/sagemaker/Makefile` and `docs/sagemaker/scripts` for automatically generating the examples and updating the `_toctree.yml` - Most of those things are ported from https://github.com/huggingface/Google-Cloud-Containers
fgbelidji
reviewed
May 7, 2025
|
|
||
| **How to choose the right container for my use case?** | ||
|
|
||
|  |
There was a problem hiding this comment.
Great image but it's a bit compressed
|
|
||
| ## Training | ||
|
|
||
| Pytorch Training DLC: For training, our DLCs are available for PyTorch via :hugging_face: Transformers. They include support for training on GPUs and AWS AI chips with libraries such as :hugging_face: TRL, Sentence Transformers, or :firecracker: Diffusers. |
| - `dlc-aws-account-id`: The AWS account ID of the account that owns the ECR repository. You can find them in the [here](https://github.com/aws/sagemaker-python-sdk/blob/e0b9d38e1e3b48647a02af23c4be54980e53dc61/src/sagemaker/image_uri_config/huggingface.json#L21) | ||
| - `region`: The AWS region where you want to use it. | ||
|
|
||
| ## Training |
There was a problem hiding this comment.
|
|
||
| ## Inference | ||
|
|
||
| ### Pytorch Inference DLC |
There was a problem hiding this comment.
|
|
||
| ### Text Generation Inference | ||
|
|
||
| There is also the Text Generation Inference (TGI) DLC for high-performance text generation of LLMs on GPU and AWS AI chips. |
There was a problem hiding this comment.
Add TGI container release page https://github.com/aws/deep-learning-containers/releases?q=tgi+AND+gpu&expanded=true
| | | Requirement | Notes | | ||
| |---|-------------| | ||
| | AWS account in a Bedrock Region | Marketplace is regional; switch the console to one of the 14 supported Regions first. | | ||
| | Permissions | For a quick trial, attach AmazonBedrockFullAccess and AmazonSageMakerFullAccess.| |
There was a problem hiding this comment.
AmazonBedrockFullAccess and AmazonSageMakerFullAccess
|
|
||
| When registering your Sagemaker Jumpstart endpoints in Amazon Bedrock, you only pay for the SageMaker compute resources and regular Amazon Bedrock APIs prices are applicable. | ||
|
|
||
| ## 2. Endpoint deployment |
There was a problem hiding this comment.
Some screenshots from the AWS console here would be great
| | AWS account with SageMaker enabled | An AWS account that will contain all your AWS resources. | | ||
| | An IAM role to access SageMaker AI | Learn more about how IAM works with SageMaker AI in this [guide](https://docs.aws.amazon.com/sagemaker/latest/dg/security-iam.html). | | ||
| | SageMaker Studio domain and user profile | We recommend using SageMaker Studio for straightforward deployment and inference. Follow this [guide](https://docs.aws.amazon.com/sagemaker/latest/dg/onboard-quick-start.html). | | ||
| | Service quotas | Most LLMs need GPU instances (e.g. ml.g5). Verify you have quota for ml.g5.24xlarge or [request it](https://docs.aws.amazon.com/sagemaker/latest/dg/canvas-requesting-quota-increases.html). | |
| | SageMaker Studio domain and user profile | We recommend using SageMaker Studio for straightforward deployment and inference. Follow this [guide](https://docs.aws.amazon.com/sagemaker/latest/dg/onboard-quick-start.html). | | ||
| | Service quotas | Most LLMs need GPU instances (e.g. ml.g5). Verify you have quota for ml.g5.24xlarge or [request it](https://docs.aws.amazon.com/sagemaker/latest/dg/canvas-requesting-quota-increases.html). | | ||
|
|
||
| ## 2· Endpoint deployment |
There was a problem hiding this comment.
Some screenshots of jumpstart console also maybe
| - local: index | ||
| title: Hugging Face on AWS | ||
| - local: resources | ||
| title: Other Resources |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Goal:
Here is a V0 of a documentation revamp to change the scope from Sagemaker to AWS.
There are many things to add but I'd like to validate the structure with you before continuing.
Next steps:
Thank you @alvarobartt for your help!