Skip to content

Feat: add custom inference port control to enable accept-bind-to-port for MCE + Inference Pipelines #796

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Feat: add custom inference port control to enable accept-bind-to-port for MCE + Inference Pipelines #796

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions template/v3/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ ARG PINNED_ENV_IN_FILENAME
ARG ARG_BASED_ENV_IN_FILENAME
ARG IMAGE_VERSION
LABEL "org.amazon.sagemaker-distribution.image.version"=$IMAGE_VERSION
LABEL com.amazonaws.sagemaker.capabilities.accept-bind-to-port=true

ARG AMZN_BASE="/opt/amazon/sagemaker"
ARG DB_ROOT_DIR="/opt/db"
Expand Down
17 changes: 15 additions & 2 deletions template/v3/dirs/etc/sagemaker-inference-server/utils/environment.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,14 @@
from enum import Enum


class SageMakerPlatform(str, Enum):
"""Simple enum to define environment variables injected by the SageMaker platform."""

PLATFORM_PORT = "SAGEMAKER_BIND_TO_PORT"


class SageMakerInference(str, Enum):
"""Simple enum to define the mapping between dictionary key and environement variable."""
"""Simple enum to define the mapping between dictionary key and environment variable."""

BASE_DIRECTORY = "SAGEMAKER_INFERENCE_BASE_DIRECTORY"
REQUIREMENTS = "SAGEMAKER_INFERENCE_REQUIREMENTS"
Expand All @@ -28,7 +34,7 @@ def __init__(self):
SageMakerInference.CODE_DIRECTORY: os.getenv(SageMakerInference.CODE_DIRECTORY, None),
SageMakerInference.CODE: os.getenv(SageMakerInference.CODE, "inference.handler"),
SageMakerInference.LOG_LEVEL: os.getenv(SageMakerInference.LOG_LEVEL, 10),
SageMakerInference.PORT: 8080,
SageMakerInference.PORT: self._resolve_port(),
}

def __str__(self):
Expand Down Expand Up @@ -57,3 +63,10 @@ def logging_level(self):
@property
def port(self):
return self._environment_variables.get(SageMakerInference.PORT)

def _resolve_port(self) -> int:
if os.getenv(SageMakerPlatform.PLATFORM_PORT, None):
return int(os.getenv(SageMakerPlatform.PLATFORM_PORT))
if os.getenv(SageMakerInference.PORT, None):
return int(os.getenv(SageMakerInference.PORT))
return 8080
Loading