ProcessingStep serializes pipeline parameters as strings in SDK v2.243.2 #5137
Unanswered
Srinivasaraonarlajarla
asked this question in
Help
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi team,
I'm encountering a regression in the SageMaker Python SDK when using ProcessingStep with pipeline parameters as job_arguments.
SDK Version Comparison :-
This works as expected in version 2.229.0, but breaks in 2.243.2.
Working (v2.229.0)
`
month = ParameterString(name="CurrentMonth", default_value=curr_month)
year = ParameterString(name= "CurrentYear", default_value =curr_year)
preprocess_step = ProcessingStep(
processor=preprocess_processor,
name='DataPreprocessing',
code=os.path.join(PROJECT_DIR, 'manifest_file_creation.py'),
job_arguments=['--year', year, '--month', month],
)
`
Relevant part of the generated pipeline definition:
"ContainerArguments": [ "--year", { "Get": "Parameters.CurrentYear" }, "--month", { "Get": "Parameters.CurrentMonth" } ]
Broken (v2.243.2):-
"ContainerArguments": [ "--year", "{\"Get\": \"Parameters.CurrentYear\"}", "--month", "{\"Get\": \"Parameters.CurrentMonth\"}" ]
Here, the parameter references are serialized as strings, rather than objects, causing the pipeline to treat them as literal strings instead of dynamically resolving them during execution.
Expected Behavior:-
{ "Get": "Parameters.CurrentYear" }
Am I missing something in v2.243.2 ?
Beta Was this translation helpful? Give feedback.
All reactions