Bug: Batch Resolver – List(Dict) Not Resolving

[swarnim-kamal]

Expected Behaviour

The batch resolver should return the resolved values for each item in the input list, regardless of the data structure being processed. This principle should hold true even when the resolver is passing a list(dictionary).

Current Behaviour

When passing a list of dictionaries as the output (with the order matching the input list), the batch resolver fails to resolve the output correctly and instead returns null.

Code snippet

#graph QL:
type Item {
  id: ID!
  name: String!
  singleItem: SingleItem
}

type SingleItem {
  id: ID!
  name: String!
}

input ItemFilter {
  id: ID
  name: String
}

type Query {
  getItem(filter: ItemFilter): [Item] @aws_api_key
}



#Resolver
@router.batch_resolver(type_name="Item", field_name="singleItem", aggregate=True)

@tracer.capture_method

@cdk_gql.appsync_batch(2000) #Still need to use this to provide batch size

def query_getSingleItem(event: List[AppSyncResolverEvent]) ->List[SingleItem]:

    # Get source IDs for DB lookup

    logger.info(f"router 1:{router.current_event.field_name}")

    item_ids = batch_util.get_required_ids(event_list = event, fieldname = "id")

    logger.info(f"router 2:{router.current_event.field_name}")



    # Clean up IDs

    logger.debug(f"item_ids: {item_ids}")

    unique_item_ids = set(item_ids)

    if None in unique_item_ids:

        unique_item_ids.remove(None)

    unique_item_ids = list(unique_item_ids)

    logger.debug(f"unique item_ids: {item_ids}")

    

    # Get data from DB

    single_items_dict = dao_item.get_single_items(unique_item_ids)

        

    logger.debug(f"single_items_dict is : {single_items_dict}")

    # creating output list

    to_return =  [single_items_dict.get(item_id) for item_id in item_ids]

    logger.debug(f"to_return1: {to_return1}")

    return to_return

Possible Solution

Since the resolver successfully handles list(list(dictionary)) (as verified), it should logically be able to resolve list(dictionary) as well. However, it fails to do so, suggesting that a minor adjustment in the resolver logic could be enough to fix the issue.

Steps to Reproduce

1. Define the GraphQL Schema
   Create a GraphQL schema where the Item type contains a single child object (singleItem) that expects a SingleItem dictionary.
   type Item {
   id: ID!
   name: String!
   singleItem: SingleItem
   }

type SingleItem {
id: ID!
name: String!
}

input ItemFilter {
id: ID
name: String
}

type Query {
getItem(filter: ItemFilter): [Item] @aws_api_key
}

2. Implement the Batch Resolver
   Use a batch resolver for the singleItem field in the Item type.
   @router.batch_resolver(type_name="Item", field_name="singleItem", aggregate=True)
   @tracer.capture_method
   @cdk_gql.appsync_batch(2000) # Define batch size
   def query_getSingleItem(event: List[AppSyncResolverEvent]) -> List[Any]:

   Resolver logic here

3. Execute the Query
   Run the following GraphQL query to fetch Item details along with its singleItem child:
   query MyQuery {
   getItem {
   id
   name
   singleItem {
   id
   name
   }
   }
   }

4. Observe the Output
   The query unexpectedly returns null for singleItem, even though the resolver executes without errors.

Powertools for AWS Lambda (Python) version

latest

AWS Lambda function runtime

3.9

Packaging format used

PyPi

Debugging logs

2025-03-11T11:51:48.222Z
{
    "level": "DEBUG",
    "location": "query_getSingleItem:83",
    "message": "unique item_ids: ['abc', 'def']",
    "timestamp": "2025-03-11 11:51:48,221+0000",
    "service": "App-Lambda",
    "sampling_rate": "1",
    "cold_start": false,
    "function_name": "PreProd-US-PiVDCStack-GraphQLtestgraphqllambdaFF03-GPRwAJengh8X",
    "function_memory_size": "128",
    "function_arn": "arn:aws:lambda:ap-south-1:479714207670:function:PreProd-US-PiVDCStack-GraphQLtestgraphqllambdaFF03-GPRwAJengh8X",
    "function_request_id": "4377fb67-a0b0-4e38-8500-4143c330518d",
    "xray_trace_id": "1-67d023d4-05b4b3f029d74d1a2e2c73fc"
}


{"level":"DEBUG","location":"query_getSingleItem:83","message":"unique item_ids: ['abc', 'def']","timestamp":"2025-03-11 11:51:48,221+0000","service":"App-Lambda","sampling_rate":"1","cold_start":false,"function_name":"PreProd-US-PiVDCStack-GraphQLtestgraphqllambdaFF03-GPRwAJengh8X","function_memory_size":"128","function_arn":"arn:aws:lambda:ap-south-1:479714207670:function:PreProd-US-PiVDCStack-GraphQLtestgraphqllambdaFF03-GPRwAJengh8X","function_request_id":"4377fb67-a0b0-4e38-8500-4143c330518d","xray_trace_id":"1-67d023d4-05b4b3f029d74d1a2e2c73fc"}
2025-03-11T11:51:48.222Z
{
    "level": "DEBUG",
    "location": "query_getSingleItem:88",
    "message": "single_items_dict is : {'abc': {'id': '135', 'name': 'SingleItem 1'}, 'def': {'id': '246', 'name': 'SingleItem 2'}}",
    "timestamp": "2025-03-11 11:51:48,222+0000",
    "service": "App-Lambda",
    "sampling_rate": "1",
    "cold_start": false,
    "function_name": "PreProd-US-PiVDCStack-GraphQLtestgraphqllambdaFF03-GPRwAJengh8X",
    "function_memory_size": "128",
    "function_arn": "arn:aws:lambda:ap-south-1:479714207670:function:PreProd-US-PiVDCStack-GraphQLtestgraphqllambdaFF03-GPRwAJengh8X",
    "function_request_id": "4377fb67-a0b0-4e38-8500-4143c330518d",
    "xray_trace_id": "1-67d023d4-05b4b3f029d74d1a2e2c73fc"
}


{"level":"DEBUG","location":"query_getSingleItem:88","message":"single_items_dict is : {'abc': {'id': '135', 'name': 'SingleItem 1'}, 'def': {'id': '246', 'name': 'SingleItem 2'}}","timestamp":"2025-03-11 11:51:48,222+0000","service":"App-Lambda","sampling_rate":"1","cold_start":false,"function_name":"PreProd-US-PiVDCStack-GraphQLtestgraphqllambdaFF03-GPRwAJengh8X","function_memory_size":"128","function_arn":"arn:aws:lambda:ap-south-1:479714207670:function:PreProd-US-PiVDCStack-GraphQLtestgraphqllambdaFF03-GPRwAJengh8X","function_request_id":"4377fb67-a0b0-4e38-8500-4143c330518d","xray_trace_id":"1-67d023d4-05b4b3f029d74d1a2e2c73fc"}
2025-03-11T11:51:48.222Z
{
    "level": "DEBUG",
    "location": "query_getSingleItem:93",
    "message": "to_return1: [{'id': '135', 'name': 'SingleItem 1'}, {'id': '246', 'name': 'SingleItem 2'}]",
    "timestamp": "2025-03-11 11:51:48,222+0000",
    "service": "App-Lambda",
    "sampling_rate": "1",
    "cold_start": false,
    "function_name": "PreProd-US-PiVDCStack-GraphQLtestgraphqllambdaFF03-GPRwAJengh8X",
    "function_memory_size": "128",
    "function_arn": "arn:aws:lambda:ap-south-1:479714207670:function:PreProd-US-PiVDCStack-GraphQLtestgraphqllambdaFF03-GPRwAJengh8X",
    "function_request_id": "4377fb67-a0b0-4e38-8500-4143c330518d",
    "xray_trace_id": "1-67d023d4-05b4b3f029d74d1a2e2c73fc"
}

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

[leandrodamascena]

Hi @swarnim-kamal! Thanks for opening this bug report! I haven't tried what you reported yet, but looking at our code, I think it makes sense yeah! I'll try to reproduce it today/tomorrow and update this issue.

cc @thenamanpatwari

[thenamanpatwari]

Thank you @leandrodamascena for looking into this.
We still haven't been able to upgrade to this awesome batching feature as this bug is stopping our migration. Lot of updates to our dependency tree is blocked till we can upgrade aws_lambda_powertools to 3.x.x version.

Btw @swarnim-kamal and I are co-workers and he is leading this migration effort.

[leandrodamascena]

Thanks for the additional information @thenamanpatwari! I really hope to fix this soon to unblock your migration!

I would like some additional information to understand the scenario here: I tried to simulate it here but couldn't, possibly because I didn't have enough data, but I see that you return a list, which is exactly what we expect here and what AppSync expects as a return. Where are you getting Null, in AppSync resolver? Could you provide me with a payload that you are getting in this Lambda and I can mock some data here?

And sorry for blocking this migration, we will figure it out.

[swarnim-kamal]

Thanks for clarifying your doubt @leandrodamascena ,
This is how we created our ql:

type Item {
id: ID!
name: String!
subItems: [SubItem]
singleItem: SingleItem
}

type SingleItem{
id: ID!
name: String!
}
this how we defined our query :

input ItemFilter {
id: ID
name: String
}
getItem(filter: ItemFilter): [Item] @aws_api_key

#query in App Sync Graphql playgroud:
query MyQuery {
getItem {
id
name
singleItem {
id
name
}
}
}

#Resolver:
So In the input of lambda is a simple list of ids in our case it is
lambda input = ['abc','def']
lambda output = [{'id': '135', 'name': 'SingleItem 1'}, {'id': '246', 'name': 'SingleItem 2'}]
For each input item, the output includes a dictionary that follows the pattern expected by Batch Resolver. This ensures that the GraphQL output is correctly formatted.:
Output(Excpected):
{
"id": "abc",
"name": "Item 1",
"singleItem": {
"id":"135",
"name": "SingleItem 1"
}
},
{
"id": "def",
"name": "Item 2",
"singleItem": {
"id":"246",
"name": "SingleItem 2"
}
}

Output(Actual):
{
"id": "abc",
"name": "Item 1",
"singleItem": null
},
{
"id": "def",
"name": "Item 2",
"singleItem": null
}

So that means that there is an issue with how batch resolution is happening in the output of the lambda.
You can try to reproduce this issue at your end by following this pattern.

[leandrodamascena]

Hi @thenamanpatwari and @swarnim-kamal! Sorry for the delay in responding here, but I was trying to reproduce the error and finally I managed to do it! I don't know if the issue is related to the Powertools batch resolver, but instead how AppSync works in this case. I created two Lambda without Powertools and using the same schema and got the same behavior you had.

Do you two have some free time this week to connect on a call and make some additional debug? I really want to help you guys solve this as soon as possible and unblock this migration. I want to show you the scenario I created in my account.

If we can't find the problem/solve it, I can try to connect internally with some other peer and get more information.

Thanks

[swarnim-kamal]

Hi @leandrodamascena
Thank you for your time and effort in trying to reproduce the issue. We truly appreciate your dedication in clarifying our doubts and helping us troubleshoot this.

I’d also like to mention that the implementation was working correctly in the previous version. Here’s how we initially approached it:

---

to_return = []
for _id in filter_ids:
if _id is None:
to_return.append({})
else:
try:
to_return.append({'data': unique_dict[int(_id)]})
except Exception:
logger.debug("Error encountered while processing ID")
to_return.append({})

---

Regarding our availability, we will be available for a meeting on Wednesday from 10:00 AM to 12:00 PM GMT.

Once again, we really appreciate your support, and we look forward to the discussion. Let us know if this time works for you.

[leandrodamascena]

Hey @swarnim-kamal, can you please send an email to [email protected] and then I send an invite? I don't have your email.

[swarnim-kamal]

Hi @leandrodamascena,
My team has sent an email. Please take a look at it, and we look forward to connecting with you on the meet.

[leandrodamascena]

Hi @thenamanpatwari and @swarnim-kamal! Thank you so much for attending the meeting and sharing your use case. Based on what we discussed yesterday, I can confirm that Batch Resolvers are working as expected and you probably need to add the data field to the response.

I will continue to investigate the use cases where adding the data field is necessary and will come back with an update next week. I will keep this issue open until I decide whether to modify our resolvers or just update the documentation.

Please ping me if you have any additional question.