Add Exponential/Backoff retry to GetResourceTags

[harishxr]

Summary

This PR enhances the ECS Agent's GetResourceTags function by implementing exponential backoff and retry logic for resource tag retrieval. The implementation addresses potential transient failures when interacting with the ECS API, particularly focusing on handling throttling and rate limiting scenarios.

Implementation details

The GetResourceTags function now includes an exponential backoff mechanism using retry.NewExponentialBackoff with configurable parameters. It implements context-aware retry logic, using context.WithTimeout for overall operation timeout and properly handling context cancellation. The function now differentiates between transient and non-transient errors, with special handling for throttling and rate limit exceeded cases. Structured logging has been implemented, providing detailed error information including resource ARN and attempt counts.

Testing

• Manual testing was performed using Apache Benchmark with concurrent requests to verify the exponential backoff and retry mechanism worked when facing throttling when calling ${ECS_CONTAINER_METADATA_URI_V4}/taskWithTags

New tests cover the changes: yes

The TestGetResourceTags and `TestGetResourceTagsError functions verify successful tag retrieval and error handling respectively. The TestGetResourceTagsWithRetry function covers various retry scenarios including success after throttling, server exceptions, and mixed transient errors. It also tests non-transient error handling and multiple throttling retry scenarios. The tests verify retry behavior, error message formatting, attempt counting, and different error type handling.

Description for the changelog

Add Exponential/Backoff retry to GetResourceTags

Additional Information

Does this PR include breaking model changes? If so, Have you added transformation functions?

Does this PR include the addition of new environment variables in the README?

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.