Disable tracing when not running on AWS Lambda

[Dunedan]

I find the behavior of the Tracer (or aws-xray-sdk to the same extend) when not running on AWS Lambda quite annoying. In this case I don't want to have tracing enabled, because there is no active segment anyway, so unless one explicitly creates a segment before, there'll always be an SegmentNotFoundException being thrown. Currently one has to explicitly disable tracing by either setting an environment variable (POWERTOOLS_TRACE_DISABLED) or by initializing the tracer with Tracer(disabled=True). I'd prefer if tracing would by default only be enabled when actually running on AWS Lambda.

My desired behavior can currently be achieved with something like the following, but having the same behavior out-of-the-box without additional code would be quite nice.

import os
from aws_lambda_powertools import Tracer

tracer = Tracer(disabled=True if not os.getenv("AWS_LAMBDA_FUNCTION_NAME") else False)

Is that something AWS Lambda Powertools could do by default?

[heitorlessa]

Hey @Dunedan,

Great to hear from you again.

We do this currently for when your function is being executed locally by SAM CLI or Chalice - no code change. We could totally do that in v2 as it makes sense to me too, since changing now it'd be breaking change.

How are you running it locally today?

[Dunedan]

Sorry for the late response.

Changing the behavior in a major release sounds good. 👍

We have two cases where the current default behavior bites us.

The first case is related to tests.

Imagine a file handler.py defining a handler:

from aws_lambda_powertools import Tracer

tracer = Tracer()

@tracer.capture_lambda_handler
def handler(event, context):
    print("hello world")

Then we have tests defined, which fail when we run them with pytest, because the tracer doesn't find an open subsegment:

from handler import handler

def test_handler():
    handler(None, None)
    assert True

The other case is when we have utility functions defined, which use the @tracer.capture_method decorator and we want to use these functions outside of AWS Lambda, for example as part of a CLI-script. This fails as well, if we don't disable tracing explicitly before importing the module or configuring the tracer as shown in my original post.

[heitorlessa]

cc @pankajagrawal16 @saragerion @t1agob, what do you reckon on disabling Tracer when not running in Lambda, by default?

For context, Python disables Tracer when code is run within SAM CLI as well as Chalice.

I was concerned about disabling it by default and cause customers tests to fail, but seems more the other way around. In hindsight, the first implementation assumed customers might in the future run their code with X-Ray Daemon running locally[1], and that SAM or any other framework would handle that natively one day.

Over a year has passed and I haven't seen that yet but largely unit tests. Which means this might not be a breaking change after all, and simplify running their code locally whether use SAM/Chalice or not.

[1] Running X-Ray Daemon locally with Lambda

Run as a docker container, listen on default 2000 UDP port that X-Ray SDK will try to attach to

docker run \
        --attach STDOUT \
        -v ~/.aws:/root/.aws/:ro \
        --net=host \
        -e AWS_REGION=eu-west-1 \
        --name xray-daemon \
        -p 2000:2000/udp \
        amazon/aws-xray-daemon -o

From here, you need to create any segment and all code will work, though no traces will be sent to AWS X-Ray service due to -o option, otherwise it very well could.

[heitorlessa]

100% agree on the env change for a positive annotation as I wanted to make this change in v2. It's fine to change the env var given enough notice, but not the constructor as it'll highly likely impact customers code somewhere that we don't know.

For Typescript, C#, best to start the positive annotations both env vars and constructors - in fact I should expedite publishing the list of changes in v2 to pay some tech debts. cc @sliedig @t1agob @pankajagrawal16

@Dunedan can I bother you to raise a feature request so we can work on it for the next release?

[Dunedan]

Here you go: #572
Feel free to modify the feature request, if you feel it needs additional changes. 🙂

[bahrmichael]

I'm aligned with @Dunedan's suggestion. I prefer tests to work out of the box, and not having to install and start up Docker so that my tests can pass. I'm in the camp of "unit test locally, test the rest in the cloud".

given the tenets the assumption should be that the default execution environment is the Lambda execution one. From my point of view disabling for tests should part of the test instrumentation and so be an explicit opt-out for which we already provide the ways pointed out in the OP.

I agree that the tenets point in a different direction. But I think that we should weigh the trade-offs here, and remove as much friction as possible for developers. If AWS promises to do tracing for me, then I'd trust them and not verify it myself. I have not needed tracing in unit tests yet. Therefore I would always have to disable tracing in my unit tests, which can feel repetitive. I feel strongly about not having developers find the right piece of documentation/github threads to get their unit tests working when they add tracing.

[dreamorosi]

Thank you @bahrmichael for addressing my point, I'm convinced that DX > consistency in this case (and many more maybe). I have already applied the change to the Typescript version :)

[heitorlessa]

DONE - This will be available in the next release tomorrow morning in EMEA timezone.