How do I check for exceptions in my HTTP tests?

[ncla]

I am having difficulties writing tests for something that throws an exception.

To simplify, I have a Type something like this:

class ResponsiveFieldType extends Type
{
    public function fields(): array
    {
        return [
            'breakpoints' => [
                'type' => GraphQL::listOf(GraphQL::type(BreakpointType::NAME)),
                'resolve' => function (array $field, array $args, ?array $context, ResolveInfo $info) {
                     throw new \Exception('test');
                },
            ]
        ];
    }
}

Now when I test this field resolver in GraphQL graphical interface or send a request to GraphQL end point which has that field resolving, I get an exception, the stacktrace and so on. Working as intended.

However in my feature tests, this exception gets swallowed and the field resolves to a null value, no exception is thrown.

Through my research I found that exceptions get handled differently depending on what they are and if they are nullable. To my knowledge, my field is not nullable. https://webonyx.github.io/graphql-php/error-handling/

And I see this Exception being handled in here, so it definitely has bubbled up https://github.com/rebing/graphql-laravel/blob/master/src/GraphQL.php#L532-L562

Laravel helper test method $this->withoutExceptionHandling(); in my test does nothing and the field resolves to null regardless.

It is only when I expect an exception through something like $this->expectException(AssetNotFoundException::class); in my test that I start to see the exception in the console.

Cool! But.. it says:

  Failed asserting that exception of type "Spatie\ResponsiveImages\AssetNotFoundException" is thrown.
  
  The following errors occurred during the last request:

It says there was no exception thrown with my expected exception, but then proceeds to say otherwise. I find it a bit weird that it says the error happened in "last request".

For reference, this is roughly how my test looks:

it('fails silently and returns null when asset is not found when using fieldtype', function () {
    // Other boring details..
    $query = '
        {
            entry(id: "1") {
                title
                ... on Entry_Blog_Article {
                    hero {
                        breakpoints {
                            sources {
                                srcSet
                            }
                        }
                    }
                }
            }
        }
    ';

    $this->expectException(AssetNotFoundException::class);

    $response = $this
        ->withoutExceptionHandling()
        ->postJson('/graphql/', ['query' => $query])
        ->getContent();

    $response = json_decode($response, true)['data'];

    expect($response['entry']['hero']['breakpoints'])->toBeNull();
});

What am I doing wrong? Is there perhaps a bug in how this package handles exceptions? Have I misunderstood testing conceptually? How do I correctly assert exception being thrown?

Please advise. Thank you!

[mfn]

'type' => GraphQL::listOf(GraphQL::type(BreakpointType::NAME)),

…

To my knowledge, my field is not nullable

But if this is the field we're talking about, it's even nullable on multiple places:

• first, the list may contain items which are null, so usually you write:
  GraphQL::listOf(Type::nonNull(GraphQL::type(BreakpointType::NAME)))
  But I doubt this is what the docs talk about, because it's the inner type
• second, the actual type allows the returned value to be an array or null because, again, there's no Type::nonNull:
  Type::nonNull(GraphQL::listOf(Type::nonNull(GraphQL::type(BreakpointType::NAME))))
  My philosophy for lists/arrays is usually to always return an array unless you have a reason that "empty array" has to be different from "null".

As for your specific case, it would much easier to debug if you can produce a self-contained example.
You can search through this library test suite, maybe there's similar case, or create a PR we can look at which shows this. However, for DX I often use \Rebing\GraphQL\Tests\TestCase::httpGraphql which might conceal a bit what happens, so be careful.

[ncla]

Thank you for the reply.

I am quite new to GraphQL so pardon my lack of knowledge with this.

My assumption was that GraphQL::type() alone would indicate that it cannot be null. Is does not, got it.

I cracked open a fresh Laravel project and tried to reproduce but to no avail, the exception gets thrown both in the GraphiQL and during the feature test.

I am quite deep when it comes to testing the GraphQL. For context, I am writing tests for statamic-responsive-images package, which is addon for Statamic that itself is a package to Laravel and includes your GraphQL package.

It is entirely possible that Statamic may be doing something for GraphQL, but I have not spotted anything suspicious yet.. https://github.com/statamic/cms/tree/3.4/src/GraphQL Apart from this, which I did try uncommenting before https://github.com/statamic/cms/blob/3.4/src/GraphQL/ServiceProvider.php#L63

To be continued..

[ncla]

Welp this is embarassing. The exception was always there in the response, I was dumping the wrong thing in my test..

I had this:

    $response = $this
        ->withoutExceptionHandling()
        ->postJson('/graphql/', ['query' => $query])
        ->getContent();

    // I should have been doing THIS
    dump($response);

    // NOT THIS, this goes straight to the data
    $response = json_decode($response, true)['data'];
    dump($response);

The confusion for me was that when testing through GraphiQL, the exception information is at the very top under errors key, and then at the very bottom if you scroll the response, I find the data. 🙈

I see the errors now in the GraphQL response now when testing.

[mfn]

😬

That's one of the bugs that happens to the best of us 😅 Glad you solved it 🤞🏼