[FEATURE-REQUEST] Select depth operator *>[number]

[ollyde]

Been suggested before but I'm adding another spin on it. Lot of people requesting and a lot of up-votes for, I see answers as to why it's not being implemented but they are not valid responses; there's simply no excuse for not having this key functionality;

It is possible to do this with an pre-query and by hacking together the fields; but it's not consistent or reliable and you have to make a request queue & two requests which isn't a great dev experience. Should be handled by the spec server-side.

I'm managing a few projects with many different languages and remote crash logs. (TypeScript, Rust, Dart)
Our number one crash right now is missing fields in JSON -> class mapping, across projects. That's our number 1 crash and it's caused because of GraphQL.

We have to build most of our models like this, being careful not to miss fields when adding/updating

final roomResultFields = r'''
  company {
    {companyFields}
  }
  room {
    {roomFields}
  }
  totalMembers
  isMemberOfCompany
  isInRoom
  roomMembers {
    memberDetails {
      {roomMemberFields}
    }
    userLight {
      {userLightFields}
    }
  }
'''
    .replaceFirst("{companyFields}", companyFields)
    .replaceFirst("{roomFields}", roomFields)
    .replaceFirst("{roomMemberFields}", roomMemberFields)
    .replaceFirst("{userLightFields}", userLightFields);

^^ Every model has this. It sucks. Across the GraphQA projects I've seen and manage it's always the same story, this sucky query module duplication.

We really like GraphQL but our major issue is that the select all field is missing. Instead I propose a select depth field.

The community wants it.

Instead of select *, why not select depth. For example *>1 could get level 1 fields, > could "get all", etc you get the point.

[benjie]

Hi Olly. For all those replacements, I wonder why you aren't using fragments?

But to address the topic at hand, your proposal lacks detail. What does the * or *>1 actually mean; i.e. what does it do on a polymorphic type? What does it do when a field has arguments? What does it do when a field is deprecated? What does it do when one of the fields is polymorphic? Do child list fields get handled differently? How does it avoid infinite recursion? (I'm sure there are many other questions.)

[ollyde]

@benjie we consider fragments even worse than our current method. It requires updating the fragment on the client.

Like SQL, SELECT *

So
user {*} or user *

The ">" is the depth.

user *>1 // Go n fields deep.

It's fairly simple request and many are leaving GraphQL because it doesn't exist, why bother with a limited framework when Rest API decorators can do more on a NodeJS Lib. 🙈🤪

"What does it do on a polymorphic type?"
Usually handed by serialization package. But we've found no use for it; some languages are still restricted in union types like Dart/Java etc.

"What does it do when a field has arguments?"
Ignore it, 99% of the cases I see they just serializing data into a model from a standard query, arguments or not.

"How does it avoid infinite recursion?"
With the depth selector, but I'm sure we can be creative here.. I'm not the spec writer just trying to get something essential in it, otherwise back to rest for new projects.

I'm not a spec writer, but I'm pretty sure most of these questions are straight forwards.

[benjie]

If you are serious about adding this functionality to GraphQL, you should write up an RFC including concrete specification edits, and then present it to the GraphQL Working Group by adding yourself to an upcoming agenda, for example:

https://github.com/graphql/graphql-wg/blob/main/agendas/2022/2022-09-01.md

---

"What does it do on a polymorphic type?"

Usually handed by serialization package. But we've found no use for it; some languages are still restricted in union types like Dart/Java etc.

This isn't clear to me.

"What does it do when a field has arguments?"

Ignore it, 99% of the cases I see they just serializing data into a model from a standard query, arguments or not.

I advise that you only ignore fields that have at least one required argument. Otherwise simply adding a nullable argument to a field (which is currently a non-breaking change) would become a breaking change because it could affect pre-existing queries.

I'm not a spec writer, but I'm pretty sure most of these questions are straight forwards.

There's a lot of reasons that GraphQL doesn't have this functionality. They've been covered in previous issue threads, but one of the main reasons is that GraphQL's explicit selections means that you can add fields to a GraphQL schema without affecting any pre-existing operations. Enabling versionless APIs is very much one of GraphQL's goals. In your proposal, if you had a User type that had { id: ID!, name: String!, avatarUrl: String, emails: [Email] } and you used *>1 to select it; then a few months later you added a User.connections: [User] field, now your existing clients are pulling down far more information than they need, slowing down user requests and putting significantly more stress on the server for zero gain. A better solution would be to use fragments for shared query logic, or to use a precompiler to convert your * into the equivalent fields before persisting the operation to version control.

we consider fragments even worse than our current method. It requires updating the fragment on the client.

Please can you expand on how doing string substitution (presumably also on the client?) is better than fragments for your team?

[ollyde]

@benjie I've gone though the requests for this, and I can see that it's the number 1 feature people are complaining about.

The serialization bugs are the number 1 crash issue for us across teams and projects looking at our crash analytics.

Re-guarding pulling down unnecessary information, this should be on the developers and the communication between back-end and front-end. Perhaps there can be some kind of 'exclude' can be included here.

Most of the time though, "connections" would be described as "user_connections" or "userConnections" GraphQL "endpoint" from how I see people implementing these APIs in real life.

[benjie]

The serialization bugs are the number 1 crash issue for us across teams and projects looking at our crash analytics.

You should use some GraphQL tooling to give you type safety; this is one of the major features that GraphQL offers, and you should leverage it. Your projects shouldn't even compile unless the fields they reference on GraphQL results exist in the queries you are issuing.

Re-guarding pulling down unnecessary information, this should be on the developers and the communication between back-end and front-end. Perhaps there can be some kind of 'exclude' can be included here.

I don't think you've quite followed my point. Let me try again:

You write a query in January for { user { *>1 } }. In January it would return { user: { id: ..., name: ..., avatarUrl: ..., emails: [...] } }. You bundle this up as part of your mobile client and release it to the Android app store.

In February, you add the User.connections field to the schema. Your Android app still issues the exact same GraphQL query: { user { *>1 } }, but the meaning has changed because this new field was added to the schema. The response is everything it had before plus the connections: [...] data that the client neither understands nor uses in any way. You've reduced the performance of your client and put strain on the backend and the data isn't even used because it wasn't known about when the query was written.

Most of the time though, "connections" would be described as "user_connections" or "userConnections" GraphQL "endpoint" from how I see people implementing these APIs in real life.

At first I thought you were talking about naming for the "connections" field; but on reflection I think you might be indicating that rather than adding User.connections people would instead add Query.userConnections. If so, this very much gets away from the "graph" in "GraphQL" - the reason for the "graph" in the name is that your data is interconnected and GraphQL gives you a way to query and traverse that interconnectedness. If you're building a GraphQL schema like this without connections between types, then you probably would be much better suited to another technology.

[ollyde]

@benjie I understood your 'connections' issue as you described before, and gave how I see real life GQL "endpoints" being written, they don't extend the user object, they are usually a differently named user resolver property, such as userConnections.

Example of a multi RTC Resolver endpoints for example

You should use some GraphQL tooling to give you type safety

I'm not aware of GraphQL tooling that allows us to serialize typesafe models automatically? Can you point me to one?

[benjie]

I'm not aware of GraphQL tooling that allows us to serialize typesafe models automatically? Can you point me to one?

I personally use https://www.graphql-code-generator.com/ but there are many solutions for different languages/frameworks. In my projects, whatever piece of code needs data from GraphQL (be that a function, component, class, etc) defines the fragment(s) that describes what it needs; GraphQL Codegen generates the typings for those fragments so the code knows exactly the shape of data it will be receiving. The client then combines those fragments into a query and issues that query to the server; the relevant parts of the response are then distributed the relevant functions/components/classes/etc that requested it - all completely type safe.

[ollyde]

@benjie thanks Benjie, I looked into this, but for our use case a select * operator would be the simple and elegant solution here.