generateClient observeQuery with filters does not seem to work if schema has composite identifier

[nootn]

Before opening, please confirm:

• I have searched for duplicate or closed issues and discussions.
• I have read the guide for submitting bug reports.
• I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

JavaScript Framework

React

Amplify APIs

GraphQL API

Amplify Version

v6

Amplify Categories

api

Backend

None

Environment information

  System:
    OS: Windows 11 10.0.22631
    CPU: (24) x64 AMD Ryzen 9 7845HX with Radeon Graphics
    Memory: 2.67 GB / 31.19 GB
  Binaries:
    Node: 20.12.1 - C:\Program Files\nodejs\node.EXE
    npm: 10.5.0 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Edge: Chromium (130.0.2849.46)
    Internet Explorer: 11.0.22621.3527
  npmPackages:
    %name%:  0.1.0
    @aws-amplify/backend: ^1.0.0 => 1.0.0
    @aws-amplify/backend-cli: ^1.0.1 => 1.0.1
    @aws-amplify/geo: ^3.0.35 => 3.0.35
    @aws-amplify/geo/location-service:  undefined ()
    @aws-amplify/pubsub: ^6.1.11 => 6.1.11
    @aws-amplify/ui-react: ^6.1.9 => 6.2.1
    @aws-amplify/ui-react-geo: ^2.0.16 => 2.0.16
    @aws-amplify/ui-react-internal:  undefined ()
    @aws-amplify/ui-react-server:  undefined ()
    @aws-amplify/ui-react-storage: ^3.2.1 => 3.2.1
    @aws-lambda-powertools/logger: ^2.1.1 => 2.1.1
    @aws-sdk/client-bedrock-runtime: ^3.650.0 => 3.650.0
    @aws-sdk/client-cognito-identity-provider: ^3.592.0 => 3.592.0
    @aws-sdk/client-iot: ^3.608.0 => 3.608.0
    @aws-sdk/client-ses: ^3.658.1 => 3.658.1
    @aws-sdk/client-sns: ^3.658.1 => 3.658.1
    @bugster/bugster-js: ^1.1.2 => 1.1.2
    @chromatic-com/storybook: ^1.5.0 => 1.5.0
    @mapbox/mapbox-gl-draw: ^1.4.3 => 1.4.3 (1.3.0)
    @react-three/drei: ^9.110.0 => 9.110.0
    @react-three/fiber: ^8.17.5 => 8.17.5
    @storybook/addon-essentials: ^8.1.10 => 8.1.10
    @storybook/addon-interactions: ^8.1.10 => 8.1.10
    @storybook/addon-links: ^8.1.10 => 8.1.10
    @storybook/addon-onboarding: ^8.1.10 => 8.1.10
    @storybook/addon-themes: ^8.1.10 => 8.1.10
    @storybook/blocks: ^8.1.10 => 8.1.10
    @storybook/react: ^8.1.10 => 8.1.10
    @storybook/react-vite: ^8.1.10 => 8.1.10
    @storybook/test: ^8.1.10 => 8.1.10
    @types/aws-lambda: ^8.10.138 => 8.10.138
    @types/dompurify: ^3.0.5 => 3.0.5
    @types/jest: ^29.5.12 => 29.5.12
    @types/mapbox__mapbox-gl-draw: ^1.4.7 => 1.4.7
    @types/node: ^20.14.9 => 20.14.9 (18.19.39)
    @types/react: ^18.2.66 => 18.3.1
    @types/react-dom: ^18.2.22 => 18.3.0
    @types/xml2js: ^0.4.14 => 0.4.14
    @typescript-eslint/eslint-plugin: ^7.2.0 => 7.8.0
    @typescript-eslint/parser: ^7.2.0 => 7.8.0
    @vitejs/plugin-react: ^4.2.1 => 4.2.1
    aws-amplify: ^6.5.4 => 6.5.4
    aws-amplify/adapter-core:  undefined ()
    aws-amplify/analytics:  undefined ()
    aws-amplify/analytics/kinesis:  undefined ()
    aws-amplify/analytics/kinesis-firehose:  undefined ()
    aws-amplify/analytics/personalize:  undefined ()
    aws-amplify/analytics/pinpoint:  undefined ()
    aws-amplify/api:  undefined ()
    aws-amplify/api/server:  undefined ()
    aws-amplify/auth:  undefined ()
    aws-amplify/auth/cognito:  undefined ()
    aws-amplify/auth/cognito/server:  undefined ()
    aws-amplify/auth/enable-oauth-listener:  undefined ()
    aws-amplify/auth/server:  undefined ()
    aws-amplify/data:  undefined ()
    aws-amplify/data/server:  undefined ()
    aws-amplify/datastore:  undefined ()
    aws-amplify/in-app-messaging:  undefined ()
    aws-amplify/in-app-messaging/pinpoint:  undefined ()
    aws-amplify/push-notifications:  undefined ()
    aws-amplify/push-notifications/pinpoint:  undefined ()
    aws-amplify/storage:  undefined ()
    aws-amplify/storage/s3:  undefined ()
    aws-amplify/storage/s3/server:  undefined ()
    aws-amplify/storage/server:  undefined ()
    aws-amplify/utils:  undefined ()
    aws-cdk: ^2.138.0 => 2.140.0
    aws-cdk-lib: ^2.138.0 => 2.140.0
    aws-sdk: ^2.1632.0 => 2.1632.0
    axios: ^1.7.7 => 1.7.7
    constructs: ^10.3.0 => 10.3.0
    date-fns: ^3.6.0 => 3.6.0
    dompurify: ^3.1.6 => 3.1.6
    esbuild: ^0.20.2 => 0.20.2
    eslint: ^8.57.0 => 8.57.0
    eslint-config-prettier: ^9.1.0 => 9.1.0
    eslint-plugin-prettier: ^5.1.3 => 5.1.3
    eslint-plugin-react-hooks: ^4.6.0 => 4.6.2
    eslint-plugin-react-refresh: ^0.4.6 => 0.4.6
    eslint-plugin-storybook: ^0.8.0 => 0.8.0
    geolib: ^3.3.4 => 3.3.4
    immer: ^10.1.1 => 10.1.1 (9.0.21, 9.0.6)
    jest: ^29.7.0 => 29.7.0
    prettier: ^3.2.5 => 3.2.5 (2.3.2, 2.8.8, 1.19.1)
    react: ^18.2.0 => 18.3.1
    react-dom: ^18.2.0 => 18.3.1
    react-icons: ^5.2.1 => 5.2.1
    react-router-dom: ^6.23.1 => 6.23.1
    react-toastify: ^10.0.5 => 10.0.5
    recharts: ^2.12.7 => 2.12.7
    storybook: ^8.1.10 => 8.1.10
    survey-react-ui: ^1.11.13 => 1.11.13
    three: ^0.167.1 => 0.167.1
    ts-jest: ^29.1.5 => 29.1.5
    tsx: ^4.7.2 => 4.9.1
    typescript: ^5.4.5 => 5.4.5 (4.4.4, 4.9.5)
    vite: ^5.2.0 => 5.2.11
    xml2js: ^0.6.2 => 0.6.2
  npmGlobalPackages:
    @aws-amplify/cli: 12.12.1
    @budibase/cli: 2.30.3
    appium-windows-driver: 2.12.21
    appium: 2.5.4
    azurite: 3.29.0
    corepack: 0.25.2
    generator-office: 1.9.8
    next: 14.2.1
    npm-check-updates: 16.14.18
    npm: 10.5.0
    yo: 5.0.0

Describe the bug

When using observeQuery, E.g: const client = generateClient<Schema>() then client.models.DeviceMeasures.observeQuery - if you supply a filter that uses those columns, you get a runtime error The variables input contains a field that is not defined for input object type 'ModelSubscriptionDeviceMeasuresFilterInput'. It seems there is an issue with the generated graphQL for the schema.

When using filters for any tables that just use the default in-built ID, this does not happen, the filters work fine.

Expected behavior

Data should be filtered correctly and no error shown.

Reproduction steps

In your resource.ts, set up a schema that has an identifier with a composite key (E.g. two string columns):

    DeviceMeasures: a
      .model({
        tenantId: a.string().required(),
        name: a.string().required(),
        measures: a.customType({
          ts: a.integer(),
          pwr: a.integer(),
          tankPct: a.float(),
          tankRem: a.float(),
          bmsPct: a.float(),
          bmsRem: a.float(),
          flow: a.float(),
          outPsi: a.float(),
          outKpa: a.float(),
        }),
        recentMeasures: a.json(),
      })
      .identifier(['tenantId', 'name'])
      .authorization((allow) => [allow.groupDefinedIn('tenantId')]),

We can see the "correct" looking graphQL is generated:

export const onCreateDeviceMeasures = /* GraphQL */ `
  subscription OnCreateDeviceMeasures(
    $filter: ModelSubscriptionDeviceMeasuresFilterInput
  ) {
    onCreateDeviceMeasures(filter: $filter) {
      createdAt
      measures {
        bmsPct
        bmsRem
        flow
        outKpa
        outPsi
        pwr
        tankPct
        tankRem
        ts
        __typename
      }
      name
      recentMeasures
      tenantId
      updatedAt
      __typename
    }
  }
`;

export const onDeleteDeviceMeasures = /* GraphQL */ `
  subscription OnDeleteDeviceMeasures(
    $filter: ModelSubscriptionDeviceMeasuresFilterInput
  ) {
    onDeleteDeviceMeasures(filter: $filter) {
      createdAt
      measures {
        bmsPct
        bmsRem
        flow
        outKpa
        outPsi
        pwr
        tankPct
        tankRem
        ts
        __typename
      }
      name
      recentMeasures
      tenantId
      updatedAt
      __typename
    }
  }
`;

export const onUpdateDeviceMeasures = /* GraphQL */ `
  subscription OnUpdateDeviceMeasures(
    $filter: ModelSubscriptionDeviceMeasuresFilterInput
  ) {
    onUpdateDeviceMeasures(filter: $filter) {
      createdAt
      measures {
        bmsPct
        bmsRem
        flow
        outKpa
        outPsi
        pwr
        tankPct
        tankRem
        ts
        __typename
      }
      name
      recentMeasures
      tenantId
      updatedAt
      __typename
    }
  }
`;

Code Snippet

When you create an observeQuery that tries to use the columns in the identifier:

  useEffect(() => {
    const subscription = client.models.DeviceMeasures.observeQuery({
      filter: {
        and: [
          { tenantId: { eq: device.tenantId } },
          { name: { eq: device.name } }
        ]
      }
    }).subscribe({
      next: ({ items }) => {
        setDeviceMeasures(items[0] || null);
        setLoading(false);
      },
      error: (err) => {
        notifyError('Error in DeviceMeasures subscription', err);
        setError(
          'Failed to fetch or update device data. Please try again later.'
        );
        setLoading(false);
      },
    });

    return () => subscription.unsubscribe();
  }, [device.tenantId, device.name]);

It will give the error The variables input contains a field that is not defined for input object type 'ModelSubscriptionDeviceMeasuresFilterInput'

Log output

//This is the output from my own logging code as above:

NOTIFY ERROR Error in DeviceMeasures subscription - {"type":"onUpdate","error":{"errors":[{"message":"Connection failed: {\"errors\":[{\"message\":\"The variables input contains a field that is not defined for input object type 'ModelSubscriptionDeviceMeasuresFilterInput' \"}]}"}]}}

aws-exports.js

No response

Manual configuration

No response

Additional configuration

No response

Mobile Device

No response

Mobile Operating System

No response

Mobile Browser

No response

Mobile Browser Version

No response

Additional information and screenshots

As a workaround, I can allow it to retieve all items by removing the filter, then filter in the UI, but this will not be efficient at scale as I may have many records coming back that I just ignore:

  useEffect(() => {
    const subscription = client.models.DeviceMeasures.observeQuery({
      /* we will remove the filter as there is a bug preventing it working - filter after
      filter: {
        and: [
          { tenantId: { eq: device.tenantId } },
          { name: { eq: device.name } }
        ]
      }
      */
    }).subscribe({
      next: ({ items }) => {
        //once bug is fixed we can put this back: setDeviceMeasures(items[0] || null);
        //for now, filter here
        setDeviceMeasures(items.filter(item => item.tenantId === device.tenantId && item.name === device.name)[0] || null);
        setLoading(false);
      },
      error: (err) => {
        notifyError('Error in DeviceMeasures subscription', err);
        setError(
          'Failed to fetch or update device data. Please try again later.'
        );
        setLoading(false);
      },
    });

    return () => subscription.unsubscribe();
  }, [device.tenantId, device.name]);

[chrisbonifacio]

Hi @nootn 👋 thanks for raising this issue. apologies for the delay.
I was able to consistently reproduce the error when attempting to subscribe to changes on the DeviceMeasures model with observeQuery. I've labeled this issue as a bug for the team to investigate further. we will report back with any findings.

[heeen]

FYI you don't need to have an identifier clause, just trying to filter on a field that is also used in a clause like allow.groupDefinedIn("organizationGroupName").to(["read"]), is enough to trigger the error.

[atierian]

Thanks for that info @heeen.

We've identified the root cause here -- the generated ModelSubscription<Model>FilterInput GraphQL input type omits the field referenced via groupsDefinedIn(). So when that field is included in a subscription filter (observeQuery, onCreate, onUpdate, onDelete), the GraphQL request fails.

We're exploring options to address this and will update here.

As a temporary workaround, you can include group field in a query filter and omit it for subscriptions (and filter client side if needed). Note that omitting the field in the subscription filter doesn't influence auth in any way; users will only retrieve subscription events according to the schema defined authorization rules.