Typebox QuerySchema - Querying array mongoDB

[martinfruehmorgen]

My feathers version is 5.0.0-pre.37

My service schema is:

import { resolve } from '@feathersjs/schema'
import { Type, getValidator, querySyntax } from '@feathersjs/typebox'
import { dataValidator, queryValidator } from '../../validators.js'

// Main data model schema
export const contactsSchema = Type.Object(
  {
    _id: Type.String({ objectid: true }),
    contactTags: Type.Array(Type.Number(), { default: [] }),
    deletable: Type.Boolean({ default: true, description: 'is entry deletable' })
  },
  { $id: 'Contacts', additionalProperties: false }
)
export const contactsValidator = getValidator(contactsSchema, dataValidator)
export const contactsResolver = resolve({})

export const contactsExternalResolver = resolve({})

// Schema for creating new entries
export const contactsDataSchema = Type.Pick(contactsSchema, ['contactTags', 'deletable'], {
  $id: 'ContactsData'
})
export const contactsDataValidator = getValidator(contactsDataSchema, dataValidator)
export const contactsDataResolver = resolve({})

// Schema for updating existing entries
export const contactsPatchSchema = Type.Partial(contactsDataSchema, {
  $id: 'ContactsPatch'
})
export const contactsPatchValidator = getValidator(contactsPatchSchema, dataValidator)
export const contactsPatchResolver = resolve({})

// Schema for allowed query properties
export const contactsQueryProperties = Type.Pick(contactsSchema, ['contactTags', 'deletable'])
export const contactsQuerySchema = Type.Intersect(
  [
    querySyntax(contactsQueryProperties),
    // Add additional query properties here
    Type.Object(
      {
        contactTags: Type.Number()
      },
      { additionalProperties: false }
    )
  ],
  { additionalProperties: false }
)
export const contactsQueryValidator = getValidator(contactsQuerySchema, queryValidator)
export const contactsQueryResolver = resolve({})`

The query I want to perform is:
http://localhost:3032/contacts?$or[1][contactTags]=1&$or[0][deletable]=true

query: {
          $or: [{ contactTags: 1 }, { deletable: true }]
        }

What works is
http://localhost:3032/contacts?contactTags=1

query: {
          contactTags: 1
        }

But when using the query with $or, the contactTags: Type.Number() (which is necessary to have the query working without $or) seems not to be passed to the query validator.

The error is:

{
    "name": "BadRequest",
    "message": "validation failed",
    "code": 400,
    "className": "bad-request",
    "data": [
        {
            "instancePath": "/$or/1/contactTags",
            "schemaPath": "#/properties/%24or/items/properties/contactTags/anyOf/0/type",
            "keyword": "type",
            "params": {
                "type": "array"
            },
            "message": "must be array"
        },
        {
            "instancePath": "/$or/1/contactTags",
            "schemaPath": "#/properties/%24or/items/properties/contactTags/anyOf/1/type",
            "keyword": "type",
            "params": {
                "type": "object"
            },
            "message": "must be object"
        },
        {
            "instancePath": "/$or/1/contactTags",
            "schemaPath": "#/properties/%24or/items/properties/contactTags/anyOf",
            "keyword": "anyOf",
            "params": {},
            "message": "must match a schema in anyOf"
        }
    ]
}

Perhaps I need to adjust the querySyntax with expanding $or? But how?

[daffl]

I think the best way to address this would be to customise the query properties. So instead of

export const contactsQueryProperties = Type.Pick(contactsSchema, ['contactTags', 'deletable'])

Use

export const contactsQueryProperties = Type.Object({
  contactTags: Type.Number(),
  deletable: Type.Boolean()
})

This should create the query syntax you are looking for.

[mrfrase3]

In mongo querying with arrays, you rarely directly pass an array of values, as that does an exact match.
e.g. with the following data:
{ _id: "123abc", tags: ["apple", "carrot"] }

Passing the query { tags: ["apple"] } wouldn't match, you need to use the $in operator to get a partial match, which is also broken because of this, the schema validator would want you to pass { tags: { $in: [["apple"], ["banana"]] } }, which is obviously incorrect.

The other issue with exact matching is that order matters, so querying { tags: ["carrot", "apple"] } also wouldn't return. It's usually always better to use the $all query operator to query for every element of an array.

In the past god knows how many years of dev, I don't think I've ever had to do an exact match of an array in a query. Definitely shouldn't be the default...

[gustojs]

I can confirm that this makes $or notice the property. But in a big application, with multiple services, multiple fields each, this solution means maintaining the whole schema twice.
The Type.intersect() at least limits the doubled part to array fields and makes it explicit which parts of the schema we're actually changing for query schema, so it would be nice if there's a way to make it work.

EDIT: Perhaps there's a chance the changes to Type.Intersect() coming with Typebox 0.26 might help?
https://github.com/sinclairzx81/typebox/blob/next/changelog/0.26.0.md#intersect-schema-representation

[daffl]

Querying with $in and $nin in arrays will be possible in 5.0.12 via #3352