feat(hub): list collections #1568

quanghuynt14 · 2025-06-29T20:17:48Z

Issue: #271

This PR only includes the call of list collections. Why?

It's one of my early contributions to the project, so I think I should split the PR of @hackpk into many small PRs.
I was really hoping to be able to make a collection of our best models and datasets, so that they can be displayed on our landing page.
I think list collections solve the problem of displaying collections in real time

I’m having a bit of trouble with the type, and I need to search through the project to understand it better. We don't have any public schema API?

Thanks for review it.

packages/hub/src/lib/list-collections.ts

quanghuynt14 · 2025-06-29T20:45:18Z

packages/hub/src/lib/list-collections.ts

+
+	const search = new URLSearchParams([
+		...Object.entries({
+			limit: String(Math.min(totalToFetch, 100)),


by the doc, max number of collections par page is 100.

technically it can be 10000 if all of:

owner is set

expand is set to false explicitly

but not sure we want to bother

packages/hub/src/types/api/api-collection.ts

coyotte508

Thanks for the PR!

I will look into providing a schema for the existing collection APIs

packages/hub/src/lib/list-collections.spec.ts

packages/hub/src/lib/list-collections.ts

quanghuynt14 · 2025-06-30T22:41:48Z

I will look into providing a schema for the existing collection APIs

If we’re using ⁠json-schema, we can use ⁠json-schema-to-typescript to generate the types.

Without the schema, it's very difficult to guess what it is. By reverse-engineering 10,000 collections, I identified the type, but not 100% sure it's correct.

coyotte508 · 2025-07-02T15:00:57Z

Hey @quanghuynt14 , here's an OpenAPI JSON spec for some of the hub endpoints, including collection ones: https://gist.github.com/coyotte508/ec7d12713d94a4fe4988afa3fffc23f1

It includes json schema for the various responses & requests, and you can also load the openapi schema on https://editor-next.swagger.io/ for example

do you think it's helpful for your PRs?

quanghuynt14 · 2025-07-03T10:35:34Z

Yes, it's very complete and helpful. I will check if the type in this PR is correct.

quanghuynt14 · 2025-07-03T16:34:17Z

I checked the schema and found a mismatch in the schema of "/api/collections/{namespace}/{slug}-{id}":

We have:

required prop "position": { "type": "number" }
required prop "shareUrl": { "type": "string" }

in the collection object.

After verifying "slug": "google/gemma-3n-685065323f5984ef315c93f4", I found that it does not include this property.

{
    "slug": "google/gemma-3n-685065323f5984ef315c93f4",
    "title": "Gemma 3n",
    "description": "",
    "gating": false,
    "lastUpdated": "2025-06-26T15:55:44.512Z",
    "owner": {
      ...
    },
    "items": [
      ...
    ],
    "theme": "purple",
    "private": false,
    "upvotes": 146,
    "isUpvotedByUser": false
  },

These 2 props (position and shareUrl) are present only in items of type "collection". For example:

{
    "slug": "kristenq/reasoning-684330e0ce0c4e30fe59456a",
    "title": "Reasoning",
    "description": "Advanced reasoning models. ",
    "gating": false,
    "lastUpdated": "2025-06-06T18:18:26.733Z",
    "owner": {
      ...
    },
    "items": [
      {
        "_id": "684330f27ce524322498baa7",
        "position": 0,
        "type": "collection",
        "id": "67ee7145ec3d31f7c7a75cab",
        "slug": "Tesslate/synthia-s1-reasoning-model-67ee7145ec3d31f7c7a75cab",
        "title": "Synthia-S1 REASONING MODEL",
        "description": "Creative, Scientific, and Coding",
        "lastUpdated": "2025-04-03T11:31:05.262Z",
        "numberItems": 3,
        "owner": {
          ...
        },
        "theme": "blue",
        "shareUrl": "https://huggingface.co/collections/Tesslate/synthia-s1-reasoning-model-67ee7145ec3d31f7c7a75cab", 
        "upvotes": 3,
        "isUpvotedByUser": false
      }
    ],
    "theme": "green",
    "private": false,
    "upvotes": 0,
    "isUpvotedByUser": false
  },

Here is the corrected schema: https://gist.github.com/quanghuynt14/1c55f44978248bc12889b1bde2359c0c

I deleted prop position in the collection object.
I moved prop shareUrl into items type collection.

coyotte508 · 2025-07-03T18:08:23Z

hmm not sure, https://huggingface.co/api/collections/google/gemma-3n-685065323f5984ef315c93f4 I do have position and shareUrl:

quanghuynt14 · 2025-07-03T21:32:00Z

ah sorry. I assumed that the type of collection returned by GET /api/collections/{namespace}/{slug}-{id} was the same as the ones returned by GET /api/collections but it's not :/

The collection object within the collection list does not contain position or shareUrl ✅

But the shareUrl of an item of type collection is not present in the schema; however, I see it in "open-sci/open-sci-ref-releases-684210f647554eb6e0bb9cf3"

coyotte508 · 2025-07-03T21:57:09Z

yes indeed, we'll fix the schema on our side, thanks for pointing out

quanghuynt14 · 2025-07-03T22:34:59Z

You can review the PR. I think the type is now complete.

coyotte508

Thanks a lot!

I think just one change on how owner and item params are handled and it'll be ready to merge (I'll still re-review)

coyotte508 · 2025-07-04T08:51:57Z

packages/hub/src/lib/list-collections.ts

+
+	const search = new URLSearchParams([
+		...Object.entries({
+			limit: String(Math.min(totalToFetch, 100)),


technically it can be 10000 if all of:

owner is set

expand is set to false explicitly

but not sure we want to bother

coyotte508 · 2025-07-04T08:54:51Z

packages/hub/src/lib/list-collections.ts

+	const search = new URLSearchParams([
+		...Object.entries({
+			limit: String(Math.min(totalToFetch, 100)),
+			...(params?.search?.owner ? { owner: Array.isArray(params.search.owner) ? params.search.owner.join(",") : params.search.owner } : undefined),
+			...(params?.search?.item ? { item: Array.isArray(params.search.item) ? params.search.item.join(",") : params.search.item } : undefined),
+			...(params?.search?.q ? { q: params.search.q } : undefined),
+		}),
+	]).toString();


When params.search.owner and params.search.item are arrays, we should just repeat it, eg:

for (const owner of params.search.owner) { search.append("owner", params.search.owner) }

Eg https://huggingface.co/api/collections?owner=coyotte508&owner=julien-c if owner=['coyotte508','julien-c']

In constrast, https://huggingface.co/api/collections?owner=coyotte508,julien-c seems to return all collections... which is a bug that's getting fixed now it's been found!

By the way, for the JS lib we can force the user to put arrays for both owner and item if we want to.

yes, I fixed it ✅

coyotte508

Thanks looks good!

coyotte508 · 2025-07-04T14:57:36Z

packages/hub/src/types/api/api-collection.ts

+	| {
+		avatarUrl: string;
+		fullname: string;
+		name: string;
+		isHf: boolean;
+		isHfAdmin: boolean;
+		isMod: boolean;
+		followerCount?: number;
+		type: "org";
+		isEnterprise: boolean;
+		isUserFollowing?: boolean;
+	}
+	| {
+		avatarUrl: string;
+		fullname: string;
+		name: string;
+		isHf: boolean;
+		isHfAdmin: boolean;
+		isMod: boolean;
+		followerCount?: number;
+		type: "user";
+		isPro: boolean;
+		_id: string;
+		isUserFollowing?: boolean;
+	};


Can you extract this to ApiAuthor?

Yes ad702e7

feat(hub): list collections

f34cc25

quanghuynt14 requested a review from coyotte508 as a code owner June 29, 2025 20:17

quanghuynt14 added 2 commits June 29, 2025 22:42

fix test

81f4da6

move comment items

5bc2168

quanghuynt14 commented Jun 29, 2025

View reviewed changes

coyotte508 reviewed Jun 30, 2025

View reviewed changes

packages/hub/src/lib/list-collections.spec.ts Outdated Show resolved Hide resolved

packages/hub/src/lib/list-collections.ts Outdated Show resolved Hide resolved

packages/hub/src/lib/list-collections.ts Outdated Show resolved Hide resolved

quanghuynt14 added 2 commits July 1, 2025 03:12

fix types

194d900

fix types

711b033

quanghuynt14 requested a review from coyotte508 July 1, 2025 01:20

quanghuynt14 added 2 commits July 1, 2025 12:13

fix test

ab90d87

fix test

aa21908

quanghuynt14 added 2 commits July 4, 2025 00:01

fix type

06c8387

use hub-ci.hf.co for test

0605243

fix query params

bf57b98

coyotte508 reviewed Jul 4, 2025

View reviewed changes

quanghuynt14 added 2 commits July 4, 2025 13:36

fix query params

2d87ce5

rename searchParams

f41f930

quanghuynt14 requested a review from coyotte508 July 4, 2025 11:52

coyotte508 approved these changes Jul 4, 2025

View reviewed changes

extract ApiAuthor type

ad702e7

quanghuynt14 requested a review from coyotte508 July 4, 2025 15:22

coyotte508 merged commit 2782465 into huggingface:main Jul 4, 2025
3 of 4 checks passed

feat(hub): list collections #1568

feat(hub): list collections #1568

Conversation

quanghuynt14 commented Jun 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

quanghuynt14 Jun 29, 2025

Choose a reason for hiding this comment

Uh oh!

coyotte508 Jul 4, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

coyotte508 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

quanghuynt14 commented Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coyotte508 commented Jul 2, 2025

Uh oh!

quanghuynt14 commented Jul 3, 2025

Uh oh!

quanghuynt14 commented Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coyotte508 commented Jul 3, 2025

Uh oh!

quanghuynt14 commented Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coyotte508 commented Jul 3, 2025

Uh oh!

quanghuynt14 commented Jul 3, 2025

Uh oh!

coyotte508 left a comment

Choose a reason for hiding this comment

Uh oh!

coyotte508 Jul 4, 2025

Choose a reason for hiding this comment

Uh oh!

coyotte508 Jul 4, 2025

Choose a reason for hiding this comment

Uh oh!

quanghuynt14 Jul 4, 2025

Choose a reason for hiding this comment

Uh oh!

coyotte508 left a comment

Choose a reason for hiding this comment

Uh oh!

coyotte508 Jul 4, 2025

Choose a reason for hiding this comment

Uh oh!

quanghuynt14 Jul 4, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

quanghuynt14 commented Jun 29, 2025 •

edited

Loading

quanghuynt14 commented Jun 30, 2025 •

edited

Loading

quanghuynt14 commented Jul 3, 2025 •

edited

Loading

quanghuynt14 commented Jul 3, 2025 •

edited

Loading