Skip to content

Lxdex#500

Merged
turtle0x1 merged 11 commits into
masterfrom
lxdex
Nov 4, 2025
Merged

Lxdex#500
turtle0x1 merged 11 commits into
masterfrom
lxdex

Conversation

@turtle0x1

@turtle0x1 turtle0x1 commented May 15, 2022

Copy link
Copy Markdown
Owner

The first public LXD indexer? Maybe if we ignore general purpose JSON indexers 🥳

When you have have more than 2 or 3 LXD hosts searching becomes slow because to search "fully" you have to hit multiple endpoints. This branch add supports for a multistage process to create a searchable index that should be snappy to search.

There is still quite a bit todo;

  • Filter search results based on user access
  • Scale test (beyond copying the array)
  • Grab more paramaters
  • Stop assuming user.user-data is YAML
  • Concurrency on cache builder
  • Document how it works
  • Get scripts into cron jobs
  • Adding settings to disable / enable / configure search
    • Scanning 30+ hosts to recursion level 2 every 5 mins probably isn't a good idea

Example search for "ubuntu run" (as in to mean, showing instances in Running state with a ubuntu image)
lxdex

We do this because we lots of hosts it could be very slow to gather
all the data.

Areas for improvement

  - Concurrency
    - `go doSomething()` has spoiled me, i miss go :(
The index is a simple tree that looks something like this

[
   "any_string_found_in_a_property"=>[
      // A collection of entities matching
   ]

   // I.E Running instances
   "Running"=> [0, 1, 2]
]

This allows for non-recursive searches later

Currently we dont support indexing an entire array, you have to specify
key/s within the array using regex
@turtle0x1 turtle0x1 marked this pull request as ready for review November 4, 2025 21:19
@turtle0x1 turtle0x1 merged commit 897aae7 into master Nov 4, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant