Skip to content

Commit e219534

Browse files
committed
Update documentation with new installation method, needlectl configs, and demo button
- Add demo button to homepage after video - Update installation method with virtual environments and new architecture - Add new needlectl commands: update, ui start/stop - Update needlectl parameters and options - Add profiles (dev/prod) and configuration options - Create new UI component documentation - Update all needlectl component docs with current functionality - Add macOS support to prerequisites
1 parent 9b09971 commit e219534

File tree

8 files changed

+229
-20
lines changed

8 files changed

+229
-20
lines changed

docs/src/SUMMARY.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,4 +8,5 @@
88
* [Directory](needlectl/directory.md)
99
* [Query](needlectl/query.md)
1010
* [Generator](needlectl/generator.md)
11+
* [UI](needlectl/ui.md)
1112
* [Uninstallation](uninstallation.md)

docs/src/getting-started.md

Lines changed: 38 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,34 +4,68 @@
44

55
Before installing Needle, ensure that you have the following prerequisites installed:
66

7-
- **Docker:** Needle relies on Docker to containerize its services.
7+
- **Docker:** Needle relies on Docker to containerize its infrastructure services.
88
[Install Docker](https://docs.docker.com/get-docker/)
99

1010
- **Docker Compose:** This tool is required to orchestrate the multi-container setup.
1111
[Install Docker Compose](https://docs.docker.com/compose/install/)
1212

13+
- **Python 3.8+:** Required for the backend and image generator services.
14+
[Install Python](https://www.python.org/downloads/)
15+
16+
- **Git:** Required for cloning the repository and managing updates.
17+
[Install Git](https://git-scm.com/downloads)
18+
1319
> **Warning:** Make sure your user account is added to the Docker group so you can run Docker commands (e.g., `docker ps`) without needing root privileges.
1420
15-
> **Note:** Currently, Needle is supported only on **Linux**.
21+
> **Note:** Currently, Needle is supported on **Linux** and **macOS**.
1622
1723
## Installation
1824

25+
### Quick Install (Recommended)
26+
1927
To install Needle, run the following one-liner in your terminal:
2028

2129
```bash
2230
curl -fsSL https://raw.githubusercontent.com/UIC-InDeXLab/Needle/main/scripts/install.sh -o install.sh && bash install.sh && rm install.sh
2331
```
2432

33+
### Manual Installation
34+
35+
If you prefer to install manually or need more control over the process:
36+
37+
1. **Clone the repository:**
38+
```bash
39+
git clone https://github.com/UIC-IndexLab/Needle.git
40+
cd Needle
41+
```
42+
43+
2. **Run the installation script:**
44+
```bash
45+
./scripts/install.sh
46+
```
47+
48+
### Configuration Options
49+
2550
During the installation process, you will be prompted to choose the database mode. The available options are:
2651

2752
- **Fast:** Offers quick responses and indexing with lower accuracy.
2853
- **Balanced:** Provides a compromise between speed and accuracy.
2954
- **Accurate:** Prioritizes high accuracy, which may come at the cost of slower performance.
3055

31-
3256
> **Warning:** Once the database mode is set, it cannot be changed without uninstalling and reinstalling Needle, which will result in data loss.
3357
34-
> **Note:**" Needle automatically checks for GPU accessibility and will use the GPU if available to optimize performance.
58+
> **Note:** Needle automatically checks for GPU accessibility and will use the GPU if available to optimize performance.
59+
60+
### What Gets Installed
61+
62+
The installation process sets up:
63+
64+
- **Virtual Environments:** Separate Python environments for backend and image generator services
65+
- **Docker Infrastructure:** PostgreSQL, Milvus, and Redis services via Docker Compose
66+
- **Configuration Files:** Performance-optimized settings based on your chosen mode
67+
- **needlectl CLI:** Command-line interface for managing Needle
68+
- **Web UI:** Optional web interface for easier interaction
3569

3670
After the installation completes, please close and reopen your terminal session (or source your shell configuration file) to ensure that all environment variables are correctly set.
3771

docs/src/needlectl/README.md

Lines changed: 53 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,22 +7,51 @@ needlectl is the primary command-line interface that connects to the Needle back
77
The following options are available with needlectl:
88

99
- **`--api-url`**: Specifies the backend URL to connect to.
10-
_Default:_ `127.0.0.1:8000`
10+
_Default:_ `http://127.0.0.1:8000`
1111

1212
- **`--output`**: Determines the format of the output.
13-
_Supported formats:_ human-readable, JSON, YAML.
14-
_Default:_ human-readable.
13+
_Supported formats:_ human, json, yaml.
14+
_Default:_ human.
1515

16-
- **`--version`**: Prints the version information for both the Needle backend and the needlectl tool.
16+
- **`--version`** or **`-v`**: Prints the version information for both the Needle backend and the needlectl tool.
1717

18-
- **`--install-completion`**: Installs auto-completion for your current shell.
18+
- **`--home`** or **`-H`**: Specifies the Needle installation directory.
19+
_Default:_ Auto-detected from installation
1920

21+
- **`--profile`**: Selects the runtime profile (prod or dev).
22+
_Default:_ prod
23+
24+
- **`--config-dir`**: Overrides the configuration directory path.
2025

2126
> **Note:** `--api-url` and `--output` are accessible globally and in all commands, for example, in order to get the outputs of a query in JSON format you can use following command:
2227
> ```bash
2328
> needlectl --output json query run "a wolf howling"
2429
> ```
2530
31+
## Profiles & Configuration
32+
33+
You can use the `--home`/`-H` flag to point needlectl at a custom Needle installation or local checkout, and select a runtime profile (`prod` or `dev`) that auto-configures the compose files and config directory.
34+
35+
### Development Profile
36+
```bash
37+
needlectl --home $(pwd) --profile dev service start
38+
```
39+
This uses:
40+
- `NEEDLE_CONFIG_DIR=$NEEDLE_HOME/configs/fast`
41+
- Infrastructure: `docker/docker-compose.infrastructure.yaml`
42+
- Backend: Virtual environment
43+
- Image Generator: Virtual environment
44+
45+
### Production Profile
46+
```bash
47+
needlectl --home /opt/needle --profile prod service start
48+
```
49+
This uses:
50+
- `NEEDLE_CONFIG_DIR=$NEEDLE_HOME/configs/balanced` (or your chosen mode)
51+
- Infrastructure: `docker/docker-compose.infrastructure.yaml`
52+
- Backend: Virtual environment
53+
- Image Generator: Virtual environment
54+
2655
2756
2857
## Components Overview
@@ -35,17 +64,32 @@ Manages Needle's core service operations, including:
3564
- Restarting the service
3665
- Stopping the service
3766
- Viewing logs
38-
- Upgrading the service
67+
- Updating components
68+
- Managing configuration
3969
4070
### [Directory](directory.md)
4171
Handles image directory management tasks, such as:
4272
- Adding directories
4373
- Removing directories
44-
- Indexing images
74+
- Listing directories
75+
- Modifying directory settings
76+
- Viewing directory details
4577
4678
### [Query](query.md)
47-
Executes natural language queries against the image database, providing flexible and powerful search capabilities.
79+
Executes natural language queries against the image database, providing flexible and powerful search capabilities:
80+
- Running search queries
81+
- Viewing query logs
82+
- Managing query configuration
4883
4984
### [Generator](generator.md)
50-
Manages image generation operations, allowing you to adjust parameters and generate images as needed.
85+
Manages image generation operations, allowing you to:
86+
- List available generators
87+
- Configure generator settings
88+
- Manage image generation parameters
89+
90+
### [UI](ui.md)
91+
Controls the web-based user interface:
92+
- Starting the web UI
93+
- Stopping the web UI
94+
- Managing UI configuration
5195

docs/src/needlectl/directory.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,11 +47,23 @@ This command removes an image directory from Needle.
4747

4848
This command lists all the directories currently registered in Needle.
4949

50+
- **What it does:**
51+
- Connects to the backend API
52+
- Retrieves the list of all registered directories
53+
- Displays directory information including status, file counts, and indexing progress
54+
5055
- **Usage Examples:**
5156
```bash
57+
# List all directories
5258
needlectl directory list
59+
60+
# List directories with JSON output
61+
needlectl --output json directory list
5362
```
5463

64+
- **Output:**
65+
Shows directory ID, path, name, status, file count, indexing progress, and other relevant information.
66+
5567
### `modify`
5668
This command allows you to modify the configuration for your added directories, you can enable/disable them for
5769
searching. Needle will not search in disabled directories.

docs/src/needlectl/query.md

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -10,10 +10,10 @@ This command executes a search query against the Needle image retrieval database
1010

1111
- **Options:**
1212
- **`prompt`** (string): The search prompt in natural language.
13-
- **`-n` or `--n`** (integer, optional): Specifies the number of images to retrieve.
14-
- **`-m` or `--m`** (integer, optional): Specifies the number of images per engine.
15-
- **`-k` or `--k`** (integer, optional): Specifies the number of engines to use.
16-
- **`--image-size`** (integer, optional): Sets the desired image size for image generation.
13+
- **`-n`** (integer, optional): Specifies the number of images to retrieve.
14+
- **`--num-images-to-generate`** (integer, optional): Specifies the number of images to generate.
15+
- **`--num-engines`** (integer, optional): Specifies the number of engines to use.
16+
- **`--image-size`** (string, optional): Sets the desired image size for image generation (e.g., "512", "1024").
1717
- **`--include-base-images`** (boolean, optional): Indicates whether to include base images in the results preview.
1818
- **`--use-fallback`** (boolean, optional): Indicates whether to use fallback mode if the primary engines set fails.
1919

@@ -29,7 +29,10 @@ This command executes a search query against the Needle image retrieval database
2929
needlectl query run "red cars"
3030

3131
# Run a query with additional options
32-
needlectl query run "red cars" --n 5 --m 2 --k 3 --image-size 512 --include-base-images true --use-fallback false
32+
needlectl query run "red cars" -n 5 --num-images-to-generate 2 --num-engines 3 --image-size 512 --include-base-images true --use-fallback false
33+
34+
# Run a query with JSON output
35+
needlectl --output json query run "mountain landscape" -n 10
3336
```
3437

3538
### `log`

docs/src/needlectl/service.md

Lines changed: 30 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
# Service Component
22

3-
The service component of `needlectl` manages the core operations of Needle. It connects to the Needle backend service and uses Docker Compose to start, stop, restart, check status, view logs, and configure the service environment.
3+
The service component of `needlectl` manages the core operations of Needle. It connects to the Needle backend service and uses Docker Compose to start, stop, restart, check status, view logs, update components, and configure the service environment.
44

5-
> **Note:** Most commands rely on the `--api-url` option (default: `127.0.0.1:8000`) to connect to the backend API. You can set this globally if needed.
5+
> **Note:** Most commands rely on the `--api-url` option (default: `http://127.0.0.1:8000`) to connect to the backend API. You can set this globally if needed.
66
77
## Commands Overview
88

@@ -77,6 +77,34 @@ Displays the logs from the Needle backend service.
7777
needlectl service log
7878
```
7979

80+
### `update`
81+
Updates Needle components to the latest versions.
82+
83+
- **Options:**
84+
- **`--force`** or **`-f`**: Force update even if already up to date
85+
- **`--component`** or **`-c`**: Update specific component (needlectl, backend, ui, or all)
86+
87+
- **What it does:**
88+
- Checks for the latest release information
89+
- Shows current versions of all components
90+
- Updates the specified components
91+
- Provides status updates during the process
92+
93+
- **Usage Examples:**
94+
```bash
95+
# Update all components
96+
needlectl service update
97+
98+
# Update only the backend
99+
needlectl service update --component backend
100+
101+
# Force update even if up to date
102+
needlectl service update --force
103+
```
104+
105+
- **Output:**
106+
Displays current and latest versions, then updates the specified components with progress indicators.
107+
80108
### `config`
81109
Manages the configuration for the Needle service environment.
82110

docs/src/needlectl/ui.md

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# UI Component
2+
3+
The UI component of `needlectl` manages the web-based user interface for Needle. It allows you to start and stop the web UI, which provides a graphical interface for interacting with Needle's features.
4+
5+
## Commands Overview
6+
7+
### `start`
8+
9+
Starts the Needle web UI on a specified port.
10+
11+
- **Options:**
12+
- **`--port`** or **`-p`**: Port to run the UI on (default: 3000)
13+
14+
- **What it does:**
15+
- Starts the web UI server
16+
- Serves the React-based interface
17+
- Provides access to all Needle features through a web browser
18+
19+
- **Usage Examples:**
20+
```bash
21+
# Start UI on default port (3000)
22+
needlectl ui start
23+
24+
# Start UI on custom port
25+
needlectl ui start --port 8080
26+
```
27+
28+
- **Output:**
29+
Displays the URL where the UI is accessible (e.g., `http://localhost:3000`)
30+
31+
### `stop`
32+
33+
Stops the running Needle web UI.
34+
35+
- **What it does:**
36+
- Stops the web UI server
37+
- Frees up the port for other uses
38+
39+
- **Usage Example:**
40+
```bash
41+
needlectl ui stop
42+
```
43+
44+
- **Output:**
45+
Confirms that the UI has been stopped
46+
47+
## Web UI Features
48+
49+
The web interface provides:
50+
51+
- **Search Interface**: Interactive query interface with sample queries
52+
- **Directory Management**: Visual management of image directories
53+
- **Generator Configuration**: Easy configuration of image generators
54+
- **System Status**: Real-time monitoring of service health
55+
- **Results Visualization**: Rich display of search results and generated images
56+
57+
## Accessing the UI
58+
59+
Once started, you can access the web UI by opening your browser and navigating to:
60+
61+
- **Default URL**: `http://localhost:3000`
62+
- **Custom Port**: `http://localhost:[PORT]` (where [PORT] is your chosen port)
63+
64+
The UI will automatically connect to your running Needle backend service.

docs/src/overview.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,29 @@ Needle is an open-source image retrieval database with high accuracy that can ea
1414
<source src="media/needle-demo.mp4" type="video/mp4">
1515
</video>*Watch as Needle transforms natural language queries into precise image retrieval results in real time.*
1616

17+
## 🎨 Try the Interactive Demo
18+
19+
Experience Needle's full capabilities with our interactive demo! Test different queries, explore the interface, and see how Needle works with real data.
20+
21+
<div style="text-align: center; margin: 2rem 0;">
22+
<a href="https://uic-indexlab.github.io/Needle/demo/"
23+
target="_blank"
24+
style="display: inline-block;
25+
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
26+
color: white;
27+
padding: 1rem 2rem;
28+
border-radius: 8px;
29+
text-decoration: none;
30+
font-weight: bold;
31+
font-size: 1.1rem;
32+
box-shadow: 0 4px 15px rgba(0,0,0,0.2);
33+
transition: transform 0.2s ease;">
34+
🎨 Try Needle Demo
35+
</a>
36+
</div>
37+
38+
*The demo showcases sample queries, image generation, and similarity search with pre-processed datasets.*
39+
1740
## Comparison to State-of-the-Art Methods
1841

1942
Curious how Needle measures up against other cutting-edge approaches? Here, you'll soon find performance plots that

0 commit comments

Comments
 (0)