Hister (Dev Build)

Automated build from master.
Commit: 5031ea4
Built on: 2026-05-04 10:21:35 UTC

v0.13.0

New Features

Semantic Search

Full vector search support via sentence-embedding models. Documents are chunked
and embedded at index time; search queries are embedded at query time and ranked
by cosine similarity. Two storage backends are supported:

• SQLite (default, via bundled sqlite-vec) zero extra infrastructure required
• PostgreSQL with pgvector auto-selected when the database is Postgres

Configure the embedding API endpoint, model, dimensions, and chunking parameters
in the new semantic config section. Semantic search is opt-in and off by default.
Relevance scores are shown alongside results when semantic search is active.

OAuth / SSO Authentication

OAuth 2.0 and OpenID Connect (OIDC) providers can now be configured as login
methods. Add one or more entries to the new server.oauth config section with
client_id, client_secret, configuration_url (for OIDC auto-discovery), or
manual auth_url / token_url, and optional scopes. Multiple providers can be
active at the same time alongside the built-in username/password login.

MCP Server

Hister now exposes a Model Context Protocol
endpoint at /api/mcp, enabling LLM agents and MCP-compatible tools to search
the index directly.

Persistent Crawler State Management

Recursive crawl jobs (hister index -r) are now stored in the database and
survive interruptions. Each job gets a unique ID (auto-generated or set via
--job-id). Pass --job-id <id> without --recursive to resume an
interrupted crawl from exactly where it left off, including original validator
rules and visited-URL counts.

New Extractors

• Wikipedia extracts the article body and infobox, rewrites relative links, and sanitizes the output
• GitHub project extracts repository descriptions and README content from GitHub project pages
• Lobste.rs dedicated extractor for Lobste.rs story and comment pages
• yt-dlp extracts video metadata (title, description, channel) from video pages via yt-dlp
• JSON-LD surfaces structured metadata (@type, headline, description) from pages that embed JSON-LD

All extractors now expose a Description() method, and an extractor information
page is available at /extractors in the web UI.

OpenSearch Suggestions

The server now serves an OpenSearch suggestions endpoint (/api/suggest),
allowing browsers to display search-as-you-type completions when Hister is
configured as a search engine.

Enhancements

Crawler Backend for All Index Operations

The --backend flag (and --backend-option) is now available on both
hister index (plain and --recursive) and hister import-browser, allowing
a headless Chrome/Chromium backend for JavaScript-heavy pages without running a
full recursive crawl:

hister index --backend chromedp https://example.com
hister import-browser --backend chromedp --backend-option exec_path=/usr/bin/chromium

Headers and cookies can also be injected per-invocation:

hister index --header "Accept-Language=en" --cookie "session=abc; Domain=example.com" https://example.com

Cookies use standard Set-Cookie format with a required Domain attribute.

CLI Search Improvements

• --limit N flag caps the number of results returned
• --fields flag selects which document fields to include in output
• --html flag includes raw HTML content in the output
• Paging support added to both CLI search and list-urls
• list-urls now fetches results from the server by default; --offline connects directly to the index without a running server

Quoted Field Queries

Field-qualified queries now support quoted values, enabling correct deletion and
lookup of URLs that contain spaces (common on Windows file paths):

url:"file:///C:/Users/My Documents/notes.txt"

Preview Panel Polish

• Preview title is now clickable (opens the result URL)
• Preview panel maximises available content width
• JSON-LD metadata surfaced inside the preview panel
• Dark theme font colors fixed in preview popup

NixOS / Nix Module

• systemd and launchd hardening applied to the Hister service units
• New services.hister.environmentFile option for secrets injection
• openFirewall now requires explicit opt-in
• services.hister.config renamed to services.hister.settings

Other

• Executable size reduced ~70 MB by switching to a trimmed lingua-go fork
• Sensitive content rejection errors surfaced in the browser extension
• --verbose flag on hister delete lists matched URLs before deleting
• Priority result deduplication now copies body text from the original result
• /suggest endpoint protected by auth middleware and Sec-Fetch-Site header check
• Version information included in the MCP endpoint response
• Timezone data bundled into the binary for environments without a system tzdata

Bug Fixes

• File URLs (file://) now handled correctly in the UI for both opening and deletion (#362)
• Browser extension authentication documentation corrected (#366)
• URLs no longer lowercased during query building, preventing mismatches on case-sensitive paths
• History view correctly filtered per-user in multi-user mode (#314)
• Token authentication middleware now respects NoAuth flag (#348)
• Documents with no HTML content no longer attempt HTML extraction (#351)
• Extension no longer resubmits documents after a 406 Not Acceptable response
• Priority results correctly deduplicated against standard results
• File indexing fixed on Windows
• Wide tables no longer overflow the preview panel
• Score field populated correctly in search responses
• aws_access_key sensitive content pattern tightened to reduce false positives
• Home-manager service units correctly gated on host platform in Nix module

v0.12.0

New Features

Web Crawler

New hister index -r <url> command crawls sites recursively.
Supports the standard Go HTTP backend and a headless Chrome backend (chromedp).
Configurable depth, link count, allowed/excluded domains, and URL patterns.

PostgreSQL Backend

Full PostgreSQL support as an alternative to SQLite.
Configure via a postgres:// connection string in server.database.

Extractor Pipeline Overhaul

Extractors are now configurable, have explicit states (continue/done), and expose
a Preview() method used by the preview panel. New extractors included:

• Custom pkg.go.dev extractor for Go documentation pages
• Basic Stack Overflow extractor

Desktop Readability Panel

Focused search results load automatically in a split-pane reader on the right side
on screens wider than 1280 px. The panel is togglable and its open/closed state persists.

Enhancements

• HTML sanitizer (bluemonday) applied to all extracted content
• metadata field added to documents for arbitrary key/value data
• search input type attribute on search fields for better mobile UX
• Build commit ID shown in the version string
• Admin users can create global indexes or indexes on behalf of other users
• hister index skips already-indexed URLs by default; pass --force to reindex them
• URL and domain wildcard matching automatically anchors to start and end
• Table of contents added to the API docs page
• Document indexed date shown in the preview panel
• Search query reflected in the browser tab title
• WebSocket communication optimised to reduce redundant round-trips
• Automatic redirect on zero results is now optional (configurable)
• import command renamed to import-browser to free import for index import/export

Bug Fixes

• Browser history database opened read-only to avoid lock conflicts (#304)
• History entries now deleted when their associated document is deleted (#303)
• Crawler user-agent correctly applied after redirect handling (#302)
• Fixed field-specific alternation parts in query parser (#274)
• Negated query terms no longer trimmed twice
• HTML field no longer leaks into search results (#268)
• Expanded query hint only shown when the expansion is longer than the original query
• URL changes after HTTP redirects now resolved correctly
• Crawler no longer stops on HTTP errors
• Crawler timeout now applied during browser history import (#278)
• Pinned result titles no longer truncated on narrow screens
• Dark mode handled correctly in the preview panel
• Mobile layout no longer introduces unwanted line breaks

v0.11.0

What's Changed

• Multi-user support.
• Better authentication method for the extension.
• Enhanced delete. Queries can be used to delete multiple items. --dry flag.
• Reindex does not require server stop anymore.
• Enhanced history view.
• Extension feedback if the active tab has a matching skip rule.
• Add pages/domains to skip rules directly from the extension popup.
• Plenty of bugfixes.

v0.10.0

New contributions

• Bump softprops/action-gh-release from 2.5.3 to 2.6.1 in the gha-deps group by @dependabot[bot] in #228
• [fix] include parent directory in local file titles by @FlameFlag in #234
• [feat] add POST /api/batch endpoint for bulk add/delete/get operations by @FlameFlag in #233
• [feat] add custom headers support to browser extension by @FlameFlag in #238
• [feat] add light mode & theme switcher to browser extension by @FlameFlag in #239
• [feat] switch to mode-watcher for theme management by @FlameFlag in #240
• [fix] prevent Svelte 5 proxy from being stored in Chrome storage for customHeaders by @FlameFlag in #241
• [enh] strip debug symbols from release binary by @FlameFlag in #242
• [fix] include full end date in date range search by @mvanhorn in #244

New Contributors

• @mvanhorn made their first contribution in #244

Full Changelog: v0.9.0...v0.10.0

v0.9.0

What's Changed

Added local file indexing support

see our docs for more details

Other contributions

• [fix] fix all golangci-lint errcheck and unused warnings by @FlameFlag in #182
• [enh] add dependabot support by @FlameFlag in #185
• Update Nix hashes by @github-actions[bot] in #200
• [enh] make ext use same design as app and website by @FlameFlag in #199
• [fix] fix Docker publish workflow and Dockerfile by @FlameFlag in #205
• [fix] improve list-urls performances and fix missing entries by @scumjr in #206
• [ci] add PR lint workflow for Go, NPM format, and no merge commits by @FlameFlag in #211
• [feat] update to vite 8.0.0 and use rolldown by @FlameFlag in #212
• [fix] patch workspace shebangs in nix frontend derivation by @FlameFlag in #213
• Update Nix hashes by @github-actions[bot] in #214
• [ci] add build for hister, @hister/website and @hister/ext for PR's by @FlameFlag in #215
• feat: add ability to index and view local files by @jamestthompson3 in #210
• docs: update docs on how local indexing works by @jamestthompson3 in #217
• [fix] nix/package.nix add files package to build inputs by @FlameFlag in #226

New Contributors

• @dependabot[bot] made their first contribution in #201
• @jamestthompson3 made their first contribution in #210

Full Changelog: v0.8.0...v0.9.0

v0.8.0

Changelog

• afd62ed [enh] bump version
• 0eeee97 [fix] reintroduce csp - fixes #180

What's Changed

• [feat] add shared UI components and migrate app and website to use them by @FlameFlag in #165
• [fix] wrap long TOC sidebar text instead of overflowing by @FlameFlag in #171
• [enh] add syntax highlighting with shiki by @FlameFlag in #169
• [enh] add Prettier and EditorConfig across all workspaces by @FlameFlag in #168
• [enh] migrate to Alpine base & add compose.yml by @FlameFlag in #175

Full Changelog: v0.7.0...v0.8.0

v0.7.0

Fix docker front-end build

v0.6.0

Changelog

What's Changed

• refactor(nix): rewrite packaging following best practices by @XYenon in #114
• [fix] pressing tab deletes query on no suggestion, fix inconsistent history by @FlameFlag in #117
• Enhance server configuration documentation and fix documentation layout by @sinsky in #118
• feat: migrate to npm workspaces for manging frontend by @FlameFlag in #121
• Part 1/5 of the new redesign by @FlameFlag in #126
• Part 2/5 of the new redesign by @FlameFlag in #127
• Part 3/5 of the new redesign by @FlameFlag in #128
• Part 4/5 of the new redesign by @FlameFlag in #129
• Part 5/5 of the new redesign by @FlameFlag in #130
• fix(querybuilder): handle UTF-8 queries in lexer by @yshalsager in #132
• [refactor] extract dedicated API client package by @FlameFlag in #135
• [enh] add & use more shadcn-svelte components by @FlameFlag in #134
• Allow base_url to include subfolder path by @singhAmandeep007 in #133
• [fix] add client module to nix package inputs by @antifuchs in #137
• [enh] improve mobile responsiveness & add updated_at timestamps to history by @FlameFlag in #136
• [fix] fix search results scrolling, input resizing, and keyboard nav by @FlameFlag in #138
• Update Nix hashes by @github-actions[bot] in #139
• [enh] re-design the website by @FlameFlag in #140
• better apply brutalist design on the app by @FlameFlag in #141
• [enh] redesign rules page with bold brutalist layout by @FlameFlag in #142
• [fix] {api -> api-docs} page to avoid conflict with the api endpoint by @FlameFlag in #143
• [enh] redesign api-docs page by @FlameFlag in #144
• [enh] improve mobile responsiveness by @FlameFlag in #145
• [enh] add -f to search (csv, json) by @FlameFlag in #148
• [fix] check skip rules client-side during import by @FlameFlag in #152
• [fix] pass access token from config to HTTP client by @FlameFlag in #153
• [refactor] extract mouse handling into dedicated package by @FlameFlag in #160
• [enh] switch ext to use vite for build by @FlameFlag in #162
• [enh] redesign docs page by @FlameFlag in #158

New Contributors

• @XYenon made their first contribution in #114
• @yshalsager made their first contribution in #132
• @singhAmandeep007 made their first contribution in #133
• @antifuchs made their first contribution in #137

Full Changelog: v0.5.0...v0.6.0

v0.5.0

What's Changed

• Refactor Docker build process and introduce multi-stage distroless images by @sinsky in #73
• fix: case-insensitive search for title/text/field queries by @doobidoo in #76
• [fix] TUI WebSocket connection fails by @FlameFlag in #94
• [enh] display search duration in seconds by @scumjr in #99
• [enh]: add rolling release gha by @FlameFlag in #96
• [enh] prevent claude from making ai slop by @FlameFlag in #102
• [enh] migrate search.js to svelte by @FlameFlag in #89
• Update Nix vendorHash by @github-actions[bot] in #104
• Add Viper configuration loader with environment variable support for Docker flexibility by @sinsky in #100
• [fix] update Nix hashes and fix hash extraction script by @FlameFlag in #107

New Contributors

• @sinsky made their first contribution in #73
• @doobidoo made their first contribution in #76
• @github-actions[bot] made their first contribution in #104

Full Changelog: v0.4.0...v0.5.0