-brown?logo=linux&style=for-the-badge){kind=icon}
-blue?logo=apple&style=for-the-badge){kind=icon}
Lightnovel Crawler downloads web novels and similar fiction from many online reading sites and saves them as e-book so you can read offline on a phone, tablet, or e-reader.
- Save a story you follow into a single EPUB (or another format) instead of hundreds of separate web pages.
- Search supported sites from the app and pick a title without copying long URLs by hand.
- Run a small private server on your home network so you can use the same workflow from another device on that network.
Sites publish fiction under their own terms and copyright. This tool is meant for personal use only, for example keeping a backup of material you already have access to, where the licence allows it. Do not use it to redistribute or sell someone else's work. You are responsible for how you use Lightnovel Crawler.
Pick one approach. You do not need to do all of them.
| Platform | Link | Note |
|---|---|---|
| Windows | lncrawl.exe | SmartScreen may warn about an unknown app. |
| Linux | lncrawl | pip is often easier for updates. |
| macOS | lncrawl | pip is often easier for updates. |
To get older versions visit the Releases page
Check that it works:
- Double click and run the downloaded file. It may take some time to show the first screen depending on your machine.
PyPI package: lightnovel-crawler
-
From PyPI repository.
pip install -U lightnovel-crawler
If it fails, you can try:
python3 -m pip,pip3, orpython -m pip. -
You can also install directly from GitHub.
-
Stable branch (
master):pip install -U git+https://github.com/lncrawl/lightnovel-crawler.git#egg=lightnovel-crawler
-
Development branch (
dev): may include fixes or breaking changes.pip install -U https://github.com/lncrawl/lightnovel-crawler/tarball/refs/heads/dev#egg=lightnovel-crawler
-
-
Check that it works:
lncrawl -h
If
lncrawlis not found, trypython3 -m lncrawl, orpython -m lncrawl.
You need to have Docker installed.
mkdir -p lncrawl-data
docker pull ghcr.io/lncrawl/lightnovel-crawler
docker run -v ./lncrawl-data:/data -it -p 8080:8080 --name lncrawl-server ghcr.io/lncrawl/lightnovel-crawler -ll serverCheck that it works:
- Visit http://localhost:8080 in your browser.
- You can sign in with the default account:
admin/admin.
If you only need epub, text, or json outputs, you can skip this section.
To create PDF, MOBI, DOCX, and other formats, install Calibre. Lightnovel Crawler calls Calibre's tools in the background when you pick those formats.
After installation, locate location of the ebook-convert in the Calibre installation directory, and add the folder to your Path variables. e.g.:
export PATH="$PATH:/Applications/calibre.app/Contents/MacOS"Just run the executable or type lncrawl in your terminal and you're ready to go! Browse, download, and enjoy your novels all in one place, no command line required.
Check the available options in the terminal:
$ lncrawl -h
Usage: lncrawl [OPTIONS] COMMAND [ARGS]...
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --verbose -l Log levels: -l = warn, -ll = info, -lll │
│ = debug │
│ --config -c PATH Config file │
│ --install-completion Install completion for the current │
│ shell. │
│ --show-completion Show completion for the current shell, │
│ to copy it or customize the │
│ installation. │
│ --help -h Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ───────────────────────────────────────────────────────────────────╮
│ version Show current version. │
│ config View and modify configuration settings. │
│ sources Manage sources. │
│ crawl Crawl from novel page URL. │
│ search Search for novels by query string. │
│ server Run web server. │
╰──────────────────────────────────────────────────────────────────────────────╯
Download a few chapters from a novel's main info page URL:
lncrawl -ll crawl "https://example.com/novel/1234" -f epub --first 10Want to help with code, sources, or docs?
- DeepWiki (AI-assisted project overview)
- .github/docs/ARCHITECTURE.md (architecture and CI notes)
- .github/docs/CREATING_CRAWLERS.md (add or fix a source)
- .github/docs/DOCKER.md (Docker details)
- .github/FORKING.md (CI on forks)
Install uv (or run make setup, which can install it for you). From the repo root:
git clone https://github.com/lncrawl/lightnovel-crawler.git
cd lightnovel-crawler
make install # or: make (same default target)
make startEquivalent with uv only (after submodules are initialized):
git submodule update --init --recursive
uv sync --extra dev
uv run python -m lncrawl -ll serverTargets are defined in the Makefile; install / sync / dependency targets use uv sync --extra dev.
# Setup and install
make setup # Sync submodules and install uv
make install # setup + uv sync (default target: `make` or `make all`)
make sync # uv sync only
make upgrade # setup + uv sync --upgrade
# Development servers and tooling
make start # Backend server only
make watch # Backend with auto-reload
make lint # ruff format and check
make add-source # Guided CLI to add a new source crawler
# Version (writes lncrawl/VERSION via scripts/bump.py)
make patch # bump patch
make minor # bump minor
make major # bump major
# Build
make build # print version + install + wheel + exe
make build-wheel # Python wheel (python -m build -w)
make build-exe # PyInstaller (setup_pyi.py)
# Dependencies (second word is the package name)
make add-dep <package> # e.g. make add-dep httpx
make add-dev <package> # add to optional extra `dev`
make rm-dep <package>
make rm-dev <package>
# Docker (uses compose.yml in repo root unless you pass -f elsewhere)
make docker-build # Base image, then app image
make docker-base # Base image only (Calibre + deps)
make docker-up # docker compose up -d
make docker-down # docker compose down
make docker-logs # docker compose logs -f
# Other
make version # Print version from lncrawl/VERSION
make clean # Remove .venv, logs, build, dist, *.egg-info, __pycache__
make submodule # git submodule sync + update (init, recursive, remote)
# Update repo + submodules without make
git pull && make submodule
# Run CLI from source
uv run python -m lncrawlRecommended: Scaffold with the CLI (optionally, it can use AI for generation):
make add-sourceManually: Copy one example such as [sources/_examples/_01_general_soup.py](sources/_examples/_01_general_soup.py) into the right sources/{lang}/ folder and implement the required methods.
Full guide: .github/docs/CREATING_CRAWLERS.md.
Natively supported formats:
| Format | Description | Use Case / App |
|---|---|---|
| 📚 EPUB | Standard eBook format | Most eReaders, apps |
| 📃 TXT | Plain text file | Any text editor, simple |
| 🗂️ JSON | Structured chapter data | Parsing/scripts/devs |
Supported if Calibre’s ebook-convert tool is available:
| Format | Description | Use Case / App |
|---|---|---|
| Portable Document Format | Universal, print-ready | |
| 🔲 MOBI | Kindle eBook (legacy) | Older Kindle devices |
| 🔳 AZW3 | Kindle eBook (modern) | Current Kindles |
| 📝 DOCX | Microsoft Word document | MS Word, LibreOffice |
| 📑 RTF | Rich Text Format | WordPad, others |
| 📔 FB2 | FictionBook eBook format | FB2 readers |
| 📕 LIT | Microsoft Reader (obsolete) | Old MS Reader |
| 📗 LRF | Sony eBook format | Sony Readers |
| 🗄️ PDB | PalmDoc/Plucker (legacy) | PalmOS, old devices |
| 📘 RB | RocketBook/REB1100 | Legacy readers |
| 📙 TCR | Psion eBook format | Psion readers |
To request for a new source not included in the following list, please create an issue.
We are supporting 358 sources and 393 crawlers.
| Source URL | Version | Contributors | |
|---|---|---|---|
| 🔍 | https://arnovel.me/ | 27 |   |
| 🔍 | https://kolnovel.com/ | 1 | |
| https://rewayat.club/ | 13 | |
| Source URL | Version | Contributors | |
|---|---|---|---|
| https://cclawtranslations.home.blog/ | 75 |  |
|
| https://domentranslations.wordpress.com/ | 75 | |
|
| 🔍🔑 | https://novelasligeras.net/ | 6 | |
| Source URL | Version | Contributors | |
|---|---|---|---|
| 🔍 | https://chireads.com/ | 13 |  |
| 🔍 | https://lightnovelfr.com/ | 9 |  |
| https://noveldeglace.com/ | 7 | |
|
| https://xiaowaz.fr/ | 8 | |
| Source URL | Version | Contributors | |
|---|---|---|---|
| 🤖🔍 | https://ncode.syosetu.com/ | 24 | |
| 🤖🔍 | https://syosetu.org/ | 6 | |
| Source URL | Version | Contributors | |
|---|---|---|---|
| 🔍 | https://blnovels.net/ | 10 | |
| 🔍 | https://centralnovel.com/ | 13 | |
| 🔍 | https://illusia.com.br/ | 5 | |
| https://novelmania.com.br/ | 7 | |
|
| 🔍 | https://phoenixnovels.com.br/ | 5 | |
| Source URL | Version | Contributors | |
|---|---|---|---|
| 🔍 | https://fenrirscans.com/ | 4 | |
| Source URL | Version | Contributors | |
|---|---|---|---|
| 🤖🔍 | https://docln.net/ | 15 | |
| 🤖🔍 | https://ln.hako.vn/ | 15 | |
| 🤖🔍 | https://truyenfull.vn/ | 8 | |
Questions and tips: GitHub Discussions.