LLMS.TXT Generator

A Webflow Cloud app that automatically generates llms.txt files containing links to markdown pages for live pages and collections on your Webflow site. You can choose which pages and collections to include in your llms.txt file.

Table of contents

Project description
Features
Tech Stack
Project dependencies
Getting Started
Deploy to Webflow Cloud
Troubleshooting
Contributing guidelines
Additional documentation
How to get help

Project description

With LLMS.TXT Generator you can automatically create structured documentation from your Webflow content that's optimized for Large Language Model (LLM) consumption. The app helps you expose selected pages and collections from your Webflow site as clean, markdown-formatted documentation accessible via a standardized llms.txt endpoint.

This project is intended for Webflow site owners and developers who want to make their site content accessible to LLMs and AI tools through a standardized documentation format.

✨ Features

Selective Content Exposure: Choose which pages and collections to include in your documentation
Automatic Markdown Conversion: Clean conversion of Webflow content to structured markdown
Admin Interface: User-friendly interface for managing exposed content

🛠️ Tech Stack

Webflow Cloud - Infrastructure for hosting the webapp alongside a Webflow site
Astro - Fullstack web application framework
Webflow Data APIs - Content source and site management
KV Storage - Persistent storage for content and settings

Project dependencies

Before using LLMS.TXT Generator, ensure you have:

Node.js 18+ installed
npm package manager
Git version control
Webflow account (sign up for free)
Webflow CLI installed (npm install -g @webflow/webflow-cli)
A Webflow site with content to expose

Getting Started

Get started with LLMS.TXT Generator by forking this repository and setting up your local development environment.

Install the project

Fork and clone the repository

Fork this repo into your own repositories so you have a copy to work with. Clone the repo down to your local machine
```
git clone https://github.com/your-username/llms-txt-generator-webapp.git
cd llms-txt-generator-webapp
```
Install dependencies
```
npm install
```

Configure environment variables

Generate a Webflow Site API Token

Navigate to your Webflow site settings:
- Go to Site Settings → Apps & Integrations
- In the "API access" section, click "Generate API token"
- Choose a name, ensure it has the following scopes:
  - Authorized user - Read-only
  - CMS - Read and write
  - Pages - Read and write
  - Sites - Read and write
- Generate and save the token for step 2 (you cannot retrieve it later)
Note: Only users with Adnub access (or higher) can generate an API token.
Set up environment variables

Duplicate .env.example and rename to .env and fill in the values for both of these new duplicated files:
```
cp .env.example .env
```
Add your configuration:
```
WEBFLOW_SITE_ID=your_site_id_here
WEBFLOW_SITE_API_TOKEN=your_api_token_here
DOMAIN=your-domain.webflow.io
```
- Find your WEBFLOW_SITE_ID in Site Settings → General → Overview
- Set WEBFLOW_SITE_API_TOKEN to the token value from step 1
- Set DOMAIN to your site's domain (can be the webflow staging domain or the custom domain your site is published on)
Now duplicate .dev.vars.template and rename it to .dev.vars and fill in the same values as above. In local development, .env will be used when running the app outside of wrangler dev.

Run the app locally

Start the development server
```
npm run dev
```
The development server will start and show you the local URL.
Access the admin interface

Open http://localhost:4321/llmstxt in your browser to see the admin interface.
Configure content exposure
- Navigate to "Manage Collections" to select which collections to expose
- Navigate to "Manage Pages" to select which pages to expose
- Click "Regenerate llms.txt" to generate your documentation
View generated documentation

Visit http://localhost:4321/llmstxt/llms.txt to see your generated documentation.

🚀 Deploy to Webflow Cloud

Once everything works locally, you can deploy the app to your Webflow site with Webflow Cloud. Push any local changes made to your local project up to your remote branch first.

Create Webflow Cloud app

For detailed guidance, see the Webflow Cloud documentation.

Connect GitHub to Webflow

In your Webflow site settings:
- Navigate to the Webflow Cloud tab
- Click "Install GitHub App" and follow prompts
- Grant Webflow access to your forked repository
Create the project
- Click "Create New Project" in Webflow Cloud
- Enter your project name and GitHub repository location
- When creating an Environment:
  - Branch: Select your working branch (usually main)
  - Mount Path: Enter /llmstxt
- You will need to re-publish your Webflow site at this time
Add environment variables

In the Deployments page:
- Open the "Environment Variables" tab
- Add all variables from your .env file
- These ensure your deployed app can access the configuration
Deploy the app
- Click "Deploy latest commit"
- After deployment completes, click the URL under "Environment URL"
- You should see your app running at https://your-site.webflow.io/llmstxt

Troubleshooting

Issue	Solution
Environment variables not working	Ensure you have both `WEBFLOW_SITE_API_TOKEN` and `WEBFLOW_SITE_ID` set for development, and all variables properly configured in Webflow Cloud for production.
No content appears in llms.txt	Make sure you've selected pages/collections to expose in the admin interface and clicked "Regenerate llms.txt".
Build fails during deployment	Check the "Deployment History" in Webflow Cloud for detailed error logs. Ensure all environment variables are properly set. Also check "Runtime Logs" for any errors that stick out.

Other troubleshooting resources:

Check browser console for JavaScript errors
Verify Webflow API token has correct permissions
Ensure your Webflow site has published content

Contributing guidelines

Feel free to submit issues and enhancement requests!

For major changes:

Fork the repository
Create a feature branch
Make your changes
Test thoroughly
Submit a pull request to the original repo in Webflow-Examples with a clear description of the change

Additional documentation

For more information:

How to get help

Webflow-Examples/llms-txt-generator-webapp

About

Topics

Resources

Uh oh!

Stars

Watchers

Forks

Contributors 2

Uh oh!

Languages