LLM Visibility Optimization Tool

Introduction

The LLM Visibility Optimization Tool is a comprehensive system designed to enhance the visibility of websites in responses generated by Large Language Models (LLMs). It combines advanced scraping, embedding, and benchmarking techniques to provide actionable insights for improving content quality and relevance.

Key Features

1. LocalRAG (Retrieval-Augmented Generation)

• Purpose: Converts any website into a private, conversational AI knowledge base.
• How It Works:
  • Scrapes website content using Playwright and BeautifulSoup.
  • Converts HTML to Markdown for better readability.
  • Embeds content locally using ChromaDB and HuggingFace embeddings.
  • Allows users to query the content via a locally hosted LLM (e.g., Ollama).
• Tech Stack:
  • Backend: Python (Flask)
  • Frontend: HTML, TailwindCSS, JavaScript
  • Web Scraping: Playwright, BeautifulSoup4, markdownify
  • Vector Database: ChromaDB
  • Embeddings: all-MiniLM-L6-v2
  • LLM Runtime: Ollama
  • Orchestration: LangChain

2. GEO Benchmark System

• Purpose: Evaluates how well a website is optimized for visibility in LLM-generated responses.
• Key Dimensions:
  1. Relevance: Measures alignment with user queries.
  2. Authority: Assesses trustworthiness and credibility.
  3. Comprehensiveness: Evaluates the depth and breadth of content.
  4. Clarity: Analyzes content structure and readability.
  5. Recency: Checks for up-to-date information.
  6. Actionability: Looks for clear calls-to-action.
• Scoring System:
  • Weighted average of dimension scores.
  • Provides actionable recommendations based on scores.
• Tech Stack:
  • Python modules: LangChain, ChromaDB, HuggingFace, Ollama
  • Configuration: benchmark_config.py
  • Analysis Engine: geo_benchmark.py

Installation & Setup

Prerequisites

• Python 3.8 or higher
• pip

1. Install Ollama

ollama pull llama3

2. Clone the Repository

git clone https://github.com/sh4shv4t/LLM-Visibility-Optimization-Tool.git
cd LLM-Visibility-Optimization-Tool

3. Create Virtual Environment

Windows

python -m venv venv
.\venv\Scripts\activate

Mac/Linux

python3 -m venv venv
source venv/bin/activate

4. Install Dependencies

pip install -r requirements.txt

5. Install Playwright Drivers

playwright install

Usage

Start the Server

python app.py

Open:

http://127.0.0.1:5000

Step 1: Scrape & Index

• Enter a URL.
• Click Scrape & Index.
• Wait for success message.

Step 2: Analyze Content

• Click Analyze Quality to run the GEO Benchmark System.
• Review the detailed report with scores and recommendations.

Step 3: Ask a Question

• Type a question related to the scraped content.
• Click Ask.
• Answer is generated using retrieved context + local LLM.

Project Structure

LLM-Visibility-Optimization-Tool/
├── app.py
├── scraper.py
├── vector_store.py
├── qa_app.py
├── geo_benchmark.py
├── benchmark_config.py
├── templates/
│   └── index.html
├── scraped_content.md
├── vector_db/
├── requirements.txt
└── README.md

Future Improvements

• Multi-document ingestion.
• Chat history and follow-up question memory.
• Streaming model responses.
• Support for additional file formats (e.g., PDFs, text files).
• Visual dashboards for benchmarking results.

Support

For issues or questions:

1. Check the troubleshooting section in the GEO_BENCHMARK_GUIDE.md.
2. Open an issue in the GitHub repository.
3. Contact the repository owner for further assistance.