MANTA (Multi-lingual Advanced NMF-based Topic Analysis)

A comprehensive topic modeling system using Non-negative Matrix Factorization (NMF) and Non-negative Matrix Tri-Factorization (NMTF) that supports both English and Turkish text processing. Features advanced tokenization techniques, multiple factorization algorithms including NMTF for topic relationship analysis, and rich visualization capabilities.

Quick Start

Installing locally for Development

To build and run the app locally for development: First clone the repository:

git clone https://github.com/emirkyz/manta.git

After cloning, navigate to the project directory and create a virtual environment:

cd manta
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

Next, install the required dependencies. If you have pip installed, you can run:

pip install -e .

or if you have uv installed, you can use:

uv pip install -e .

Installation from PyPI

pip install manta-topic-modelling

After that you can import and use the app.

Python API Usage

from manta import run_topic_analysis

# Simple topic modeling
results = run_topic_analysis(
    filepath="data.csv",
    column="review_text",
    language="EN",
    topic_count=5,
    lemmatize=True
)

# Turkish text analysis
results = run_topic_analysis(
    filepath="turkish_reviews.csv", 
    column="yorum_metni",
    language="TR",
    topic_count=8,
    tokenizer_type="bpe",
    generate_wordclouds=True
)

# NMTF analysis for topic relationship discovery
results = run_topic_analysis(
    filepath="data.csv",
    column="text_content",
    language="TR", 
    topic_count=6,
    nmf_method="nmtf",
    generate_wordclouds=True
)

Result Structure

{
"state": State of the analysis, either "success" or "error",
"message": Message about the result of the analysis,
"data_name": Name of the input data file,
"topic_word_scores": JSON object containing topics and their top words with scores,
"topic_doc_scores": JSON object containing topics and their top documents with scores,
"coherence_scores": JSON object containing coherence scores for each topic,
"topic_dist_img": Matplotlib plt object of topic distribution plot if `gen_topic_distribution` is True,
"topic_document_counts": Count of documents per topic,
"topic_relationships": Topic-to-topic relationship matrix (only for NMTF method),
}

For example:
{
  "state": "success",
  "message": "Analysis completed successfully",
  "data_name": "reviews.csv",
  "topic_word_scores": {
    "topic_0": {
        "word1": 0.15,
        "word2": 0.12,
        "word3": 0.10
        }
    },
  "topic_doc_scores":{
          "topic_0": [
                {
                    "document": "Sample document text...",
                    "score": 0.78
                }
            ],
    }
  "coherence_scores": {
        "gensim": {
           "umass_average": -1.4328882390292266,
            "umass_per_topic": {
                "topic_0": -1.4328882390292266,
                "topic_1": -1.1234567890123456,
                "topic_2": -0.9876543210987654
                }
        }
    },
  "topic_dist_img": "<matplotlib plot object>",
  "topic_document_counts": [____]
}

Command Line Usage

# Turkish text analysis
manta-topic-modelling analyze data.csv --column text --language TR --topics 5

# English text analysis with lemmatization and visualizations
manta-topic-modelling analyze data.csv --column content --language EN --topics 10 --lemmatize --wordclouds --excel

# Custom tokenizer for Turkish text
manta-topic-modelling analyze reviews.csv --column review_text --language TR --topics 8 --tokenizer bpe --wordclouds

# NMTF analysis for topic relationship discovery
manta-topic-modelling analyze data.csv --column text --language TR --topics 5 --nmf-method nmtf

# Filter by app name and country
manta-topic-modelling analyze reviews.csv --column REVIEW --language TR --topics 5 --filter-app MyApp --filter-country TR

# Custom filtering columns
manta-topic-modelling analyze data.csv --column text --language TR --topics 5 --filter-app-column APP_ID --filter-country-column REGION

# Disable emoji processing for faster processing
manta-topic-modelling analyze data.csv --column text --language EN --topics 5 --emoji-map False

Package Structure

manta/
├── _functions/
│   ├── common_language/          # Shared functionality across languages
│   │   ├── emoji_processor.py    # Emoji handling utilities
│   │   └── topic_extractor.py    # Cross-language topic analysis and extraction
│   ├── english/                  # English text processing modules
│   │   ├── english_entry.py             # English text processing entry point
│   │   ├── english_preprocessor.py      # Text cleaning and preprocessing
│   │   ├── english_vocabulary.py        # Vocabulary creation
│   │   ├── english_text_encoder.py      # Text-to-numerical conversion
│   │   ├── english_topic_analyzer.py    # Topic extraction utilities
│   │   ├── english_topic_output.py      # Topic visualization and output
│   │   └── english_nmf_core.py          # NMF implementation for English
│   ├── nmf/                      # NMF algorithm implementations
│   │   ├── nmf_orchestrator.py          # Main NMF interface
│   │   ├── nmf_initialization.py        # Matrix initialization strategies
│   │   ├── nmf_basic.py                 # Standard NMF algorithm
│   │   ├── nmf_projective_basic.py      # Basic projective NMF
│   │   ├── nmf_projective_enhanced.py   # Enhanced projective NMF
│   │   └── nmtf/                        # Non-negative Matrix Tri-Factorization
│   │       ├── nmtf.py                  # NMTF implementation with topic relationships
│   │       ├── nmtf_init.py             # NMTF initialization utilities
│   │       ├── nmtf_util.py             # NMTF helper functions
│   │       ├── extract_nmtf_topics.py   # Topic extraction for NMTF results
│   │       └── example_usage.py         # NMTF usage examples
│   ├── tfidf/                    # TF-IDF calculation modules
│   │   ├── tfidf_english_calculator.py  # English TF-IDF implementation
│   │   ├── tfidf_turkish_calculator.py  # Turkish TF-IDF implementation
│   │   ├── tfidf_tf_functions.py        # Term frequency functions
│   │   ├── tfidf_idf_functions.py       # Inverse document frequency functions
│   │   └── tfidf_bm25_turkish.py        # BM25 implementation for Turkish
│   └── turkish/                  # Turkish text processing modules
│       ├── turkish_entry.py             # Turkish text processing entry point
│       ├── turkish_preprocessor.py      # Turkish text cleaning
│       ├── turkish_tokenizer_factory.py # Tokenizer creation and training
│       ├── turkish_text_encoder.py      # Text-to-numerical conversion
│       └── turkish_tfidf_generator.py   # TF-IDF matrix generation
├── utils/                        # Helper utilities (organized into sub-modules)
│   ├── analysis/                       # Analysis utilities
│   │   ├── coherence_score.py              # Topic coherence evaluation
│   │   ├── distance_two_words.py           # Word distance calculation
│   │   ├── umass_test.py                   # UMass coherence testing
│   │   ├── word_cooccurrence.py            # Word co-occurrence analysis
│   │   └── word_cooccurrence_analyzer.py   # Advanced word co-occurrence analysis
│   ├── console/                        # Console management
│   │   └── console_manager.py              # Console and logging management utilities
│   ├── database/                       # Database utilities
│   │   ├── database_manager.py             # Database connection and management utilities
│   │   └── save_topics_db.py               # Topic database saving utilities
│   ├── export/                         # Export functionality
│   │   ├── export_excel.py                 # Excel export functionality
│   │   ├── json_to_excel.py                # JSON to Excel conversion utilities
│   │   ├── save_doc_score_pair.py          # Document-score pair saving utilities
│   │   └── save_word_score_pair.py         # Word-score pair saving utilities
│   ├── preprocess/                     # Preprocessing utilities
│   │   └── combine_number_suffix.py         # Number and suffix combination utilities
│   ├── visualization/                  # Visualization utilities
│   │   ├── gen_cloud.py                    # Word cloud generation
│   │   ├── image_to_base.py                # Image to base64 conversion
│   │   ├── topic_dist.py                   # Topic distribution plotting
│   │   └── visualizer.py                   # General visualization utilities
│   └── agent/                          # AI assistant utilities
│       ├── claude_prompt_generator.py       # Claude AI prompt generation utilities
│       └── claude_prompt_generator.html     # HTML interface for prompt generation
├── cli.py                        # Command-line interface
├── standalone_nmf.py             # Core NMF implementation
└── __init__.py                   # Package initialization and public API

Installation

From PyPI (Recommended)

pip install manta-topic-modelling

From Source (Development)

Clone the repository:

git clone https://github.com/emirkyz/manta.git
cd manta

Create a virtual environment:

python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

Install dependencies:

pip install -r requirements.txt

Usage

Command Line Interface

The package provides the manta-topic-modelling command with an analyze subcommand:

# Basic usage
manta-topic-modelling analyze data.csv --column text --language TR --topics 5

# Advanced usage with all options
manta-topic-modelling analyze reviews.csv \
  --column review_text \
  --language EN \
  --topics 10 \
  --words-per-topic 20 \
  --nmf-method pnmf \
  --lemmatize \
  --wordclouds \
  --excel \
  --topic-distribution \
  --output-name my_analysis

Command Line Options

Required Arguments:

filepath: Path to input CSV or Excel file
--column, -c: Name of column containing text data
--language, -l: Language ("TR" for Turkish, "EN" for English)

Optional Arguments:

--topics, -t: Number of topics to extract (default: 5)
--output-name, -o: Custom name for output files (default: auto-generated)
--tokenizer: Tokenizer type for Turkish ("bpe" or "wordpiece", default: "bpe")
--nmf-method: Factorization algorithm ("nmf", "pnmf", or "nmtf", default: "nmf")
--words-per-topic: Number of top words per topic (default: 15)
--lemmatize: Apply lemmatization for English text
--emoji-map: Enable emoji processing and mapping (default: True). Use --emoji-map False to disable
--wordclouds: Generate word cloud visualizations
--excel: Export results to Excel format
--topic-distribution: Generate topic distribution plots
--separator: CSV separator character (default: "|")
--filter-app: Filter data by specific app name
--filter-app-column: Column name for app filtering (default: "PACKAGE_NAME")
--filter-country: Filter data by country code (e.g., TR, US, GB)
--filter-country-column: Column name for country filtering (default: "COUNTRY")

Python API

from manta import run_topic_analysis

# Basic English text analysis
results = run_topic_analysis(
    filepath="data.csv",
    column="review_text",
    language="EN",
    topic_count=5,
    lemmatize=True,
    generate_wordclouds=True,
    export_excel=True
)

# Advanced Turkish text analysis with filtering
results = run_topic_analysis(
    filepath="turkish_reviews.csv",
    column="yorum_metni",
    language="TR",
    topic_count=10,
    words_per_topic=15,
    tokenizer_type="bpe",
    nmf_method="nmf",
    generate_wordclouds=True,
    export_excel=True,
    topic_distribution=True,
    filter_app=True,
    data_filter_options={
        "filter_app_name": "MyApp",
        "filter_app_column": "APP_NAME", 
        "filter_app_country": "TR",
        "filter_app_country_column": "COUNTRY_CODE"
    }
)

API Parameters

Required:

filepath (str): Path to input CSV or Excel file
column (str): Name of column containing text data

Optional:

separator (str): CSV separator character (default: ",")
language (str): "TR" for Turkish, "EN" for English (default: "EN")
topic_count (int): Number of topics to extract (default: 5)
nmf_method (str): "nmf", "pnmf", or "nmtf" algorithm variant (default: "nmf")
lemmatize (bool): Apply lemmatization for English (default: False)
tokenizer_type (str): "bpe" or "wordpiece" for Turkish (default: "bpe")
words_per_topic (int): Top words to show per topic (default: 15)
word_pairs_out (bool): Create word pairs output (default: True)
generate_wordclouds (bool): Create word cloud visualizations (default: True)
export_excel (bool): Export results to Excel (default: True)
topic_distribution (bool): Generate distribution plots (default: True)
filter_app (bool): Enable app filtering (default: False)
data_filter_options (dict): Advanced filtering options with keys (all default to empty string):
- filter_app_name (str): App name for filtering
- filter_app_column (str): Column name for app filtering (default: "PACKAGE_NAME")
- filter_app_country (str): Country code for filtering (case-insensitive)
- filter_app_country_column (str): Column name for country filtering (default: "COUNTRY")
emoji_map (bool): Enable emoji processing and mapping (default: False)
output_name (str): Custom output directory name (default: auto-generated)
save_to_db (bool): Whether to persist data to database (default: False)
output_dir (str): Base directory for outputs (default: current working directory)

Outputs

The analysis generates several outputs in an Output/ directory (created at runtime), organized in a subdirectory named after your analysis:

Topic-Word Excel File: .xlsx file containing top words for each topic and their scores
Word Clouds: PNG images of word clouds for each topic (if generate_wordclouds=True)
Topic Distribution Plot: Plot showing distribution of documents across topics (if topic_distribution=True)
Coherence Scores: JSON file with coherence scores for the topics
Top Documents: JSON file listing most representative documents for each topic

Features

Multi-language Support: Optimized processing for both Turkish and English texts
Advanced Tokenization: BPE and WordPiece tokenizers for Turkish, traditional tokenization for English
Multiple Factorization Algorithms: Standard NMF, Orthogonal Projective NMF (PNMF), and Non-negative Matrix Tri-Factorization (NMTF)
Advanced NMF Variants: Hierarchical NMF, Online NMF, and Symmetric NMF implementations
Rich Visualizations: Word clouds and topic distribution plots
Flexible Export: Excel and JSON export formats with organized export utilities
Coherence Evaluation: Built-in topic coherence scoring and advanced analysis tools
Database Management: Comprehensive SQLite database integration with dedicated management utilities
Modular Architecture: Organized utility modules for analysis, visualization, export, and preprocessing
Text Preprocessing: Language-specific text cleaning and preprocessing

Requirements

Python 3.9+
Dependencies are automatically installed with the package

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Support

For issues and questions, please open an issue on the GitHub repository

Name		Name	Last commit message	Last commit date
Latest commit History 144 Commits
manta		manta
tests		tests
.gitignore		.gitignore
.python-version		.python-version
CHANGELOG.md		CHANGELOG.md
DOCUMENTATION.md		DOCUMENTATION.md
DOCUMENTATION_TR.md		DOCUMENTATION_TR.md
LICENSE		LICENSE
README.md		README.md
README_tr.md		README_tr.md
pyproject.toml		pyproject.toml
requirements.txt		requirements.txt
uv.lock		uv.lock

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Repository files navigation

MANTA (Multi-lingual Advanced NMF-based Topic Analysis)

Quick Start

Installing locally for Development

Installation from PyPI

Python API Usage

Result Structure

Command Line Usage

Package Structure

Installation

From PyPI (Recommended)

From Source (Development)

Usage

Command Line Interface

Command Line Options

Python API

API Parameters

Outputs

Features

Requirements

License

Contributing

Support

About

Uh oh!

Releases

Packages

Languages

License

ElsevierSoftwareX/SOFTX-D-25-00535

Folders and files

Latest commit

History

Repository files navigation

MANTA (Multi-lingual Advanced NMF-based Topic Analysis)

Quick Start

Installing locally for Development

Installation from PyPI

Python API Usage

Result Structure

Command Line Usage

Package Structure

Installation

From PyPI (Recommended)

From Source (Development)

Usage

Command Line Interface

Command Line Options

Python API

API Parameters

Outputs

Features

Requirements

License

Contributing

Support

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages