Skip to content

Add OpenAI API Support#8

Open
ib-bsb-br wants to merge 33 commits intoPerminovEugene:mainfrom
ib-bsb-br:main
Open

Add OpenAI API Support#8
ib-bsb-br wants to merge 33 commits intoPerminovEugene:mainfrom
ib-bsb-br:main

Conversation

@ib-bsb-br
Copy link

This pull request introduces two major updates to the messy-folder-reorganizer-ai tool:

OpenAI API Integration: Provides users with the option to use OpenAI's models for language processing (folder naming) and generating embeddings. This serves as an alternative to the existing local Ollama setup, offering greater flexibility and access to different AI capabilities.

Release Workflow Refinement: Streamlines the GitHub Actions release workflow. These changes aim to improve maintainability, simplify the release process by focusing on Linux builds, and ensure correct binary naming for release artifacts.

🚀 OpenAI API Integration
This enhancement allows users to choose their preferred AI provider, either local via Ollama or remote via the OpenAI API.

Key Changes & Features:

New CLI Argument --ai-provider:

Accepts local (default, for Ollama) or openai.

OpenAI Specific CLI Arguments:

--openai-api-key: Your OpenAI API key. This can also be set via the OPENAI_API_KEY environment variable.

--openai-llm-model: Specifies the OpenAI model for folder name generation (default: gpt-4o-mini).

--openai-embedding-model: Specifies the OpenAI model for embeddings (default: text-embedding-ada-002).

--openai-api-base: Optional custom base URL for OpenAI-compatible APIs (default: https://api.openai.com/v1).

--openai-temperature: Optional sampling temperature for OpenAI LLM.

--openai-max-tokens: Optional max completion tokens for OpenAI LLM.

--openai-embedding-dimensions: Optional output dimensions for newer OpenAI embedding models.

Ollama Specific CLI Arguments (Usage Clarified with --ai-provider):

--language-model (or -L): Specifies the Ollama LLM model name. This argument is required when --ai-provider is set to local.

--embedding-model (or -E): Specifies the Ollama embedding model name. This argument is required when --ai-provider is set to local.

--ollama-server-address (or -n): Specifies the Ollama server address (default: http://localhost:11434), used when --ai-provider is local.

Core Logic Updates:

Introduced src/ai/openai_client.rs and src/ai/openai_protocol.rs to manage interactions with the OpenAI API.

Refactored src/ai/embeddings_request.rs (function get_embeddings to get_ai_embeddings) and src/ai/llm_request.rs (function ask_ai_for_reordering_plan to get_ai_reordering_plan) to dynamically call the appropriate AI provider based on user selection.

Updated src/configuration/args.rs to include the new AiProvider enum and all associated arguments.

Enhanced src/errors/app_error.rs with specific error types for OpenAI (e.g., OpenAIAPIError, OpenAIAuthError) and for missing Ollama model configurations when local provider is selected.

Modified src/commands/process.rs to validate required arguments based on the selected AI provider.

Documentation (README.md):

The main description now highlights the dual AI provider capability.

A new "AI Provider Configuration" section has been added, providing detailed instructions and examples for setting up and using both Ollama and OpenAI.

The "Command-Line Arguments" table has been comprehensively updated to reflect all new and modified arguments.

⚙️ Release Workflow Refinement (.github/workflows/release.yml)
This section details refinements to the GitHub Actions workflow used for creating releases. These changes focus on enhancing maintainability, simplifying the process by targeting Linux builds, and correcting binary naming conventions.

Key Changes:

Targeted Linux Build: The workflow now exclusively builds on ubuntu-latest, removing the previous multi-OS build matrix (which included macOS variants). This simplifies the release process and focuses efforts on a primary platform.

Binary Naming Correction: The output binary name in the release archive is now consistently messy_folder_reorganizer_ai (using underscores), aligning with the fix in commit Fix(workflow): Update binary name to use underscore and resolving previous inconsistencies.

Toolchain Update: Switched from dtolnay/rust-toolchain@stable to actions-rs/toolchain@v1 for Rust environment setup.

Simplified Build Command: The cargo build --release command no longer specifies a --target, relying on the native target of the ubuntu-latest runner.

Archive Naming: The generated .tar.gz file is now named messy-folder-reorganizer-ai-${{ env.VERSION }}-x86_64-unknown-linux-gnu.tar.gz, explicitly indicating the Linux target.

Tag Trigger: The workflow is triggered on pushes to tags matching the pattern v* (e.g., v0.1.0).

Affected Files:
.github/workflows/release.yml

README.md

src/ai/mod.rs

src/ai/embeddings_request.rs

src/ai/llm_request.rs

src/ai/openai_client.rs (New)

src/ai/openai_protocol.rs (New)

src/app_core/cluster_processor.rs

src/app_core/destination_processor.rs

src/app_core/sources_processor.rs

src/commands/process.rs

src/configuration/args.rs

src/errors/app_error.rs

devin-ai-integration bot and others added 30 commits May 18, 2025 22:09
Co-Authored-By: github.com/ib-bsb-br/ib-bsb-br.github.io <marioseixascardosoneto@gmail.com>
…gration

Add OpenAI API integration as an alternative to Ollama
Adjusts the GitHub Actions release workflow to look for
`messy_folder_reorganizer_ai` (with an underscore) instead of
`messy-folder-reorganizer-ai` (with a hyphen).

This change is based on troubleshooting steps for a "binary not found"
error during the release build, aligning the workflow with a potential
naming discrepancy where the compiled binary name might directly reflect
the `Cargo.toml` package name.
Fix(workflow): Update binary name to use underscore
Modifies the GitHub Actions release workflow to support building
the arm64 version of the application linked against OpenSSL 1.1.1w.

Changes include:
- Added steps to download and compile OpenSSL 1.1.1w from source
  on the ubuntu-24.04-arm runner.
- Set appropriate environment variables (OPENSSL_DIR, OPENSSL_STATIC, PKG_CONFIG_PATH)
  to ensure Cargo and the openssl-sys crate use the custom-built OpenSSL.
- Corrected the output binary artifact name in the packaging and upload
  steps to reflect the aarch64 architecture (aarch64-unknown-linux-gnu).
- Configured OpenSSL build for static linking of libcrypto and libssl.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This commit applies a series of improvements to the release build workflow:

- Consolidates package installations into a single step with one `apt-get update`.
- Simplifies OpenSSL environment variable handling by relying on GITHUB_ENV.
- Adds caching for Cargo dependencies and the build target directory to speed up builds.
- Makes the `cargo build` target explicit (`aarch64-unknown-linux-gnu`) for clarity and updates relevant paths.
- Switches to using the standard `secrets.GITHUB_TOKEN` for release uploads and ensures appropriate job-level permissions (`contents: write`).
- Adds and refines comments throughout the workflow for better understanding and maintainability.
- Updates the commented-out manual OpenSSL compilation step to suggest `linux-aarch64` for the configure script.

These changes address the points raised in the critical review and enhancement strategy, making the workflow more efficient, robust, and aligned with best practices, while maintaining the core GLIBC 2.31 compatibility.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Refine GitHub Actions workflow for GLIBC compatibility
@PerminovEugene
Copy link
Owner

Hey! Thank you for pr, it seems like a good stuff. I am currently on vacation with no laptop. I'll check pr in the beginning of June. Sorry for delay

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants