$$\      $$\               $$$$$$$$\      $$\$$\ 
$$$\    $$$ |              \__$$  __|     $$ $$ |
$$$$\  $$$$ |$$$$$$\  $$$$$$$\$$ |$$$$$$\ $$ $$ |
$$\$$\$$ $$ |\____$$\$$  _____$$ $$  __$$\$$ $$ |
$$ \$$$  $$ |$$$$$$$ $$ /     $$ $$$$$$$$ $$ $$ |
$$ |\$  /$$ $$  __$$ $$ |     $$ $$   ____$$ $$ |
$$ | \_/ $$ \$$$$$$$ \$$$$$$$\$$ \$$$$$$$\$$ $$ |
\__|     \__|\_______|\_______\__|\_______\__\__|

MacTell: an LLM-Based Code Interpreter

Overview

This application, developed in Swift for macOS, interfaces with a Large Language Model (LLM) backend to translate natural language user inputs into executable CLI commands to streamline user interactions.

• Natural Language Processing: Translates user inputs into executable CLI commands.
• User Interaction Logging: Implements an interaction logging mechanism for personalized command processing.
• Swift-based GUI: Provides an intuitive graphical user interface, enhancing user experience on macOS systems.
• New Window: Temporarily stores active session data.
• Saved Window: Archives saved user interactions for re-running previous LLM-generated commands or responses.

Developer Docs

To review the Swift project structure, one can view index.html file. It shows the used classes, enums, structs and the respective methods.

System Structure:

• User is sending request, which goes to server.
• Next step is logging user input to DB + resending this input to LLM API
• After getting responce from LLM, if resulting program is "red", we ask user for permission to execute it.
• Last step: executing the program and sending result to chat with user.

Flow of Events:

• User Request: "Delete all files in the Downloads folder."

• Server Receives Request

• Logging to Database

• The server then forwards user's request to the LLM API for processing.

• LLM API Processes Request: generates a corresponding executable command.

• Let's assume the LLM responds with a command that is flagged as "red" due to its potential risk.

• User Permission for Execution: The server prompts user: "Accept?"

• User Confirmation

• Executing the Command

• Result Sent to User

• The command execution is complete and the outcome is logged in DB.

Built-in Functionality of MacTell 🚀

MacTell is equipped with a variety of built-in functions to streamline your macOS experience. Here's what you can do directly from the app:

🟢 Green Functions

Safe operations that enhance your productivity without needing additional confirmation.

• 🖥 Open MacOS Application: Opens a specified MacOS application such as 🎥 Facetime, 📓 Notes, 💬 Messages, 🌐 Safari, etc.
• 🕰 Tell Date and Time: Uses VoiceOver to announce the current date and time.
• ✉️ Compose Email: Prepares an email in the Mail app (sending requires user confirmation).
• 🎶 Music: Plays a track from the user's library in the Music app.
• 📝 Note Composing: Creates a note with provided content in the Notes app.
• 🔍 Google Search: Conducts a Google search and displays results in a new Safari tab.
• 📬 Tell Number of Unread Messages: Informs about unread messages count using VoiceOver.

🔴 Red Functions

Operations that can significantly alter system state or require careful execution, potentially needing user confirmation.

• 📞 Making a Call by Contact Name: Initiates a FaceTime call using a contact's name.
• 📞 Making a Call by Phone Number: Makes a FaceTime call to a given phone number.
• 👨‍💻 Run Generated Code: Executes code scripts generated by LLM (AppleScript, Shell, Python).
• 📆 Schedule Command: Sets up commands to run at specified intervals.
• 💬 Write a Message Using Contact Name: Sends a message to a contact via the Messages app.
• 💬 Write a Message Using Phone Number: Sends a message to a phone number via the Messages app.

Enjoy the convenience of interacting with your macOS with ease and efficiency! 💡

Installation and Execution Guide

1. Python Library Installation:

   To install all Python-specific libraries, refer to requirements.txt:

   $ pip install -r requirements.txt

2. Server Initialization:

   Start the server by running server.py:

   $ python ./SWIFT-interraction/server.py

3. Swift App Building and Execution:

   Build your Swift app located in ./MacTell and execute MacTellApp.swift.