Synthetic-Data-Generation-by-AI-Agents

RailsCrew Crew

Welcome to the RailsCrew Crew project, powered by crewAI. This template is designed to help you set up a multi-agent AI system with ease, leveraging the powerful and flexible framework provided by crewAI. Our goal is to enable your agents to collaborate effectively on complex tasks, maximizing their collective intelligence and capabilities.

Installation

Ensure you have Python >=3.10 <3.13 installed on your system. This project uses UV for dependency management and package handling, offering a seamless setup and execution experience.

First, if you haven't already, install uv:

pip install uv

Next, navigate to your project directory and install the dependencies:

(Optional) Lock the dependencies and install them by using the CLI command:

crewai install

Customizing

Add your OPENAI_API_KEY into the .env file

• Modify src/rails_crew/config/agents.yaml to define your agents
• Modify src/rails_crew/config/tasks.yaml to define your tasks
• Modify src/rails_crew/crew.py to add your own logic, tools and specific args
• Modify src/rails_crew/main.py to add custom inputs for your agents and tasks

Running the Project

To kickstart your crew of AI agents and begin task execution, run this from the root folder of your project:

$ crewai run

This command initializes the rails_crew Crew, assembling the agents and assigning them tasks as defined in your configuration.

This example, unmodified, will run the create a report.md file with the output of a research on LLMs in the root folder.

Understanding Your Crew

The rails_crew Crew is composed of multiple AI agents, each with unique roles, goals, and tools. These agents collaborate on a series of tasks, defined in config/tasks.yaml, leveraging their collective skills to achieve complex objectives. The config/agents.yaml file outlines the capabilities and configurations of each agent in your crew.

Support

For support, questions, or feedback regarding the RailsCrew Crew or crewAI.

• Visit our documentation
• Reach out to us through our GitHub repository
• Join our Discord
• Chat with our docs

Let's create wonders together with the power and simplicity of crewAI.

Run the Crew

Copy the entire contents of <base_path>\rails_crew\src\rails_crew\crew_agent_executor.py to .venv\Lib\site-packages\crewai\agents\crew_agent_executor.py

Run crewai run

Collect information after running the crew

Logs generated

After running the crew, logs are saved as chat_logs.json, email_logs.json and meeting_logs.json.

Run python convert_json.py to convert the generated logs to proper format.

This generates logs are stored in the same directory with names chat_logs_array.json, email_logs_array.json and meeting_logs_array.json.

Generate Aggregated Data

Step 1: run python agg_chats.py, python agg_emails.py and python agg_meetings.py This generates aggregated_chats.json, aggregated_emails.json and aggregated_meetings.json files.

Step 2: run python aggregate.py This generates the final aggregated data aggregated_data.json

Generate Topic Rails data

Run create_topic_rails.py. This generates topic_rails.json file.

Generate weekly user activities data

Run weekly_activities.py. This generates weekly_activities.json file.

Generate File Authors data

Run python file_authors.py. Note, the authors need to be updated if we add any to the agents.yaml file.

Generate File Metadata

Run python files_metadata.py to generate files metadata.

Generate plots

Run python plots_2.py, python files_plot.py, python user-user-plot.py, python topics_plot.py, python .\plot_table.py

Authoring of files

The tasks are arranged in crew.py such that agents first author their own files and then refer them during collaboration.

Passing Knowledge base to agents via FileReader tool

In the provided crew.py file, knowledge sources are passed to agents through tools. Specifically, the FileReaderTool is used to read files from a specified directory and provide the content of those files to the agents. Here's how the knowledge source is passed to agents step by step:

1. Define the Knowledge Source Tool The FileReaderTool class is defined as a tool that reads files from a specified directory. It uses the os module to list and read files from the directory.

This tool is initialized with a directory path where the files (knowledge sources) are stored.

2. Initialize the Tool The FileReaderTool is initialized in the RailsCrew class with the directory path where the knowledge source files are located.

3. Pass the Tool to Agents The FileReaderTool is passed to agents as part of their tools parameter. Each agent is configured to use the tool and is also provided with a list of specific files (knowledge sources) it needs to access.

For example:

@agent def user_1(self) -> Agent: return Agent( config=self.agents_config['user_1'], verbose=True, tools=[self.file_reader_tool], # Tool passed to the agent files=["product_vision.txt", "product_roadmap.txt", "requirements.txt"], # Specific files for this agent get_output_json=True )

Here:

• tools=[self.file_reader_tool]: The FileReaderTool is passed to the agent.
• files=["product_vision.txt", "product_roadmap.txt", "requirements.txt"]: The specific files (knowledge sources) that the agent will use are specified.

4. Agent Uses the Tool When the agent is executed, it uses the FileReaderTool to read the specified files. The read_files method of the tool reads the content of the files and provides it to the agent.

5. Knowledge Source Integration The knowledge source (file content) is now available to the agent for processing. The agent can use this knowledge to perform its tasks, such as generating outputs, making decisions, or collaborating with other agents.