Skip to content

Interactive Model Selection, Custom Prompts, and Expanded Model Support #247

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 49 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
3ed4ae4
feat: update README and configuration for new model integrations and …
telmalah Jul 3, 2025
f2dbae0
feat: Add interactive model selection and custom system prompt feature
telmalah Jul 4, 2025
29e83a1
docs: Add screenshots for interactive model selection and custom syst…
telmalah Jul 4, 2025
901be5a
docs: Add image assets for README
telmalah Jul 4, 2025
169249d
add image
telmalah Jul 4, 2025
1c5236a
add immages
telmalah Jul 4, 2025
3f852f8
add
telmalah Jul 4, 2025
618deb5
update the images
telmalah Jul 4, 2025
65e689f
fix path
telmalah Jul 4, 2025
0c53c29
fix path
telmalah Jul 4, 2025
9a50036
fix path
telmalah Jul 4, 2025
ee85ed7
fix path
telmalah Jul 4, 2025
9dfcdbc
docs: Add release notes summary to README
telmalah Jul 4, 2025
544e84a
docs: Update README with correct pip uninstall command
telmalah Jul 4, 2025
4671d1b
Create python-publish.yml
malah-code Jul 4, 2025
3d1cf00
ci: Configure python-publish.yml for PyPI deployment
telmalah Jul 4, 2025
4fc4439
feat: add support for new OpenAI models and enhance Gemini coordinate…
telmalah Jul 4, 2025
2d249f9
remove image base URLs
telmalah Jul 6, 2025
fc26458
remove image base URLs
telmalah Jul 6, 2025
29748fc
ci: Include requirements.txt in sdist using MANIFEST.in
telmalah Jul 6, 2025
250232e
ci: Fix FileNotFoundError for requirements.txt in setup.py
telmalah Jul 6, 2025
2d8b56e
chore: Update version to 2.0.4 in setup.py
telmalah Jul 6, 2025
3b5fb9d
ci: Embed install_requires directly in setup.py to fix build error
telmalah Jul 6, 2025
84e808c
chore: Update version to 2.0.5 in setup.py
telmalah Jul 6, 2025
5028b38
chore: Update version to 2.0.6 in setup.py
telmalah Jul 6, 2025
89a3890
chore: Update version to 2.0.7 in setup.py
telmalah Jul 6, 2025
fd34a87
chore: Update version to 2.0.8 in setup.py
telmalah Jul 6, 2025
b68717b
chore: Update version to 2.0.9 in setup.py
telmalah Jul 7, 2025
14c7730
chore: Update version to 2.0.10 in setup.py
telmalah Jul 7, 2025
76c90ae
feat: Include all uncommitted changes for v2.0.11 release
telmalah Jul 7, 2025
3139141
Bump version to 2.0.12 for PyPI upload"
telmalah Jul 7, 2025
719178d
ci: Update python-publish.yml with latest changes
telmalah Jul 7, 2025
483536b
chore: Update version to 2.0.13 in setup.py
telmalah Jul 7, 2025
da00a8b
chore: Update version to 2.0.14 in setup.py
telmalah Jul 7, 2025
8134243
feat: release version 2.0.15
telmalah Jul 7, 2025
1ac0fdb
docs: Add guide for adding new models to README.md
telmalah Jul 7, 2025
08a9e08
feat: Add OpenRouter support with dynamic model fetching and environm…
telmalah Jul 7, 2025
1047a69
feat: Simplify OpenRouter model selection to manual input
telmalah Jul 7, 2025
0c3a901
feat: Implement interactive selection of free OpenRouter vision models
telmalah Jul 8, 2025
376c2ed
fix: Improve OpenRouter model detection in get_next_action
telmalah Jul 8, 2025
a340578
feat: Implement internal prefix for OpenRouter models
telmalah Jul 8, 2025
7e031fa
fix: Revert OpenRouter model selection to manual input and update README
telmalah Jul 8, 2025
80c41c7
fix: Handle None content from OpenRouter model to prevent NoneType error
telmalah Jul 8, 2025
630b08a
fix: Robustify OpenRouter API response handling in call_openrouter_model
telmalah Jul 8, 2025
c0c501b
feat: Improve OpenRouter model input screen with input_dialog
telmalah Jul 8, 2025
d8d5a2f
docs: Update README with latest OpenRouter changes and bug fixes
telmalah Jul 8, 2025
f899e5a
docs: Update README version to v2.0.16
telmalah Jul 8, 2025
ee6f899
chore: Remove obsolete open-router-selection-2.png image file
telmalah Jul 8, 2025
99f06f9
Bump version to 2.0.16
telmalah Jul 8, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .env-sample
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
OPENAI_API_KEY=YOUR_OPENAI_API_KEY
GOOGLE_API_KEY=YOUR_GOOGLE_API_KEY
OLLAMA_HOST='http://localhost:11434/'
CUSTOM_SYSTEM_PROMPT='You are a helpful assistant that can perform tasks on a computer. You can execute commands, answer questions, and assist with various tasks. Your goal is to help the user efficiently and effectively.'
OPENROUTER_API_KEY='sddfsdf'
OPENROUTER_MODEL="google/gemini-2.0-flash-exp:free"
48 changes: 48 additions & 0 deletions .github/workflows/python-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: Publish Python Package

on:
release:
types: [published]

permissions:
contents: read
id-token: write # Required for PyPI trusted publishing

jobs:
publish:
runs-on: ubuntu-latest
environment:
name: pypi
url: https://pypi.org/p/self-ai-operating-computer

steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.x'

- name: Install build tools
run: python -m pip install build setuptools wheel

- name: Build package
run: python -m build

- name: Check if version exists on PyPI
id: check-version
run: |
VERSION=$(python setup.py --version)
if curl -s "https://pypi.org/pypi/self-ai-operating-computer/$VERSION/json" | grep -q "Not Found"; then
echo "version_exists=false" >> $GITHUB_OUTPUT
else
echo "version_exists=true" >> $GITHUB_OUTPUT
echo "Version $VERSION already exists on PyPI. Skipping upload."
exit 1
fi

- name: Publish to PyPI
if: steps.check-version.outputs.version_exists == 'false'
uses: pypa/gh-action-pypi-publish@release/v1
with:
packages-dir: dist/
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -162,5 +162,6 @@ cython_debug/
.DS_Store

# Avoid sending testing screenshots up
*.png
# *.png
operate/screenshots/
screenshots/*
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@ It is recommended that a screenshot of the `evaluate.py` output is included in a
- **Improve the `SUMMARY_PROMPT`**
- **Improve Linux and Windows compatibility**: There are still some issues with Linux and Windows compatibility. PRs to fix the issues are encouraged.
- **Adding New Multimodal Models**: Integration of new multimodal models is welcomed. If you have a specific model in mind that you believe would be a valuable addition, please feel free to integrate it and submit a PR.
- **Iterate `--accurate` flag functionality**: Look at https://github.com/OthersideAI/self-operating-computer/pull/57 for previous iteration
- **Enhanced Security**: A feature request to implement a _robust security feature_ that prompts users for _confirmation before executing potentially harmful actions_. This feature aims to _prevent unintended actions_ and _safeguard user data_ as mentioned here in this [OtherSide#25](https://github.com/OthersideAI/self-operating-computer/issues/25)
- **Iterate `--accurate` flag functionality**: Look at https://github.com/malah-code/self-ai-operating-computer/pull/57 for previous iteration
- **Enhanced Security**: A feature request to implement a _robust security feature_ that prompts users for _confirmation before executing potentially harmful actions_. This feature aims to _prevent unintended actions_ and _safeguard user data_ as mentioned here in this [OtherSide#25](https://github.com/malah-code/self-ai-operating-computer/issues/25)


## Guidelines
Expand Down
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
MIT License

Copyright (c) 2023 OthersideAI
Copyright (c) 2023 malah-code

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Expand Down
1 change: 1 addition & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
include requirements.txt
Loading