feat / Improved Order Polling, Enhanced Connector Configuration, and Rate Oracle API #106

cardosofede · 2026-01-07T23:35:01Z

This PR enhances the order management system by implementing automatic synchronization between the connector's in-flight orders and the database. A new polling mechanism runs every minute to sync order states, ensuring the database accurately reflects order status as reported by the exchange (OPEN, FILLED, CANCELLED, etc.). Terminal orders (filled, cancelled, failed) are automatically cleaned up from the connector's in-flight orders to prevent memory buildup and stale data.

A new Rate Oracle router has been added to expose price feed configuration and rate querying capabilities via REST API. The /rate-oracle endpoints allow clients to view and update the rate oracle source (binance, kucoin, etc.) and global token settings, with changes persisted to conf_client.yml and applied to the running instance immediately. Additional endpoints provide access to cached prices, single pair rates, and async rate fetching directly from the exchange source.

The connector configuration endpoint has been enhanced to return richer metadata about each field including type information, required status, and allowed values for Literal types. A critical bug fix also addresses an issue where adding new connector credentials would fail with 'NoneType' object has no attribute 'encrypt_secret_value' - this was caused by Python's class attribute inheritance where the secrets manager wasn't being set on the parent Security class that hummingbot's internal methods access directly.

QA Test Plan

Adding Connector Credentials - Add a new connector (e.g., binance) via POST /accounts/add-credential/{account}/binance and verify it succeeds without errors
Config Map Endpoint - Call GET /connectors/binance/config-map and verify the response includes type, required, and allowed_values fields
Order Status Sync - Place an order, wait 1-2 minutes, then verify the order status in the database matches the exchange status
Order Cleanup - Place and cancel an order, verify it's removed from active orders after the next sync cycle
API Restart Recovery - Place an order, restart the API, verify the order is reloaded from the database and continues to be tracked
Rate Oracle Sources - Call GET /rate-oracle/sources and verify list of available sources is returned
Rate Oracle Config - Call GET /rate-oracle/config and verify current configuration is returned
Update Rate Oracle - Call PUT /rate-oracle/config to change the source (e.g., to kucoin), verify it persists and the running instance updates
Get Rates - Call POST /rate-oracle/rates with trading pairs like ["BTC-USDT", "ETH-USDT"] and verify rates are returned
Get Single Rate - Call GET /rate-oracle/rate/BTC-USDT and verify the rate is returned
Cached Prices - Call GET /rate-oracle/prices and verify all cached prices are returned

rapcmia · 2026-01-09T09:23:14Z

Commit 36c87a6

build local docker images

make setup ✅

both source and docker does not start due to error ModuleNotFoundError: No module named 'models.rate_oracle' ❌

###$ make run
docker compose up emqx postgres -d
WARN[0000] No services to build
[+] up 2/2
 ✔ Container hummingbot-postgres Running                                                                                                                                                                                                  0.0s
 ✔ Container hummingbot-broker   Running                                                                                                                                                                                                  0.0s
conda run --no-capture-output -n hummingbot-api uvicorn main:app --reload
INFO:     Will watch for changes in these directories: ['/home/yawnyunehh/hummingbot/hummingbot-api/106']
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [2881456] using WatchFiles
Process SpawnProcess-1:
Traceback (most recent call last):
  File "/home/yawnyunehh/anaconda3/envs/hummingbot-api/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/home/yawnyunehh/anaconda3/envs/hummingbot-api/lib/python3.12/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/yawnyunehh/anaconda3/envs/hummingbot-api/lib/python3.12/site-packages/uvicorn/_subprocess.py", line 80, in subprocess_started
    target(sockets=sockets)
  File "/home/yawnyunehh/anaconda3/envs/hummingbot-api/lib/python3.12/site-packages/uvicorn/server.py", line 67, in run
    return asyncio_run(self.serve(sockets=sockets), loop_factory=self.config.get_loop_factory())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yawnyunehh/anaconda3/envs/hummingbot-api/lib/python3.12/asyncio/runners.py", line 195, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/home/yawnyunehh/anaconda3/envs/hummingbot-api/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
  File "/home/yawnyunehh/anaconda3/envs/hummingbot-api/lib/python3.12/site-packages/uvicorn/server.py", line 71, in serve
    await self._serve(sockets)
  File "/home/yawnyunehh/anaconda3/envs/hummingbot-api/lib/python3.12/site-packages/uvicorn/server.py", line 78, in _serve
    config.load()
  File "/home/yawnyunehh/anaconda3/envs/hummingbot-api/lib/python3.12/site-packages/uvicorn/config.py", line 439, in load
    self.loaded_app = import_from_string(self.app)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yawnyunehh/anaconda3/envs/hummingbot-api/lib/python3.12/site-packages/uvicorn/importer.py", line 22, in import_from_string
    raise exc from None
  File "/home/yawnyunehh/anaconda3/envs/hummingbot-api/lib/python3.12/site-packages/uvicorn/importer.py", line 19, in import_from_string
    module = importlib.import_module(module_str)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yawnyunehh/anaconda3/envs/hummingbot-api/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 999, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/home/yawnyunehh/hummingbot/hummingbot-api/106/main.py", line 38, in <module>
    from services.bots_orchestrator import BotsOrchestrator
  File "/home/yawnyunehh/hummingbot/hummingbot-api/106/services/__init__.py", line 3, in <module>
    from .docker_service import DockerService
  File "/home/yawnyunehh/hummingbot/hummingbot-api/106/services/docker_service.py", line 16, in <module>
    from models import V2ScriptDeployment
  File "/home/yawnyunehh/hummingbot/hummingbot-api/106/models/__init__.py", line 195, in <module>
    from .rate_oracle import (
ModuleNotFoundError: No module named 'models.rate_oracle'

rapcmia · 2026-01-09T13:48:28Z

Commit dae36a2

Adding Connector Credentials, hyperliquid_perpetual, binance and kucoin ✅
Config Map Endpoint - Review each of connectors config map used ✅
Order Status Sync - Place an order, wait 1-2 minutes, then verify the order status in the database matches the exchange status ✅
Order Cleanup - Place and cancel an order, verify it's removed from active orders after the next sync cycle ✅
- Successfully cancelled order and use POST trading/orders/active to check status
- data is empty, no active orders fetched
API Restart Recovery - Place an order, restart the API, verify the order is reloaded from the database and continues to be tracked ✅
- Placed order POST trading/orders
- Check status POST trading/orders/active, then stop hummingbot-api
- Start hummingbot-api again (after 60s), check order status, ok
- Check order again using POST trading/order/search (after 30s), ok
- Check DB if any possible duplication of data, all ok

Test rate-oracle routes

Call GET /rate-oracle/sources and verify list of available sources is returned ✅

[
  "binance",
  "coin_gecko",
  "coin_cap",
  "kucoin",
  "ascend_ex",
  "gate_io",
  "coinbase_advanced_trade",
  "cube",
  "dexalot",
  "hyperliquid",
  "derive",
  "mexc"
]

Call GET /rate-oracle/config and verify current configuration is returned ✅

  "rate_oracle_source": {
    "name": "binance"
  },
  "global_token": {
    "global_token_name": "USDT",
    "global_token_symbol": "$"
  },
  "available_sources": [
    "binance",
    "coin_gecko",
    "coin_cap",
    "kucoin",
    "ascend_ex",
    "gate_io",
    "coinbase_advanced_trade",
    "cube",
    "dexalot",
    "hyperliquid",
    "derive",
    "mexc"
  ]

Call PUT /rate-oracle/config to change the source (e.g., to kucoin), verify it persists and the running instance updates

  "success": true,
  "message": "rate_oracle_source updated to hyperliquid",
  "config": {
    "rate_oracle_source": {
      "name": "hyperliquid"
    },
    "global_token": {
      "global_token_name": "USDT",
      "global_token_symbol": "$"
    }
  }

Call POST /rate-oracle/rates with trading pairs like ["BTC-USDT", "ETH-USDT"] and verify rates are returned ✅

{
  "source": "binance",
  "quote_token": "USDT",
  "rates": {
    "HYPE-USDC": null,
    "HYPE-USD": null,
    "UBTC-USDC": null,
    "SOL-USDT": 138.635
  }
}

Call GET /rate-oracle/rate/BTC-USDT and verify the rate is returned ✅
Call GET /rate-oracle/prices and verify all cached prices are returned ✅
- Displays the list of token pairs which cached prices
- Test with specific token pairs, all ok
- Test with interval of 30s for 3/3, prices are updated

Test rate-oracle/config with invalid source ❗

curl -s -u admin:admin -X PUT "http://localhost:8000/rate-oracle/config" -H "Content-Type: application/json" -d '{"rate_oracle_source":{"name":"ralph-handsome"}}' | jq
{
  "detail": [
    {
      "type": "enum",
      "loc": [
        "body",
        "rate_oracle_source",
        "name"
      ],
      "msg": "Input should be 'binance', 'binance_us', 'coin_gecko', 'coin_cap', 'kucoin', 'ascend_ex', 'gate_io', 'coinbase_advanced_trade', 'cube', 'dexalot', 'hyperliquid', 'derive' or 'tegro'",
      "input": "ralph-handsome",
      "ctx": {
        "expected": "'binance', 'binance_us', 'coin_gecko', 'coin_cap', 'kucoin', 'ascend_ex', 'gate_io', 'coinbase_advanced_trade', 'cube', 'dexalot', 'hyperliquid', 'derive' or 'tegro'"
      }
    }
  ]
}
logs: INFO:     172.18.0.1:45658 - "PUT /rate-oracle/config HTTP/1.1" 422 Unprocessable Entity

I confirmed no changes made on rate_oracle_source however i think we can improve the response of the log file? perhaps same like msg “Input should be 'binance', 'binance_us', 'coin_gecko', 'coin_cap', 'kucoin', 'ascend_ex', 'gate_io', 'coinbase_advanced_trade', 'cube', 'dexalot'”

Incase needed see attached test logs 09012026.log

rapcmia · 2026-01-09T15:22:05Z

Im not able to start docker, ImportError: cannot import name 'executors' from 'routers' (/hummingbot-api/routers/__init__.py) ❌

cardosofede added 4 commits January 7, 2026 15:19

(feat) improve order polling

dbca1fe

(feat) expose more info of the config

d7505c3

(feat) improve errorhandling

8bb327a

(feat) add rate oracle router

36c87a6

cardosofede changed the title ~~(feat) improve order polling~~ (feat) Improved Order Polling, Enhanced Connector Configuration, and Rate Oracle API Jan 9, 2026

rapcmia changed the title ~~(feat) Improved Order Polling, Enhanced Connector Configuration, and Rate Oracle API~~ feat / Improved Order Polling, Enhanced Connector Configuration, and Rate Oracle API Jan 9, 2026

(feat) add rate oracle models

dae36a2

(feat) add validation error

014a457

cardosofede added 2 commits January 9, 2026 12:35

(feat) remove executors

0491b74

(feat) remove executors service

1618319

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat / Improved Order Polling, Enhanced Connector Configuration, and Rate Oracle API #106

feat / Improved Order Polling, Enhanced Connector Configuration, and Rate Oracle API #106

Uh oh!

cardosofede commented Jan 7, 2026 •

edited

Loading

Uh oh!

rapcmia commented Jan 9, 2026

Uh oh!

rapcmia commented Jan 9, 2026

Uh oh!

rapcmia commented Jan 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

feat / Improved Order Polling, Enhanced Connector Configuration, and Rate Oracle API #106

Are you sure you want to change the base?

feat / Improved Order Polling, Enhanced Connector Configuration, and Rate Oracle API #106

Uh oh!

Conversation

cardosofede commented Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rapcmia commented Jan 9, 2026

Uh oh!

rapcmia commented Jan 9, 2026

Test rate-oracle routes

Test rate-oracle/config with invalid source ❗

Uh oh!

rapcmia commented Jan 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

cardosofede commented Jan 7, 2026 •

edited

Loading