Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -147,3 +147,4 @@ examples/sparse_lr/mlruns/
Cargo.lock
.trunk/
proptest-regressions/
benchmark/outputs/
105 changes: 68 additions & 37 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
# Hyperparameter
<p align="center">
<img src="hyperparameter.svg" alt="Hyperparameter Logo" width="180" height="180">
</p>

<h3 align="center">
<p style="text-align: center;">
<a href="README.md" target="_blank">ENGLISH</a> | <a href="README.zh.md">中文文档</a>
</p>
</h3>
<h1 align="center">Hyperparameter</h1>

<p align="center">
<a href="README.md" target="_blank">ENGLISH</a> | <a href="README.zh.md">中文文档</a>
</p>

**Hyperparameter, Make configurable AI applications. Build for Python/Rust hackers.**

<p align="center">
<strong>Make configurable AI applications. Build for Python/Rust hackers.</strong>
</p>

Hyperparameter is a versatile library designed to streamline the management and control of hyperparameters in machine learning algorithms and system development. Tailored for AI researchers and Machine Learning Systems (MLSYS) developers, Hyperparameter offers a unified solution with a focus on ease of use in Python, high-performance access in Rust and C++, and a set of macros for seamless hyperparameter management.
Expand All @@ -22,33 +22,64 @@ pip install hyperparameter
# Run a ready-to-use demo
python -m hyperparameter.examples.quickstart

# Try the @auto_param CLI: override defaults from the command line
# Try the @hp.param CLI: override defaults from the command line
python -m hyperparameter.examples.quickstart --define greet.name=Alice --enthusiasm=3

# Inspect params and defaults
python -m hyperparameter.examples.quickstart -lps
python -m hyperparameter.examples.quickstart -ep greet.name

# Running from source? Use module mode or install editable
# python -m hyperparameter.examples.quickstart
# or: pip install -e .
```

What it shows:
- default values vs scoped overrides (`param_scope`)
- `@auto_param` + `launch` exposing a CLI with `-D/--define` for quick overrides

## Key Features
python -m hyperparameter.examples.quickstart -ep greet.name

# Running from source? Use module mode or install editable
# python -m hyperparameter.examples.quickstart
# or: pip install -e .
```

## Why Hyperparameter?

### 🚀 Unmatched Performance (vs Hydra)

Hyperparameter is built on a high-performance Rust backend, making it significantly faster than pure Python alternatives like Hydra, especially in inner-loop parameter access.

| Method | Time (1M iters) | Speedup (vs Hydra) |
| :--- | :--- | :--- |
| **HP: Injected (Native Speed)** | **0.0184s** | **856.73x** 🚀 |
| **HP: Dynamic (Optimized)** | **2.4255s** | **6.50x** ⚡️ |
| **Hydra (Baseline)** | 15.7638s | 1.00x |

> Benchmark scenario: Accessing a nested parameter `model.layers.0.size` 1,000,000 times in a loop.
> See `benchmark/` folder for reproduction scripts.

### ✨ Zero-Dependency Schema Validation

Hyperparameter supports structural validation using standard Python type hints without introducing heavy dependencies (like Pydantic or OmegaConf).

```python
from dataclasses import dataclass
import hyperparameter as hp

@dataclass
class AppConfig:
host: str
port: int
debug: bool = False

# Validates types and converts automatically: "8080" -> 8080 (int)
cfg = hp.config("config.toml", schema=AppConfig)
```

## Key Features

### For Python Users

- **Pythonic Syntax:** Define hyperparameters using keyword argument syntax;

- **Intuitive Scoping:** Control parameter scope through `with` statement;

- **Configuration File:** Easy to load parameters from config files;

### For Rust and C++ Users
- **Intuitive Scoping:** Control parameter scope through `with` statement;

- **Configuration File:** Easy to load parameters from config files (JSON/TOML/YAML) with composition and interpolation support;

- **Zero-Overhead Validation:** Optional schema validation using standard Python type hints;

### For Rust and C++ Users

- **High-Performance Backend:** Hyperparameter is implemented in Rust, providing a robust and high-performance backend for hyperparameter management. Access hyperparameters in Rust and C++ with minimal overhead, making it ideal for ML and system developers who prioritize performance.

Expand All @@ -67,15 +98,15 @@ pip install hyperparameter
### Python

```python
from hyperparameter import auto_param, param_scope
import hyperparameter as hp

@auto_param("foo")
@hp.param("foo")
def foo(x=1, y="a"):
return f"x={x}, y={y}"

foo() # x=1, y='a'

with param_scope(**{"foo.x": 2}):
with hp.scope(**{"foo.x": 2}):
foo() # x=2, y='a'
```

Expand Down Expand Up @@ -124,7 +155,7 @@ ASSERT(1 == GET_PARAM(a.b, 1), "get undefined param");
#### Python

```python
x = param_scope.foo.x | "default value"
x = hp.scope.foo.x | "default value"
```

#### Rust
Expand All @@ -138,9 +169,9 @@ x = param_scope.foo.x | "default value"
#### Python

```python
with param_scope() as ps: # 1st scope start
with hp.scope() as ps: # 1st scope start
ps.foo.x=1
with param_scope() as ps2: # 2nd scope start
with hp.scope() as ps2: # 2nd scope start
ps.foo.y=2
# 2nd scope end
# 1st scope end
Expand All @@ -165,11 +196,11 @@ with_params!{ // 1st scope start
#### Python

```python
@auto_param("foo")
@hp.param("foo")
def foo(x=1): # Print hyperparameter foo.x
print(f"foo.x={x}")

with param_scope() as ps:
with hp.scope() as ps:
ps.foo.x=2 # Modify foo.x in the current thread

foo() # foo.x=2
Expand Down Expand Up @@ -205,9 +236,9 @@ In command line applications, it's common to define hyperparameters using comman

```python
# example.py
from hyperparameter import param_scope, auto_param
import hyperparameter as hp

@auto_param("example")
@hp.param("example")
def main(a=0, b=1):
print(f"example.a={a}, example.b={b}")

Expand All @@ -218,7 +249,7 @@ if __name__ == "__main__":
parser.add_argument("-D", "--define", nargs="*", default=[], action="extend")
args = parser.parse_args()

with param_scope(*args.define):
with hp.scope(*args.define):
main()
```

Expand Down
54 changes: 27 additions & 27 deletions README.zh.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
# Hyperparameter
<p align="center">
<img src="hyperparameter.svg" alt="Hyperparameter Logo" width="180" height="180">
</p>

<h3 align="center">
<p style="text-align: center;">
<a href="README.md" target="_blank">ENGLISH</a> | <a href="README.zh.md">中文文档</a>
</p>
</h3>
<h1 align="center">Hyperparameter</h1>

<p align="center">
<a href="README.md" target="_blank">ENGLISH</a> | <a href="README.zh.md">中文文档</a>
</p>

**Hyperparameter, Make configurable AI applications. Build for Python/Rust hackers.**

<p align="center">
<strong>Make configurable AI applications. Build for Python/Rust hackers.</strong>
</p>

`Hyperparameter` 是一个多功能超参数管理库,旨在简化机器学习算法和系统开发中超参数的管理和控制。专为机器学习系统(MLSYS)开发者设计,超参数提供了一个统一的解决方案,侧重于在Python中易于使用、在Rust和C++中高性能访问,并提供了一组宏,以实现无缝超参数管理。
Expand All @@ -20,11 +20,13 @@

- **Pythonic语法:** 使用keyword参数语法定义超参数;

- **直观的作用域:** 通过`with`语句控制参数的作用域;

- **配置文件:** 从配置文件轻松加载参数;

### 针对Rust和C++用户
- **直观的作用域:** 通过`with`语句控制参数的作用域;

- **强大的配置加载:** 支持 JSON/TOML/YAML 多文件组合加载 (Composition) 与变量插值 (Interpolation);

- **零开销校验:** 支持可选的基于 Python Type Hints 的 Schema 校验;

### 针对Rust和C++用户

- **高性能后端:** 超参数在Rust中实现,提供了强大且高性能的超参数管理后端。在Rust和C++中以最小开销访问超参数,非常适合注重性能的ML和系统开发者。

Expand All @@ -43,15 +45,15 @@ pip install hyperparameter
### Python

```python
from hyperparameter import auto_param, param_scope
import hyperparameter as hp

@auto_param("foo")
@hp.param("foo")
def foo(x=1, y="a"):
return f"x={x}, y={y}"

foo() # x=1, y='a'

with param_scope(**{"foo.x": 2}):
with hp.scope(**{"foo.x": 2}):
foo() # x=2, y='a'
```

Expand Down Expand Up @@ -100,7 +102,7 @@ ASSERT(1 == GET_PARAM(a.b, 1), "get undefined param");
#### Python

```python
x = param_scope.foo.x | "default value"
x = hp.scope.foo.x | "default value"
```

#### Rust
Expand All @@ -114,9 +116,9 @@ x = param_scope.foo.x | "default value"
#### Python

```python
with param_scope() as ps: # 第1个作用域开始
with hp.scope() as ps: # 第1个作用域开始
ps.foo.x=1
with param_scope() as ps2: # 第2个作用域开始
with hp.scope() as ps2: # 第2个作用域开始
ps.foo.y=2
# 第2个作用域结束
# 第1个作用域结束
Expand All @@ -141,14 +143,12 @@ with_params!{ // 第1个作用域开始
#### Python

```python
@auto_param("foo")
@hp.param("foo")
def foo(x=1): # 打印超参数 foo.x
print(f"foo.x={x}")

with param_scope() as ps:
ps.foo.x=2 # 在当前线程设置foo.x

中修改 foo.x
with hp.scope() as ps:
ps.foo.x=2 # 在当前线程中修改 foo.x

foo() # foo.x=2
threading.Thread(target=foo).start() # foo.x=1,新线程的超参数值不受主线程的影响
Expand Down Expand Up @@ -183,9 +183,9 @@ fn main() {

```python
# example.py
from hyperparameter import param_scope, auto_param
import hyperparameter as hp

@auto_param("example")
@hp.param("example")
def main(a=0, b=1):
print(f"example.a={a}, example.b={b}")

Expand All @@ -196,7 +196,7 @@ if __name__ == "__main__":
parser.add_argument("-D", "--define", nargs="*", default=[], action="extend")
args = parser.parse_args()

with param_scope(*args.define):
with hp.scope(*args.define):
main()
```

Expand Down
22 changes: 22 additions & 0 deletions benchmark/bench_hp.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
import time
import hyperparameter as hp

@hp.param
def main():
start = time.time()
acc = 0

# We use hp.scope directly, which is the idiomatic way
# to access parameters anywhere in the code.
with hp.scope() as ps:
for _ in range(1_000_000):
acc += ps.model.layers._0.size | 10

duration = time.time() - start
print(f"Hyperparameter Time: {duration:.4f} seconds (acc={acc})")
return duration

if __name__ == "__main__":
# Pre-populate scope to simulate loaded config
with hp.scope(**{"model.layers._0.size": 10}):
main()
17 changes: 17 additions & 0 deletions benchmark/bench_hp_dynamic_global.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import time
import hyperparameter as hp

@hp.param
def main():
start = time.time()
acc = 0

for _ in range(1_000_000):
acc += hp.scope.model.layers._0.size | 10

duration = time.time() - start
print(f"Hyperparameter Time: {duration:.4f} seconds (acc={acc})")
return duration

if __name__ == "__main__":
hp.launch()
20 changes: 20 additions & 0 deletions benchmark/bench_hp_dynamic_local.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
import time
import hyperparameter as hp

@hp.param
def main():
start = time.time()
acc = 0

for _ in range(1_000_000):
with hp.scope() as ps:
acc += ps.model.layers._0.size | 10

duration = time.time() - start
print(f"Hyperparameter Time: {duration:.4f} seconds (acc={acc})")
return duration

if __name__ == "__main__":
# Pre-populate scope to simulate loaded config
with hp.scope(**{"model.layers._0.size": 10}):
main()
17 changes: 17 additions & 0 deletions benchmark/bench_hp_injected.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import time
import hyperparameter as hp

@hp.param
def main(layer_size: int = 10):
start = time.time()
acc = 0

for _ in range(1_000_000):
acc += layer_size

duration = time.time() - start
print(f"Hyperparameter Time: {duration:.4f} seconds (acc={acc})")
return duration

if __name__ == "__main__":
hp.launch()
Loading
Loading