Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@

[![Crates.io](https://img.shields.io/crates/v/toon-format.svg)](https://crates.io/crates/toon-format)
[![Documentation](https://docs.rs/toon-format/badge.svg)](https://docs.rs/toon-format)
[![Spec v2.0](https://img.shields.io/badge/spec-v2.0-brightgreen.svg)](https://github.com/toon-format/spec/blob/main/SPEC.md)
[![Spec v3.0](https://img.shields.io/badge/spec-v3.0-brightgreen.svg)](https://github.com/toon-format/spec/blob/main/SPEC.md)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](./LICENSE)
[![Tests](https://img.shields.io/badge/tests-%20passing-success.svg)]()

**Token-Oriented Object Notation (TOON)** is a compact, human-readable format designed for passing structured data to Large Language Models with significantly reduced token usage.

This crate provides the official, **spec-compliant Rust implementation** of TOON v2.0 with v1.5 optional features, offering both a library (`toon-format`) and a full-featured command-line tool (`toon`).
This crate provides the official, **spec-compliant Rust implementation** of TOON v3.0, offering both a library (`toon-format`) and a full-featured command-line tool (`toon`).

## Quick Example

Expand All @@ -32,8 +32,8 @@ users[2]{id,name}:
## Features

- **Generic API**: Works with any `Serialize`/`Deserialize` type - custom structs, enums, JSON values, and more
- **Spec-Compliant**: Fully compliant with [TOON Specification v2.0](https://github.com/toon-format/spec/blob/main/SPEC.md)
- **v1.5 Optional Features**: Key folding and path expansion
- **Spec-Compliant**: Fully compliant with [TOON Specification v3.0](https://github.com/toon-format/spec/blob/main/SPEC.md)
- **Key Folding & Path Expansion**: Collapse and expand dotted key paths
- **Safe & Performant**: Built with safe, fast Rust
- **Powerful CLI**: Full-featured command-line tool
- **Strict Validation**: Enforces all spec rules (configurable)
Expand Down Expand Up @@ -547,7 +547,7 @@ Run with `cargo run --example examples` to see all examples:

## Resources

- 📖 [TOON Specification v2.0](https://github.com/toon-format/spec/blob/main/SPEC.md)
- 📖 [TOON Specification v3.0](https://github.com/toon-format/spec/blob/main/SPEC.md)
- 📦 [Crates.io Package](https://crates.io/crates/toon-format)
- 📚 [API Documentation](https://docs.rs/toon-format)
- 🔧 [Main Repository (JS/TS)](https://github.com/toon-format/toon)
Expand Down
7 changes: 7 additions & 0 deletions src/decode/parser.rs
Original file line number Diff line number Diff line change
Expand Up @@ -995,6 +995,13 @@ impl<'a> Parser<'a> {
if !self.options.strict {
self.skip_newlines()?;
}
} else if matches!(self.current_token, Token::String(_, _)) {
// Tabular array parser already consumed the newline
// and advanced to the next token — check indent
let current_indent = self.scanner.get_last_line_indent();
if current_indent != field_indent {
break;
}
} else {
break;
}
Expand Down
16 changes: 16 additions & 0 deletions src/decode/scanner.rs
Original file line number Diff line number Diff line change
Expand Up @@ -358,6 +358,14 @@ impl Scanner {
}
}

// Negative leading zeros like "-05" are also strings
if s.starts_with("-0") && s.len() > 2 {
let third_char = s.chars().nth(2).unwrap();
if third_char.is_ascii_digit() {
return Ok(Token::String(s.to_string(), false));
}
}

if s.contains('.') || s.contains('e') || s.contains('E') {
if let Ok(f) = s.parse::<f64>() {
Ok(Token::Number(f))
Expand Down Expand Up @@ -467,6 +475,14 @@ impl Scanner {
}
}

// Negative leading zeros like "-05" are also strings
if trimmed.starts_with("-0") && trimmed.len() > 2 {
let third_char = trimmed.chars().nth(2).unwrap();
if third_char.is_ascii_digit() {
return Ok(Token::String(trimmed.to_string(), false));
}
}

if trimmed.contains('.') || trimmed.contains('e') || trimmed.contains('E') {
if let Ok(f) = trimmed.parse::<f64>() {
let normalized = if f == -0.0 { 0.0 } else { f };
Expand Down
2 changes: 1 addition & 1 deletion src/encode/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -620,7 +620,7 @@ fn encode_nested_array(
match value {
Value::Array(arr) => {
writer.write_key(key)?;
write_array(writer, None, arr, depth + 1)?;
write_array(writer, None, arr, depth + 2)?;
}
Value::Object(nested_obj) => {
writer.write_key(key)?;
Expand Down