Skip to content

feat: Enhanced JSONB Parser with Decimal Support and Extended Syntax#86

Merged
b41sh merged 1 commit intodatabendlabs:mainfrom
b41sh:feat-parse-dec
Jun 27, 2025
Merged

feat: Enhanced JSONB Parser with Decimal Support and Extended Syntax#86
b41sh merged 1 commit intodatabendlabs:mainfrom
b41sh:feat-parse-dec

Conversation

@b41sh
Copy link
Member

@b41sh b41sh commented Jun 27, 2025

Summary

This PR significantly enhances the jsonb parser with improved decimal type support, extended JSON syntax compatibility, comprehensive testing with proptest, and extensive code documentation. These changes make the parser more robust, versatile, and maintainable.

Key Changes:

  1. Decimal Support for Numbers:
  • Implemented full decimal number support with three precision levels:
    • Decimal64: 64-bit precision for common financial calculations
    • Decimal128: 128-bit precision for larger decimal values
    • Decimal256: 256-bit precision for extremely large values requiring exact representation
  • All decimal types maintain full precision without floating-point errors
  • Automatic selection of the most appropriate numeric representation based on input value characteristics
  • Comprehensive comparison operations between different numeric types with precision preservation
  1. Extended JSON Syntax Support
  • Added support for non-standard JSON syntax to improve compatibility with various data sources:
    • Leading plus sign in numbers (e.g., +123)
    • Multiple leading zeros in numbers (e.g., 000123)
    • Decimal points without preceding digits (e.g., .123)
    • Decimal points without following digits (e.g., 123.)
    • Empty elements in arrays (e.g., [1,,3] or [1,2,])
  • Implemented a high-performance single-pass number parsing algorithm with multiple fallback strategies
  1. Property-Based Testing with Proptest
    Implemented comprehensive property-based testing using the proptest framework, test cases include:
    • Standard JSON values (null, boolean, number, string, array, object)
    • Extended numeric types (all decimal precisions)
    • Complex nested structures with mixed types
    • Edge cases for number parsing and string escaping

@b41sh b41sh requested a review from sundy-li June 27, 2025 05:54
@b41sh b41sh merged commit 4649f52 into databendlabs:main Jun 27, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants