Skip to content

Conversation

@shivasavade
Copy link

  1. Fix incorrect MOD calculation for DECIMAL operands in ColumnStore.
    Root Cause:
    • The divisor was truncated to an integer (div), losing its scale. May not work is the interger part is zero. (e.g: 0.5)
    • Modulus computation used integer arithmetic instead of scaled decimal arithmetic.
    • Precision and scale were not correctly aligned between the dividend and divisor.

Solution:
• Retrieve both operands as IDB_Decimal values using getDecimalVal().
• Normalize the scales of both operands using max(d.scale, div.scale).
• Perform modulus on properly scaled 128-bit integer representations.
• Return a result with consistent scale and divisor precision.

  1. Fix incorrect DIV calculation for DECIMAL operands in ColumnStore.
    Root Cause:
    • Operands were converted to double, losing precision for DECIMAL values.
    • Scale differences between dividend and divisor were not handled.
    • Also, double cannot exactly represent 1.1 in binary.
    1.1 in base 10 equals:
    1 + 1/10 = 1 + 0.0001100110011001100...(in binary)
    That repeating binary fraction never ends.
    When stored in a finite 64-bit double, it gets rounded to the closest representable number, which is actually: 1.1000000000000001

Solution:
• For exact decimal display, Its better we use the DECIMAL, not double.
• When printing doubles, format to a fixed precision: 1.1000000000000001
• Retrieve operands as IDB_Decimal instead of double.
• Normalize scales between dividend (dec1) and divisor (dec2) before performing division.
• Use 128-bit integer arithmetic for precise scaled division.

@mariadb-LeonidFedorov
Copy link
Collaborator

mariadb-LeonidFedorov commented Nov 9, 2025

Thank you for the contribution!
Could you please add the tests covering your fix columnstore/basic/t/mcs227_div_function.test
To run the test and to update the reference output columnstore/basic/r/mcs227_div_function.result, you can use
--record of following script

➜  sudo tests/scripts/run_mtr.sh --help
usage: tests/scripts/run_mtr.sh [OPTIONS]
OPTIONS:
        
	-s --suite:                  whole suite to run
	-t --test_full_name:         Testname with suite as like bugfixes.mcol-4899
	-f --full:                   Run full MTR [default:false]
	-r --record:                 Record the result [default:false]
	-e --no-extern:              Run with --extern [default:false]
	-c --color:                  run with unbufer to colorize output [default:false]
        -? --help  :  usage

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants