Skip to content

fix: handle compare_vals turn into str when parse IgnoreColumnType #50205

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

phakawatfong
Copy link

when I parse ignoreColumnType=True when call the function, it cast all values to be 'str' which causing the comparison between val1 and val2 to be failed.
for example

val1 = 1505.76189560
val2= 1505.761896
assertDataFrameEqual(source_df, databricks_df, ignoreColumnType=True,)

I will convert the original data into string with this clause first

if ignoreColumnType:
    actual = cast_columns_to_string(actual)
    expected = cast_columns_to_string(expected)
.
.
compare_vals(val1, val2)

it will become string comparison instead of compare by using the tolerance equation
'1505.76189560' != '1505.761896'

Hence assertError, even if the atol and rtol already assigned

What changes were proposed in this pull request?

add more elif case to convert str to float and parse into tolerant calculations

Why are the changes needed?

I think it might not be the only way to fix it, but just to be your ideas Thank you !

Does this PR introduce any user-facing change?

No

How was this patch tested?

Test on my local data source, comparing 2 dataframes
by patch the code into my local env.

Was this patch authored or co-authored using generative AI tooling?

No

@github-actions github-actions bot added the PYTHON label Mar 7, 2025
Copy link

We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!

@github-actions github-actions bot added the Stale label Jun 16, 2025
@github-actions github-actions bot closed this Jun 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant