Skip to content

Conversation

@phakawatfong
Copy link

when I parse ignoreColumnType=True when call the function, it cast all values to be 'str' which causing the comparison between val1 and val2 to be failed.
for example

val1 = 1505.76189560
val2= 1505.761896
assertDataFrameEqual(source_df, databricks_df, ignoreColumnType=True,)

I will convert the original data into string with this clause first

if ignoreColumnType:
    actual = cast_columns_to_string(actual)
    expected = cast_columns_to_string(expected)
.
.
compare_vals(val1, val2)

it will become string comparison instead of compare by using the tolerance equation
'1505.76189560' != '1505.761896'

Hence assertError, even if the atol and rtol already assigned

What changes were proposed in this pull request?

add more elif case to convert str to float and parse into tolerant calculations

Why are the changes needed?

I think it might not be the only way to fix it, but just to be your ideas Thank you !

Does this PR introduce any user-facing change?

No

How was this patch tested?

Test on my local data source, comparing 2 dataframes
by patch the code into my local env.

Was this patch authored or co-authored using generative AI tooling?

No

@github-actions github-actions bot added the PYTHON label Mar 7, 2025
@github-actions
Copy link

We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!

@github-actions github-actions bot added the Stale label Jun 16, 2025
@github-actions github-actions bot closed this Jun 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant