Skip to content

Commit a8a5719

Browse files
committed
add table self_destruct
1 parent 096cc8c commit a8a5719

File tree

1 file changed

+4
-2
lines changed
  • src/nested_pandas/nestedframe

1 file changed

+4
-2
lines changed

src/nested_pandas/nestedframe/io.py

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -113,8 +113,10 @@ def read_parquet(
113113
table = table.append_column(col, struct)
114114

115115
# Convert to NestedFrame
116-
# TODO: How much of a problem is it that this is not zero_copy? True below fails
117-
df = NestedFrame(table.to_pandas(types_mapper=lambda ty: pd.ArrowDtype(ty), zero_copy_only=False))
116+
# TODO: How much of a problem is it that this is not zero_copy?
117+
# https://arrow.apache.org/docs/python/pandas.html#reducing-memory-use-in-table-to-pandas
118+
df = NestedFrame(table.to_pandas(types_mapper=lambda ty: pd.ArrowDtype(ty), self_destruct=True))
119+
del table
118120

119121
# Attempt to cast struct columns to NestedDTypes
120122
df = _cast_struct_cols_to_nested(df, reject_nesting)

0 commit comments

Comments
 (0)