Skip to content

Transforming Old Pickle Files into Protobuf #1403

@paul-jdfagan

Description

@paul-jdfagan

Hi Meridian Team,

Given the deprecation of the pickle method in the latest release, how do you recommend transforming old pickle files into the new Protobuf format? This is an issue as we could have multiple previous saved pickle files for each past month and might need to reload a previous model.

Also, when I try to actually save a model I get this warning serialising the prior:
`---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/tmp/ipython-input-1787322961.py in <cell line: 0>()
1 file_path = '/content/drive/MyDrive/acorns_m11_summary_ds.binpb'
----> 2 meridian_serde.save_meridian(mmm, file_path)

5 frames
/usr/local/lib/python3.12/dist-packages/schema/serde/meridian_serde.py in save_meridian(mmm, file_path, distribution_function_registry, eda_function_registry)
355 with _file_open(file_path, 'wb') as f:
356 # Creates an MmmKernel.
--> 357 serialized_kernel = MeridianSerde().serialize(
358 mmm,
359 distribution_function_registry=distribution_function_registry,

/usr/local/lib/python3.12/dist-packages/schema/serde/meridian_serde.py in serialize(self, obj, model_id, meridian_version, include_convergence_info, distribution_function_registry, eda_function_registry)
112 else function_registry_utils.FunctionRegistry()
113 )
--> 114 meridian_model_proto = self._make_meridian_model_proto(
115 mmm=obj,
116 model_id=model_id,

/usr/local/lib/python3.12/dist-packages/schema/serde/meridian_serde.py in _make_meridian_model_proto(self, mmm, model_id, meridian_version, distribution_function_registry, eda_function_registry, include_convergence_info)
162 prior_tfp_distributions=distribution.DistributionSerde(
163 distribution_function_registry
--> 164 ).serialize(mmm.model_spec.prior),
165 inference_data=inference_data.InferenceDataSerde().serialize(
166 mmm.inference_data

/usr/local/lib/python3.12/dist-packages/schema/serde/distribution.py in serialize(self, obj)
72 continue
73 getattr(proto, param).CopyFrom(
---> 74 self._to_distribution_proto(getattr(obj, param))
75 )
76 proto.function_registry.update(self.function_registry.hashed_registry)

/usr/local/lib/python3.12/dist-packages/schema/serde/distribution.py in _to_distribution_proto(self, dist)
161 distribution_type=dist_name,
162 parameters={
--> 163 name: self._to_parameter_value_proto(name, value, dist_class)
164 for name, value in dist.parameters.items()
165 },

/usr/local/lib/python3.12/dist-packages/schema/serde/distribution.py in _to_parameter_value_proto(self, param_name, value, dist)
260
261 # Handle unsupported types.
--> 262 raise TypeError(f"Unsupported type: {type(value)}, {value}")
263
264 def _from_distribution_proto(

TypeError: Unsupported type: <class 'numpy.ndarray'>, [ 1.6 1.6 4.5 1.6 4.5 4. 6. 1.6 1.6 5. 1. 18. ]`

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions