Consider revisiting shared file reader trait for JSON and Parquet

[sebastiantia]

Please describe why this is necessary.

Previously, we had a shared file reader trait for handling both JSON and Parquet files, but it was removed to reduce unnecessary coupling. However, in create_checkpoint_stream, we now have a case where such a trait could have been beneficial.

Right now, we handle JSON and Parquet separately via:

  let actions = match self.checkpoint_parts.first() {
            Some(parsed_log_path) if parsed_log_path.extension == "json" => {
                engine.get_json_handler().read_json_files(
                    &checkpoint_file_meta,
                    checkpoint_read_schema.clone(),
                    meta_predicate,
                )?
            }
            Some(parsed_log_path) if parsed_log_path.extension == "parquet" => parquet_handler
                .read_parquet_files(
                    &checkpoint_file_meta,
                    checkpoint_read_schema.clone(),
                    meta_predicate,
                )?,
            Some(parsed_log_path) => {
                return Err(Error::generic(format!(
                    "Unsupported checkpoint file type: {}",
                    parsed_log_path.extension,
                )));
            }
            None => Box::new(std::iter::empty()),
        };

If similar patterns start appearing elsewhere, it may be worth reconsidering the shared file reader abstraction.

Describe the functionality you are proposing.

Action Items

• Track occurrences of this pattern in other parts of the codebase.
• Evaluate whether a shared file reader trait would be useful again.
• Consider potential trade-offs in reintroducing a shared abstraction vs. keeping them separate.

Places Where This Appears

1. create_checkpoint_stream in log replay.
2. (Add any other instances as they come up).

This issue will serve as a reference point if we decide to revisit the shared file reader approach in the future.

Additional context

No response