[BUG]: Prevent hnsw cleanup if use_direct_hnsw is true (#5933)

## Description of changes

As titled.

- Improvements & Bug fixes
  - ...
- New functionality
  - ...

## Test plan

_How are these changes tested?_

- [ ] Tests pass locally with `pytest` for python, `yarn test` for js, `cargo test` for rust

## Migration plan

_Are there any migrations, or any forwards/backwards compatibility changes needed in order to make sure this change deploys reliably?_

## Observability plan

_What is the plan to instrument and monitor this change?_

## Documentation Changes

_Are all docstrings for user-facing APIs updated if required? Do we need to make documentation changes in the [docs section](https://github.com/chroma-core/chroma/tree/main/docs/docs.trychroma.com)?_

Author: tanujnay112

rust/index/src/hnsw_provider.rs

@@ -60,7 +60,7 @@ pub struct HnswIndexProvider {
     // TODO(tanujnay112): This feature flag is a temporary measure to gate
     // the hnsw loading from memory feature. Remove this after that feature
     // stabilizes.
-    use_direct_hnsw: bool,
+    pub use_direct_hnsw: bool,
 }
 
 pub struct HnswIndexFlusher {

rust/worker/src/execution/orchestration/compact.rs

@@ -834,6 +834,11 @@ impl CompactionContext {
     }
 
     pub(crate) async fn cleanup(self) {
+        if self.hnsw_provider.use_direct_hnsw {
+            return;
+        }
+
+        // TODO(tanujnay112): Remove when use_direct_hnsw is fully deprecated
         for hnsw_index_uuid in self.hnsw_index_uuids {
             let _ = HnswIndexProvider::purge_one_id(
                 self.hnsw_provider.temporary_storage_path.as_path(),