Update docs

Author: daniel-thom

README.md

@@ -16,7 +16,7 @@ To use DuckDB or SQLite as the backend:
 $ pip install chronify
 ```
 
-To use Apache Spark via Apache Thrift Server as the backend:
+To use Apache Spark as the backend:
 ```
 $ pip install "chronify[spark]"
 ```

docs/how_tos/getting_started/installation.md

@@ -37,8 +37,8 @@ To use DuckDB or SQLite as the backend:
     $ pip install chronify
 ```
 
-To use Apache Spark via Apache Thrift Server as the backend, you must install pyhive.
-This command will install the necessary dependencies.
+To use Apache Spark as the backend, install chronify with the ``spark`` extra,
+which pulls in PySpark:
 
 ```{eval-rst}
 .. code-block:: console

docs/how_tos/ingest_multiple_tables.md

@@ -1,7 +1,6 @@
 # How to Ingest Multiple Tables Efficiently
 
 There are a few important considerations when ingesting many tables:
-- Use one database connection.
 - Avoid loading all tables into memory at once, if possible.
 - Ensure additions are atomic. If anything fails, the final state should be the same as the initial
 state.
@@ -48,25 +47,23 @@ dst_schema = TableSchema(
 Chronify will manage the database connection and errors.
 ```python
 store.ingest_from_csvs(
-    src_schema,
-    dst_schema,
     (
         "/path/to/file1.csv",
         "/path/to/file2.csv",
         "/path/to/file3.csv",
     ),
- )
+    src_schema,
+    dst_schema,
+)
 
 ```
 
 ## Self-Managed
-Open one connection to the database for the duration of your additions. Handle errors.
+Wrap the additions in a backend transaction. Any tables or views created within the block are
+automatically dropped if an exception is raised.
 ```python
-with store.engine.connect() as conn:
-    try:
-        store.ingest_from_csv(src_schema, dst_schema, "/path/to/file1.csv")
-        store.ingest_from_csv(src_schema, dst_schema, "/path/to/file2.csv")
-        store.ingest_from_csv(src_schema, dst_schema, "/path/to/file3.csv")
-    except Exception:
-        conn.rollback()
+with store.backend.transaction():
+    store.ingest_from_csv("/path/to/file1.csv", src_schema, dst_schema)
+    store.ingest_from_csv("/path/to/file2.csv", src_schema, dst_schema)
+    store.ingest_from_csv("/path/to/file3.csv", src_schema, dst_schema)
 ```

docs/how_tos/spark_backend.md

@@ -62,9 +62,18 @@ schema = TableSchema(
 
 ```python
 from chronify import Store
+from chronify.ibis.spark_backend import SparkBackend
 
-store = Store.create_new_hive_store("hive://localhost:10000/default")
-store.create_view_from_parquet("data.parquet")
+store = Store(backend=SparkBackend())
+store.create_view_from_parquet("data.parquet", schema)
+```
+
+Alternatively, pass a pre-configured PySpark session:
+```python
+from pyspark.sql import SparkSession
+
+session = SparkSession.builder.master("local").getOrCreate()
+store = Store(backend=SparkBackend(session=session))
 ```
 
 Verify the data:

docs/index.md

@@ -4,7 +4,7 @@ This package implements validation, mapping, and storage of time series data in
 Python-based modeling packages.
 
 ## Features
-- Stores time series data in any database supported by SQLAlchemy.
+- Stores time series data in any database supported by Ibis (DuckDB, SQLite, and Spark).
 - Supports data ingestion in a variety of file formats and configurations.
 - Supports efficient retrieval of time series through SQL queries.
 - Validates consistency of timestamps and resolution.
@@ -23,24 +23,12 @@ Python-based modeling packages.
 ```
 
 ## Supported Backends
-While chronify should work with any database supported by SQLAlchemy, it has been tested with
-the following:
+Chronify uses [Ibis](https://ibis-project.org) for all database operations. The following
+backends are supported:
 
 - DuckDB (default)
 - SQLite
-- Apache Spark through Apache Thrift Server
-
-DuckDB and SQLite are fully supported.
-
-Because of limitations in the backend software, chronify functionality with Spark is limited to
-the following:
-
-- Create a view into an existing Parquet file (or directory).
-- Perform time series checks.
-- Map between time configurations.
-- Write output data to Parquet files.
-
-There is no support for creating tables and ingesting data with Spark.
+- Apache Spark (via PySpark)
 
 ## How to use this guide
 - Refer to [How Tos](#how-tos-page) for step-by-step instructions for creating store and ingesting data.