DOCSP-36546 Scan Multiple Collections

source/release-notes.txt

@@ -2,6 +2,38 @@
 Release Notes
 =============
 
+MongoDB Connector for Spark 10.3
+--------------------------------
+
+The 10.3 connector release includes the following new features:
+
+- Added support for reading multiple collections when using micro-batch or
+  continuous streaming modes.
+
+  .. warning:: Breaking Change
+
+     Support for reading multiple collections introduces the following breaking
+     changes:
+
+     - If the name of a collection used in your ``collection`` configuration
+       option contains a comma (,), the
+       {+connector-short+} treats it as two different collections. To avoid
+       this, you must escape the comma by preceding it with a backslash (\\).
+
+     - If the name of a collection used in your ``collection`` configuration
+       option is "*", the {+connector-short+} interprets it as a specification
+       to scan all collections. To avoid this, you must escape the asterisk by preceding it
+       with a backslash (\\).
+
+     - If the name of a collection used in your ``collection`` configuration
+       option contains a backslash (\\), the
+       {+connector-short+} treats the backslash as an escape character, which
+       might change how the value is interpreted. To avoid this, you must escape
+       the backslash by preceding it with another backslash.
+
+  To learn more about scanning multiple collections, see the :ref:`collection
+  configuration property <spark-streaming-input-conf>` description.
+
 MongoDB Connector for Spark 10.2
 --------------------------------
 

source/streaming-mode/streaming-read-config.txt

@@ -46,6 +46,26 @@ You can configure the following properties when reading data from MongoDB in str
    * - ``collection``
      - | **Required.**
        | The collection name configuration.
+       | You can specify multiple collections by separating each collection name
+         with a comma (,). Do not add a space between the collections
+         unless the space is a part of the collection name.
+       | Specify multiple collections as follows:
+
+       .. code-block:: java
+
+          ...
+          .option("spark.mongodb.collection", "collectionOne,collectionTwo")
+
+       | You can scan from all collections in the database by using an asterisk (*).
+       | Specify all collections as follows:
+
+       .. code-block:: java
+
+          ...
+          .option("spark.mongodb.collection", "*")
+
+       If you create a collection while scanning from all collections, it is
+       automatically picked up for scanning.
 
    * - ``comment``
      - | The comment to append to the read operation. Comments appear in the 
@@ -168,7 +188,7 @@ You can configure the following properties when reading a change stream from Mon
          omit the ``fullDocument`` field and publishes only the value of the
          field.
        - If you don't specify a schema, the connector infers the schema
-         from the change stream document rather than from the underlying collection.
+         from the change stream document.
 
        **Default**: ``false``
 

source/streaming-mode/streaming-read.txt

@@ -15,6 +15,13 @@ Read from MongoDB in Streaming Mode
    :depth: 1
    :class: singlecol 
 
+.. facet::
+   :name: genre
+   :values: reference
+
+.. meta::
+   :keywords: change stream
+
 Overview
 --------
 
@@ -344,12 +351,13 @@ The following example shows how to stream data from MongoDB to your console.
 
 .. important:: Inferring the Schema of a Change Stream
 
-   When the {+connector-short+} infers the schema of a DataFrame
-   read from a change stream, by default,
-   it uses the schema of the underlying collection rather than that
-   of the change stream. If you set the ``change.stream.publish.full.document.only``
-   option to ``true``, the connector uses the schema of the 
-   change stream instead.
+   If you set the ``change.stream.publish.full.document.only``
+   option to ``true``, the {+connector-short+} infers the schema of a ``DataFrame``
+   by using the schema of the scanned documents. If you set the option to
+   ``false`` you must specify a schema.
+
+   Schema inference happens at the beginning of scanning, and does not take into
+   account collections that are created while scanning.
 
    For more information about this setting, and to see a full list of change stream
    configuration options, see the