Skip to content

DOCSP-43088: Aggregations #38

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Sep 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
188 changes: 188 additions & 0 deletions source/aggregation.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,188 @@
.. _c-aggregation:

====================================
Transform Your Data with Aggregation
====================================

.. facet::
:name: genre
:values: reference

.. meta::
:keywords: code example, transform, computed, pipeline
:description: Learn how to use the C driver to perform aggregation operations.

.. contents:: On this page
:local:
:backlinks: none
:depth: 2
:class: singlecol

Overview
--------

In this guide, you can learn how to use the {+driver-short+} to perform
**aggregation operations**.

You can use aggregation operations to process data in your MongoDB collections and
return computed results. The MongoDB Aggregation framework, which is
part of the Query API, is modeled on the concept of a data processing
pipeline. Documents enter a pipeline that contains one or more stages,
and each stage transforms the documents to output a final aggregated result.

You can think of an aggregation operation as similar to a car factory. A car factory has
an assembly line, which contains assembly stations with specialized
tools to do specific jobs, like drills and welders. Raw parts enter the
factory, and then the assembly line transforms and assembles them into a
finished product.

The **aggregation pipeline** is the assembly line, **aggregation stages** are the
assembly stations, and **operator expressions** are the
specialized tools.

Compare Aggregation and Find Operations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You can use find operations to perform the following actions:

- Select which documents to return
- Select which fields to return
- Sort the results

You can use aggregation operations to perform the following actions:

- Perform find operations
- Rename fields
- Calculate fields
- Summarize data
- Group values

Limitations
~~~~~~~~~~~

The following limitations apply when using aggregation operations:

- Returned documents must not violate the
:manual:`BSON document size limit </reference/limits/#mongodb-limit-BSON-Document-Size>`
of 16 megabytes.
- Pipeline stages have a memory limit of 100 megabytes by default. You can exceed this
limit by setting the ``allowDiskUse`` option to ``true``.

.. important:: $graphLookup exception

The :manual:`$graphLookup
</reference/operator/aggregation/graphLookup/>` stage has a strict
memory limit of 100 megabytes and ignores the ``allowDiskUse`` option.

Aggregation Example
-------------------

The examples in this section use the ``restaurants`` collection in the ``sample_restaurants``
database from the :atlas:`Atlas sample datasets </sample-data>`. To learn how to create a
free MongoDB Atlas cluster and load the sample datasets, see the
:atlas:`Get Started with Atlas </getting-started>` guide.

Build and Execute an Aggregation Pipeline
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To perform an aggregation on the documents in a collection, pass a ``bson_t`` structure
that represents the pipeline stages to the ``mongoc_collection_aggregate()`` function.

This example outputs a count of the number of bakeries in each borough
of New York City. The following code creates an aggregation pipeline that contains the
following stages:

- A :manual:`$match </reference/operator/aggregation/match/>` stage to filter for documents
in which the value of the ``cuisine`` field is ``"Bakery"``.

- A :manual:`$group </reference/operator/aggregation/group/>` stage to group the matching
documents by the ``borough`` field, producing a count of documents for each distinct
value of that field.

.. io-code-block::

.. input:: /includes/aggregation/aggregation.c
:language: c
:start-after: start-aggregation-pipeline
:end-before: end-aggregation-pipeline
:dedent:

.. output::
:visible: false

{ "_id" : "Queens", "count" : { "$numberInt" : "204" } }
{ "_id" : "Staten Island", "count" : { "$numberInt" : "20" } }
{ "_id" : "Missing", "count" : { "$numberInt" : "2" } }
{ "_id" : "Bronx", "count" : { "$numberInt" : "71" } }
{ "_id" : "Brooklyn", "count" : { "$numberInt" : "173" } }
{ "_id" : "Manhattan", "count" : { "$numberInt" : "221" } }

Explain an Aggregation
~~~~~~~~~~~~~~~~~~~~~~

To view information about how MongoDB executes your operation, you can
run the the ``explain`` operation on your pipeline. When MongoDB explains an
operation, it returns **execution plans** and performance statistics. An execution
plan is a potential way MongoDB can complete an operation.
When you instruct MongoDB to explain an operation, it returns both the
plan MongoDB selected for the operation and any rejected execution plans.

The following code example runs the same aggregation shown in the preceding section, but
uses the ``mongoc_client_command_simple()`` function to explain the operation details:

.. io-code-block::

.. input:: /includes/aggregation/aggregation.c
:language: c
:start-after: start-aggregation-explain
:end-before: end-aggregation-explain
:dedent:

.. output::
:visible: false

{
"explainVersion": "2",
"queryPlanner": {
"namespace": "sample_restaurants.restaurants"
"indexFilterSet": false,
"parsedQuery": {
"cuisine": {"$eq": "Bakery"}
},
"queryHash": "865F14C3",
"planCacheKey": "0697561B",
"optimizedPipeline": true,
"maxIndexedOrSolutionsReached": false,
"maxIndexedAndSolutionsReached": false,
"maxScansToExplodeReached": false,
"winningPlan": { ... },
"rejectedPlans": []
...
}
...
}

Additional Information
----------------------

To view a full list of expression operators, see :manual:`Aggregation
Operators </reference/operator/aggregation/>` in the {+mdb-server+} manual.

To learn about assembling an aggregation pipeline and view examples, see
:manual:`Aggregation Pipeline </core/aggregation-pipeline/>` in the {+mdb-server+} manual.

To learn more about creating pipeline stages, see :manual:`Aggregation
Stages </reference/operator/aggregation-pipeline/>` in the {+mdb-server+} manual.

To learn more about explaining MongoDB operations, see
:manual:`Explain Output </reference/explain-results/>` and
:manual:`Query Plans </core/query-plans/>` in the {+mdb-server+} manual.

API Documentation
~~~~~~~~~~~~~~~~~

For more information about executing aggregation operations with the {+driver-short+},
see the following API documentation:

- `mongoc_collection_aggregate() <{+api-libmongoc+}/mongoc_collection_aggregate.html>`__
- `mongoc_client_command_simple() <{+api-libmongoc+}/mongoc_client_command_simple.html>`__
2 changes: 1 addition & 1 deletion source/get-started.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
.. _c-get-started:

=============================
Get Started with the {+driver-short+}
Get Started with the C Driver
=============================

.. contents:: On this page
Expand Down
83 changes: 83 additions & 0 deletions source/includes/aggregation/aggregation.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
#include <bson/bson.h>
#include <mongoc/mongoc.h>
#include <stdio.h>

int
main (int argc, char *argv[])
{
mongoc_client_t *client;
mongoc_collection_t *collection;
mongoc_init ();

client =
mongoc_client_new ("<connection string URI>");
collection = mongoc_client_get_collection (client, "sample_restaurants", "restaurants");

{
// Executes an aggregation pipeline containing the $match and $group stages and prints the results
// start-aggregation-pipeline
const bson_t *doc;
bson_t *pipeline = BCON_NEW ("pipeline",
"[",
"{", "$match", "{", "cuisine", BCON_UTF8 ("Bakery"), "}", "}",
"{", "$group", "{",
"_id", BCON_UTF8 ("$borough"), "count", "{", "$sum", BCON_INT32 (1), "}", "}",
"}",
"]");

mongoc_cursor_t *results =
mongoc_collection_aggregate (collection, MONGOC_QUERY_NONE, pipeline, NULL, NULL);

bson_error_t error;
if (mongoc_cursor_error (results, &error))
{
fprintf (stderr, "Aggregate failed: %s\n", error.message);
} else {
while (mongoc_cursor_next (results, &doc)) {
char *str = bson_as_canonical_extended_json (doc, NULL);
printf ("%s\n", str);
bson_free (str);
}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If error handling is relevant to the example (as it is below) suggest adding:

bson_error_t error;
if (mongoc_cursor_error(results, &error))
{
    fprintf(stderr, "Aggregate failed: %s\n", error.message);
}


bson_destroy (pipeline);
mongoc_cursor_destroy (results);
// end-aggregation-pipeline
}

{
// Runs a command to explain the logic behind the aggregation
// start-aggregation-explain
bson_t reply;
bson_error_t error;

bson_t *command = BCON_NEW (
"aggregate", BCON_UTF8 ("restaurants"),
"explain", BCON_BOOL(true),
"pipeline",
"[",
"{", "$match", "{", "cuisine", BCON_UTF8("Bakery"), "}", "}",
"{", "$group", "{",
"_id", BCON_UTF8("$borough"), "count", "{", "$sum", BCON_INT32(1), "}", "}",
"}",
"]");

if (mongoc_client_command_simple (client, "sample_restaurants", command, NULL, &reply, &error)) {
char *str = bson_as_canonical_extended_json (&reply, NULL);
printf ("%s\n", str);
bson_free (str);
} else {
fprintf (stderr, "Command failed: %s\n", error.message);
}

bson_destroy (command);
bson_destroy (&reply);
// end-aggregation-explain
}

mongoc_collection_destroy (collection);
mongoc_client_destroy (client);
mongoc_cleanup ();

return EXIT_SUCCESS;
}
16 changes: 6 additions & 10 deletions source/index.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,10 @@
:titlesonly:
:maxdepth: 1

/get-started
Get Started </get-started>
/databases-collections
/read
/aggregation
/whats-new
libbson API Documentation <{+api-libbson+}>
libmongoc API Documentation <{+api-libmongoc+}>
Expand Down Expand Up @@ -65,16 +66,11 @@ Learn how you can retrieve data from MongoDB in the :ref:`c-read` section.
.. Learn how to work with common types of indexes in the :ref:`c-indexes`
.. section.

.. TODO
.. Transform Your Data with Aggregation
.. ------------------------------------

.. Learn how to use the {+driver-short+} to perform aggregation operations in the
.. :ref:`c-aggregation` section.

.. Learn how to use aggregation expression operations to build
.. aggregation stages in the :ref:`c-aggregation-expression-operations` section.
Transform Your Data with Aggregation
------------------------------------

Learn how to use the {+driver-short+} to perform aggregation operations in the
:ref:`c-aggregation` section.

.. TODO:
.. FAQ
Expand Down
Loading