Merge branch 'master' of https://github.com/datacommonsorg/docsite

kmoscoe · kmoscoe · commit d80ed795d69b · 2026-01-13T11:03:03.000-08:00
diff --git a/custom_dc/build_image.md b/custom_dc/build_image.md
@@ -17,9 +17,11 @@ While you are just testing out data changes, you don't need to build the website
 
 Data Commons provides two prebuilt images in the Google Artifact Registry that you can download to run in a Docker container:
 
-- `gcr.io/datcom-ci/datacommons-data:stable` and `gcr.io/datcom-ci/datacommons-services:stable`. These are tested, stable versions but may be several weeks old.
+- `gcr.io/datcom-ci/datacommons-data:stable` and `gcr.io/datcom-ci/datacommons-services:stable`. These are tested, stable versions but may be several weeks old. 
 - `gcr.io/datcom-ci/datacommons-data:latest` and `gcr.io/datcom-ci/datacommons-services:latest`. These are the latest versions built from head.
 
+You can see the images with their tags at <https://console.cloud.google.com/artifacts/docker/datcom-ci/us/gcr.io/datacommons-data> and <https://console.cloud.google.com/artifacts/docker/datcom-ci/us/gcr.io/datacommons-services>.
+
 If you want to pick up the latest prebuilt version, do the following:
 
 1. From the root directory (e.g. `website`), run the following command:
diff --git a/custom_dc/deploy_cloud.md b/custom_dc/deploy_cloud.md
@@ -51,9 +51,9 @@ The first time you run it, may be prompted to specify a quota project for billin
 <pre>
 gcloud auth application-default set-quota-project <var>PROJECT_ID</var></pre>
 
-## One-time setup: Create service accounts and enable all APIs
+## One-time setup: Enable APIs
 
-`website/deploy/terraform-custom-datacommons/setup.sh` is a convenience script to set up service account roles and all necessary Cloud APIs. To run it:
+`website/deploy/terraform-custom-datacommons/setup.sh` is a convenience script to set up all necessary Cloud APIs. To run it:
 
 <pre>
  cd website/deploy/terraform-custom-datacommons
@@ -80,6 +80,7 @@ We recommend using the Data Commons Terraform scripts to greatly simplify and au
 
 Terraform provisions and runs all the necessary Cloud Platform services:
 
+- Creates a service account for your project and namespace and assigns it various permissions ([IAM roles](https://docs.cloud.google.com/iam/docs/roles-overview){: target="_blank}).
 - Creates a Cloud Storage bucket and top-level folder, which will store your data files. You will upload your input data in the subsequent steps.
 - Creates a Cloud SQL MySQL instance, with basic resources, a default database user and a random password.
 - Creates the Data Commons data management container as a Cloud Run job, with basic resources.
@@ -262,7 +263,7 @@ To view the tables:
 1. In the left panel, select **Cloud SQL Studio**.
 1. In the **Sign in to SQL Studio** page, from the **Database** field, select the database created by the Terraform script.
 1. In the **User** field, select the user created by the Terraform script.
-1. In the **Password** field, enter the password you have retrieved from the Cloud Secret Manager
+1. In the **Password** field, enter the password you have retrieved from the Cloud Secret Manager.
 1. In the left Explorer pane that appears, expand the **Databases** icon, your database name, and **Tables**. The table of interest is **observations**. You can see column names and other metadata.
 1. To view the actual data, in the main window, click **New SQL Editor tab**. This opens an environment in which you can enter and run SQL queries.
 1. Enter a query and click **Run**. For example, for the sample OECD data, if you do `select * from observations limit 10;`, you should see output like this:
@@ -294,20 +295,19 @@ Any time you make changes to the website and want to deploy your changes to the
    If you don't specify the <code>--package</code> option, the package name and tag will be the same as the source image.
    </div>
     <div><ol><li>Build a local version of the Docker image, following the procedure in <a href="/custom_dc/build_image.html#build-repo">Build a local image</a>.</li>
-      <li>Generate credentials for the Docker package. 
+      <li>Generate credentials for the Docker package: 
     <pre>gcloud auth configure-docker <var>REGION</var>-docker.pkg.dev</pre></li>
    <li>Create a package from the source image you created in step 1:
     <pre>docker tag <var>SOURCE_IMAGE_NAME</var>:<var>SOURCE_IMAGE_TAG</var> \
    <var>REGION</var>-docker.pkg.dev/<var>PROJECT_ID</var>/<var>ARTIFACT_REPO</var>/<var>TARGET_IMAGE_NAME</var>:<var>TARGET_IMAGE_TAG</var></pre>
    The artifact repo is <code><var>PROJECT_ID</var>-artifacts</code>.</li>
    <li>Push the image to the registry:
    <pre>docker push <var>CONTAINER_IMAGE_URL</var></pre>
-    The container image URL is the full name of the package you created in the previous step, including the tag.</li>
+    The container image URL is the full name of the package you created in the previous step, including the tag. For example: `us-central1-docker-pkg.dev/myproject/myrepo/datacommons:latest`.</li>
   </ol>
    </div>
   </div>
 </div>
-
 - The target image name and tag can be the same as the source or different.
 - Docker package names must be in the format <code><var>REGION</var>-docker-pkg.dev</code>. The default region in the Terraform scripts is `us-central1`.
 
@@ -346,29 +346,22 @@ You need to restart the services container every time you make changes to the co
   <div class="active">
            <ol>
            <li>Go to the <a href="https://console.cloud.google.com/run/services" target="_blank">https://console.cloud.google.com/run/services</a> page for your project.</li>
-             <li>From the list of services, click the link of the service created by the Terraform scripts</li>
-             <li>click <b>Edit & Deploy Revision</b>.</li>
+             <li>From the list of services, click the link of the service created by the Terraform scripts.</li>
+             <li>Click <b>Edit & Deploy Revision</b>.</li>
            <li>Under <b>Container image URL</b>, click <b>Select</b>.</li>
            <li>Expand the package name you created in the previous step.</li>
            <li>Expand the image name of the container, and select the tag you created in the previous step.</li>
            <li>Click <b>Deploy</b>. It will take several minutes for the service to start. You can click the <b>Logs</b> tab to view the progress.</li>
         </ol>
       </div>
-    <div>
-      <li>From any local directory, run the following command:
-        <pre>gcloud run deploy <var>SERVICE_NAME</var> --image <var>CONTAINER_IMAGE_URL</var></pre></li>
-        <li>To view the startup status, run the following command:
+    <div><p>From any local directory, run the following command:
+      <pre>gcloud run deploy <var>SERVICE_NAME</var> --image <var>CONTAINER_IMAGE_URL</var></pre></p>
+      <p> To view the startup status, run the following command:
             <pre>gcloud beta run jobs logs tail <var>SERVICE_NAME</var></pre>
-          </li>
-          The service name is <code><var>NAMESPACE</var>-datacommons-web-service</code>.
-          The container image URL is the name of the package you created in the previous step.
-      </ol>
+    </p>
      </div>
-   <div>
-  </div>
+   </div>
   </div>
-</div>
-
 
 ### View your running application {#view-app}
 
@@ -403,6 +396,7 @@ To create additional deployments:
 cp terraform.tfvars terraform_prod.tfvars
 ```
 > Tip: You may wish to rename the original `terraform.tfvars` to something more descriptive as well.
+
 1. Do any of the following:
    - If you intend to run the new deployment in a different GCP project, edit the `project_id` variable and specify the project ID.
    - If you intend to run the new deployment in the same GCP project, edit the `namespace` variable to name it according to the environment you are creating, e.g. `-prod`. When you run the deployment, all created services will use the new namespace.
diff --git a/custom_dc/deploy_mcp_cloud.md b/custom_dc/deploy_mcp_cloud.md
@@ -0,0 +1,115 @@
+---
+layout: default
+title: Run an MCP server in Google Cloud
+nav_order: 9
+parent: Build your own Data Commons
+---
+
+{:.no_toc}
+# Run an MCP server in Google Cloud
+
+If you have built a custom agent or Gemini CLI extension which you want to make publicly available, this page describes how to run the [Data Commons MCP server](https://pypi.org/project/datacommons-mcp/) in the cloud, using Google Cloud Run. 
+
+Since setting up an MCP server is a simple, one-time setup, there's no need to use Terraform to manage it. Data Commons provides a prebuilt Docker image in the Artifact Registry, so you only need to set up a new Cloud Run service to point to it. 
+
+## Prebuilt images
+
+There are several versions of the image available, viewable at <https://console.cloud.google.com/artifacts/docker/datcom-ci/us/gcr.io/datacommons-mcp-server>. We recommend that you choose a production version with a specific version number, to ensure that changes introduced by the Data Commons team don't break your application.
+
+## Before you start: decide on a hosting model
+
+There are several ways you can host the MCP server in Cloud Run, namely:
+
+- As a standalone service. In this case, any client simply connects to it over HTTP, including your own MCP agent running as a separate Cloud Run service or locally. You can choose whether to make the internal Cloud Run app URL publicly available, or whether to put a load balancer in front of the service and map a domain name. 
+- As a ["sidecar"](https://docs.cloud.google.com/run/docs/deploying#sidecars){: target="_blank"} to an MCP client. If you are hosting your own MCP client in Cloud Run as well, this may be a useful option. In this case, the MCP server is not directly addressable; all external connections are managed by the client.
+
+In this page, we provide steps for running the Data Commons MCP server as a standalone container. If you want to go with the sidecar option, please see [Deploying multiple containers to a service (sidecars)](https://docs.cloud.google.com/run/docs/deploying#sidecars){: target="_blank"} for additional requirements and setup procedures.
+
+## Prerequisites
+
+The following procedures assume that you have set up the following Google Cloud Platform services, using the [Terraform scripts](deploy_cloud.md#terraform):
+- A service account and roles. 
+- A Google Cloud Secret Manager secret for storing your Data Commons API key. 
+
+## Create a Cloud Run Service for the MCP server
+
+The following procedure sets up a bare-bones container service. To set additional options, such as request timeouts, instance replication, etc., please see [Configure Cloud Run services](https://docs.cloud.google.com/run/docs/configuring){: target="_blank"} for details.
+
+<div class="gcp-tab-group">
+  <ul class="gcp-tab-headers">
+  <li class="active">Cloud Console</li>
+  <li>gcloud CLI</li>
+  </ul>
+  <div class="gcp-tab-content">
+  <div class="active">
+    <ol>
+      <li>Go to the <a href="https://console.cloud.google.com/run/services" target="_blank">https://console.cloud.google.com/run/services</a> page for your project.</li>
+      <li>Click <b>Deploy container</b>.</li>
+      <li>In the <b>Container image URL</b> field, click <b>Select</b>.</li>
+      <li>In the Artifact Registry panel that appears in the right side of the window, that appears, click <b>Change</b>.</li>
+      <li>In the project search bar, enter <code>datcom-ci</code> and click on the link that appears.</li>
+      <li>Expand <b>gcr.io/datcom-ci</b> and expand <b>datacommons-mcp-server</b>.</li>
+      <li>From the list of images, select a production image, e.g. <code>production-v1.1.4</code>.</li>
+      <li>Under <b>Configure</b>, select the desired region for the service, e.g. <code>us-central1</code>.</li> 
+      <li>Expand <b>Containers, Networking, Security</b>.</li>
+      <li>Click the <b>Variables & secrets</b> tab.</li>
+      <li>Under <b>Environment variables</b>, click <b>Add variable</b> and set the following variables:
+        <ul>
+          <li>name: <code>DC_TYPE</code>, value: <code>custom</code></li>
+          <li>name: <code>CUSTOM_DC_URL</code>, value: <code><var>YOUR_INSTANCE_URL</var></code></li>
+        </ul></li>
+      <li>Under <b>Secrets exposed as environment variables</b>, click <b>Reference a secret</b>.</li>
+      <li>In the <b>Name</b> field, enter <code>DC_API_KEY</code>, and from the <b>Secret</b> field, select  the secret previously created by the Terraform scripts. It is in the form <code><var>NAMESPACE</var>-datacommons-dc-api-key-<var>FINGERPRINT</var></code>.</li> 
+      <li>In the <b>Version</b> field, select the desired version, e.g. <b>latest</b>.</li>
+      <li>Click <b>Done</b>.</li>
+      <li>Click the <b>Security</b> tab. From the <b>Service account</b> field, select the service account for your namespace and project, previously created by the Terraform scripts.</li>
+      <li>Click <b>Create</b>. If correctly configured, the service will deploy automatically. It may take several minutes to start up.</li></ol>
+    </div>
+    <div>
+      <ol>
+        <li>If you haven't recently refreshed your Google Cloud credentials, run <code>gcloud auth application-default login</code> and authenticate.</li>
+        <li>From any local directory, run the following command:
+        <pre>gcloud run deploy datacommons-mcp-server --image <var>CONTAINER_IMAGE_URL</var> \
+        --service-account <var>SERVICE_ACCOUNT</var> --region <var>REGION</var> \
+        --allow-unauthenticated \
+        --set-secrets="DC_API_KEY=<var>SECRET_NAME</var>:latest" \
+        --set-env-vars="DC_TYPE=custom" --set-env-vars="CUSTOM_DC_URL=<var>INSTANCE_URL</var>"</pre></li>
+        </ol>
+        <ul>
+          <li>The container image URL is <code>gcr.io/datcom-ci/datacommons-mcp-server:<var>TAG</var></code>. The tag should be a production image with a version number, e.g. <code>production-v1.1.4</code>.</li>
+          <li>The service account was created when you ran Terraform. It is in the form <code><var>NAMESPACE</var>-datacommons-sa@<var>PROJECT_ID</var>.iam.gserviceaccount.com</code>.</li>
+          <li>The region is the Cloud region where you want to run the service, e.g. <code>us-central1</code>.</li>
+          <li>The secret name is the one created when you ran the Terraform scripts, in the form <code><var>NAMESPACE</var>-datacommons-dc-api-key-<var>FINGERPRINT</var></code>. If you're not sure about the name or fingerprint, go to <a href="https://console.cloud.google.com/security/secret-manager" target="_blank">https://console.cloud.google.com/security/secret-manager</a> for your project and look it up.</li>
+        </ul>
+    To view the startup status, run the following command:
+    <pre>gcloud run services logs tail datacommons-mcp-server --region <var>REGION</var></pre>
+    </div>
+  </div>
+</div>
+
+<script src="/assets/js/customdc-doc-tabs.js"></script>
+
+## Connect to the server from a remote client
+
+For details, see the following pages:
+- [Connect to the server from a local Gemini CLI client](/mcp/run_tools.html#gemini-cli-remote)
+- [Connect to the server from a local agent](/mcp/run_tools.html#remote)
+
+The HTTP URL parameter is the Cloud Run App URL, if you are exposing the service directly, or a custom domain URL if you are using a load balancer and domain mapping.
+
+## Troubleshoot deployment issues
+
+### Container fails to start
+
+If you see this error message:
+
+```
+The user-provided container failed to start and listen on the port defined provided by the PORT=8080 environment variable within the allocated timeout...
+```
+This is a generic message that could indicate a number of configuration problems. Check all of these:
+- Be sure you have specified the `DC_API_KEY` environment variable.
+- Be sure you have specified the correct service account.
+- Try increasing the health check timeout.
+
+
+
diff --git a/mcp/run_tools.md b/mcp/run_tools.md
@@ -15,7 +15,7 @@ This page shows you how to run a local agent and connect to a Data Commons MCP s
 
 We provide specific instructions for the following agents. All may be used to query datacommons.org or a [Custom Data Commons instance](/custom_dc).
 
-- [Gemini CLI extension](#use-the-gemini-cli-extension) 
+- [Gemini CLI extension](#extension) 
    - Best for querying datacommons.org
    - Provides a built-in "agent" and context file for Data Commons
    - Downloads extension files locally
@@ -222,7 +222,7 @@ To configure Gemini CLI to recognize the Data Commons server, edit the relevant
 </pre>
 
 {:.no_toc}
-### Configure to connect to a remote server
+### Configure to connect to a remote server {#gemini-cli-remote}
 
 1. Start up the MCP server in standalone mode, as described in [Run a standalone server](#run-a-standalone-server).
 1. In the `settings.json` file, replace the `datacommons-mcp` specification as follows:
@@ -298,9 +298,8 @@ By default, the agent will spawn a local server and connect to it over Stdio. If
    ```
 1. Enter your [queries](#sample-queries) at the `User` prompt in the terminal.
 
-{: #remote}
 {:.no_toc}
-### Configure to connect to a remote server
+### Configure to connect to a remote server {#remote}
 
 If you want to connect to a remote MCP server, follow this procedure before starting the agent:
 
@@ -343,6 +342,8 @@ Here are some examples of such queries:
 
 ## Run a standalone server
 
+The following procedure starts the MCP server in a local environment. To run the server in Google Cloud against a Custom Data Commons instance, see [Run an MCP server in Google Cloud](/custom_dc/deploy_mcp_cloud.html)
+
 1. Ensure you've set up the relevant server [environment variables](#configure-environment-variables). If you're using a `.env` file, go to the directory where the file is stored.
 1. Run:
    <pre>
