v0.26.0

0.26.0

Features

• Gas metering #2414
• Adds support for Solidity Custom Errors #2577
• Debug fork tool #2995 #3292
• Automatically remove unused deployments #3023
• Fix fulltextsearch space handling #3048
• Allow placing new deployments onto one of several shards #3049
• Make NEAR subgraphs update their sync status #3108
• GraphQL validations #3164
• Add special treatment for immutable entities #3201
• Tendermint integration #3212
• Skip block updates when triggers are empty #3223 #3268
• Use new GraphiQL version #3252
• GraphQL prefetching #3256
• Allow using Bytes as well as String/ID for the id of entities #3271
• GraphQL route for dumping entity changes in subgraph and block #3275
• Firehose filters #3323
• NEAR filters #3372

Robustness

• Improve our CacheWeight estimates #2935
• Refactor GraphQL execution #3005
• Setup databases in parallel #3019
• Block ingestor now fetches receipts in parallel #3030
• Prevent subscriptions from back-pressuring the notification queue #3053
• Avoid parsing X triggers if the filter is empty #3083
• Pipeline BlockStream #3085
• More robust proofOfIndexing GraphQL route #3348

graphman

• Add run command, for running a subgraph up to a block #3079
• Add analyze command, for analyzing a PostgreSQL table, which can improve performance #3170
• Add index create command, for adding an index to certain attributes #3175
• Add index list command, for listing indexes #3198
• Add index drop command, for dropping indexes #3198

Dependency Updates

These are the main ones:

• Updated protobuf to latest version for NEAR #2947
• Update web3 crate #2916 #3120 #3338
• Update graphql-parser to v0.4.0 #3020
• Bump itertools from 0.10.1 to 0.10.3 #3037
• Bump clap from 2.33.3 to 2.34.0 #3039
• Bump serde_yaml from 0.8.21 to 0.8.23 #3065
• Bump tokio from 1.14.0 to 1.15.0 #3092
• Bump indexmap from 1.7.0 to 1.8.0 #3143
• Update ethabi to its latest version #3144
• Bump structopt from 0.3.25 to 0.3.26 #3180
• Bump anyhow from 1.0.45 to 1.0.53 #3182
• Bump quote from 1.0.9 to 1.0.16 #3112 #3183 #3384
• Bump tokio from 1.15.0 to 1.16.1 #3208
• Bump semver from 1.0.4 to 1.0.5 #3229
• Bump async-stream from 0.3.2 to 0.3.3 #3361
• Update jsonrpc-server #3313

Misc

• More context when logging RPC calls #3128
• Increase default reorg threshold to 250 for Ethereum #3308
• Improve traces error logs #3353
• Add warning and continue on parse input failures for Ethereum #3326

Upgrade Notes

When upgrading to this version, we recommend taking a brief look into these changes:

• Gas metering #2414
  • Now there's a gas limit for subgraph mappings, if the limit is reached the subgraph will fail with a non-deterministic error, you can make them recover via the environment variable GRAPH_MAX_GAS_PER_HANDLER
• Improve our CacheWeight estimates #2935
  • This is relevant because a couple of releases back we've added a limit for the memory size of a query result. That limit is based of the CacheWeight.

These are some of the features that will probably be helpful for indexers 😊

• Allow placing new deployments onto one of several shards #3049
• GraphQL route for dumping entity changes in subgraph and block #3275
• Unused deployments are automatically removed now #3023
  • The interval can be set via GRAPH_REMOVE_UNUSED_INTERVAL
• Setup databases in parallel #3019
• Block ingestor now fetches receipts in parallel #3030
  • GRAPH_ETHEREUM_FETCH_TXN_RECEIPTS_IN_BATCHES can be set to true for the old fetching behavior
• More robust proofOfIndexing GraphQL route #3348
  • A token can be set via GRAPH_POI_ACCESS_TOKEN to limit access to the POI route
• The new graphman commands 🙂

v0.25.2

This release includes two changes:

• Bug fix of blocks being skipped from processing when: a deterministic error happens and the index-node gets restarted. Issue #3236, Pull Request: #3316.
• Automatic retries for non-deterministic errors. Issue #2945, Pull Request: #2988.

This is the last patch on the 0.25 minor version, soon 0.26.0 will be released. While that we recommend updating to this version to avoid determinism issues that could be caused on graph-node restarts.

v0.25.1

This release only adds two fixes:

• The first is to address an issue with decoding the input of some calls #3194 where subgraphs that would try to index contracts related to those would fail. Now they can advance normally.
• The second one is to fix a non-determinism issue with the retry mechanism for errors. Whenever a non-deterministic error happened, we would keep retrying to process the block, however we should've clear the EntityCache on each run so that the error entity changes don't get transacted/saved in the database in the next run. This could make the POI generation non-deterministic for subgraphs that failed and retried for non-deterministic reasons, adding a new entry to the database for the POI.

We strongly recommend updating to this version as quickly as possible.

v0.25.0

Api Version 0.0.6

This release ships support for API version 0.0.6 in mappings:

• Added nonce field for Transaction objects.
• Added baseFeePerGas field for Block objects (EIP-1559).

Block Cache Invalidation and Reset

All cached block data must be refetched to account for the new Block and Trasaction
struct versions, so this release includes a graph-node startup check that will:

1. Truncate all block cache tables.
2. Bump the db_version value from 2 to 3.

(Table truncation is a fast operation and no downtime will occur because of that.)

Ethereum

• 'Out of gas' errors on contract calls are now considered deterministic errors,
  so they can be handled by try_ calls. The gas limit is 50 million.

Environment Variables

• The GRAPH_ETH_CALL_GAS environment is removed to prevent misuse, its value
  is now hardcoded to 50 million.

Multiblockchain

• Initial support for NEAR subgraphs.
• Added FirehoseBlockStream implementation of BlockStream (#2716)

Misc

• Rust docker image is now based on Debian Buster.
• Optimizations to the PostgreSQL notification queue.
• Improve PostgreSQL robustness in multi-sharded setups. (#2815)
• Added 'networks' to the 'subgraphFeatures' endpoint. (#2826)
• Check and limit the size of GraphQL query results. (#2845)
• Allow _in and _not_in GraphQL filters. (#2841)
• Add PoI for failed subgraphs. (#2748)
• Make graphman rewind safer to use. (#2879)
• Add subgraphErrors for all GraphQL schemas. (#2894)
• Add Graph-Attestable response header. (#2946)
• Add support for minimum block constraint in GraphQL queries (number_gte) (#2868).
• Handle revert cases from Hardhat and Ganache (#2984)
• Fix bug on experimental prefetching optimization feature (#2899)

v0.24.2

This release only adds a fix for an issue where certain GraphQL queries
could lead to graph-node running out of memory even on very large
systems. This release adds code that checks the size of GraphQL responses
as they are assembled, and can warn about large responses in the logs
resp. abort query execution based on the values of the two new environment
variables GRAPH_GRAPHQL_WARN_RESULT_SIZE and
GRAPH_GRAPHQL_ERROR_RESULT_SIZE. It also adds Prometheus metrics
query_result_size and query_result_max to track the memory consumption
of successful GraphQL queries. The unit for the two environment variables
is bytes, based on an estimate of the memory used by the result; it is best
to set them after observing the Prometheus metrics for a while to establish
what constitutes a reasonable limit for them.

We strongly recommend updating to this version as quickly as possible.

v0.24.1

Feature Management

This release supports the upcoming Spec Version 0.0.4 that enables subgraph features to be declared in the manifest and
validated during subgraph deployment
#2682
#2746.

Subgraphs using previous versions are still supported and won't be affected by this change.

New Indexer GraphQL query: subgraphFeatures

It is now possible to query for the features a subgraph uses given its Qm-hash ID.

For instance, the following query...

{
  subgraphFeatures(subgraphId: "QmW9ajg2oTyPfdWKyUkxc7cTJejwdyCbRrSivfryTfFe5D") {
    features
    errors
  }
}

... would produce this result:

{
  "data": {
    "subgraphFeatures": {
      "errors": [],
      "features": [
        "nonFatalErrors",
        "ipfsOnEthereumContracts"
      ]
    }
  }
}

Subraphs with any Spec Version can be queried that way.

Api Version 0.0.5

• Added better error message for null pointers in the runtime #2780.

Environment Variables

• When GETH_ETH_CALL_ERRORS_ENV is unset, it doesn't make eth_call errors to be considered determinsistic anymore #2784

Robustness

• Tolerate a non-primary shard being down during startup #2727.
• Check that at least one replica for each shard has a non-zero weight #2749.
• Reduce locking for the chain head listener #2763.

Logs

• Improve block ingestor error reporting for missing receipts #2743.

v0.24.0

0.24

Api Version 0.0.5

This release ships support for API version 0.0.5 in mappings. It contains a fix for call handlers
and the long awaited AssemblyScript version upgrade!

• AssemblyScript upgrade: The mapping runtime is updated to support up-to-date versions of the
  AssemblyScript compiler. The graph-cli/-ts releases to support this are in alpha, soon they will
  be released along with a migration guide for subgraphs.
• Call handlers fix: Call handlers will never be triggered on transactions with a failed status,
  resolving issue #2409. Done in #2511.

Logs

• The log "Skipping handler because the event parameters do not match the event signature." was downgraded from info to trace level.
• Some block ingestor error logs were upgrded from debug to info level #2666.

Metrics

• query_semaphore_wait_ms is now by shard, and has the pool and shard labels.
• deployment_failed metric added, it is 1 if the subgraph has failed and 0 otherwise.

Other

• Upgrade to tokio 1.0 and futures 0.3 #2679, the first major contribution by StreamingFast!
• Support Celo block reward events #2670.
• Reduce the maximum WASM stack size and make it configurable #2719.
• For robustness, ensure periodic updates to the chain head listener #2725.

v0.23.1

• Fix ipfs timeout detection #2584.
• Fix discrepancy between a database table and its Diesel model #2586.

v0.23.0

The Graph Node internals are being heavily refactored to prepare it for the multichain future.
In the meantime, here are the changes for this release:

• The GRAPH_ETH_CALL_BY_NUMBER environment variable has been removed. Graph Node requires an
  Ethereum client that supports EIP-1898, which all major clients support.
• Added support for IPFS versions larger than 0.4. Several changes to make
  graph-node more tolerant of slow/flaky IPFS nodes.
• Added Ethereum ABI encoding and decoding functionality #2348.
• Experimental support for configuration files, see the documentation here.
• Better PoI performance #2329.
• Improve grafting performance and robustness by copying in batches #2293.
• Subgraph metadata storage has been simplified and reorganized. External
  tools (e.g., Grafana dashboards) that access the database directly will need to be updated.
• Ordering in GraphQL queries is now truly reversible
  #2214
• The GRAPH_SQL_STATEMENT_TIMEOUT environment variable can be used to
  enforce a timeout for individual SQL queries that are run in the course of
  processing a GraphQL query
  #2285
• Using ethereum.call in mappings in globals is deprecated

Graphman

Graphman is a CLI tool to manage your subgraphs. It is now included in the Docker container
#2289. And new commands have been added:

• graphman copy can copy subgraphs across DB shards #2313.
• graphman rewind to rewind a deployment to a given block #2373.
• graphman query to log info about a GraphQL query #2206.
• graphman create to create a subgraph name #2419.

Metrics

• The deployment_blocks_behind metric has been removed, and a
  deployment_head metric has been added. To see how far a deployment is
  behind, use the difference between ethereum_chain_head_number and
  deployment_head.
• The trigger_type label was removed from the metric deployment_trigger_processing_duration.

v0.22.0

Feature: Block store sharding

This release makes it possible to shard the block and call cache for chain
data across multiple independent Postgres databases. This feature is considered experimental. We
encourage users to try this out in a test environment, but do not recommend it yet for production
use. In particular, the details of how sharding is configured may change in backwards-incompatible
ways in the future.

Feature: Non-fatal errors update

Non-fatal errors (see release 0.20 for details) is documented and can now be enabled on graph-cli.
Various related bug fixes have been made #2121 #2136 #2149 #2160.

Improvements

• Add bitwise operations and string constructor to BigInt #2151.
• docker: Allow custom ethereum poll interval #2139.
• Deterministic error work in preparation for gas #2112

Bug fixes

• Fix not contains filter #2146.
• Resolve __typename in _meta field #2118
• Add CORS for all HTTP responses #2196