Open Raadsinformatie API

Open Raadsinformatie (ORI) aims to collect and standardize governmental decision making documents of Dutch municipalities (gemeenten, provincies, waterschappen). Open Raadsinformatie is a collaborative effort of the Open State Foundation, Ontola and VNG Realisatie.

Important links

• Docs for API
• Search engine
• Open Raadsinformatie homepage
• Official source code repository
• Issue tracker

Installation and usage

Clone the git repository:

• git clone https://github.com/openstate/open-raadsinformatie.git
• cd open-raadsinformatie/
• In development:
  • sudo docker compose --compatibility -f docker-compose.yml -f docker-compose.dev.yml up --build -d
• In production:
  • sudo docker compose --compatibility -f docker-compose.yml up --build -d
• After first time installation on a server, run ocd_backend/setup.sh to create the database.

Latest docker version uses "-" as separator when creating container names instead of "_". You can still get the old functionality by adding "--compatibility" to docker compose. So when using the latest Docker add --compatibility.

Check out the maintenance guide for more info on how to manage this project in production.

The Nginx container was installed separately in production. To mimic this in development, clone https://github.com/openstate/nginx-load-balancer/, follow the instructions in INSTALL.txt and start the container with docker compose --compatibility up -d

The log file was made persistent and is located in /data. To prevent this file from growing indefinitely, add the following to /etc/logrotate.d/orilog:

/data/ori.log
{
    rotate 100
    size 50M
    missingok
    notifempty
    compress
    delaycompress
    copytruncate
}

You can test it using sudo logrotate -d /etc/logrotate.d/orilog

In development Flower provides insight in the queues of Celery. You can access the Flower dashboard via http://localhost:81/workers.

Enabling ssl

The DNS for openraadsinformatie.nl is not maintained by us in TransIP but by Ontola. Enabling SSL therefore works differently than described in https://github.com/openstate/nginx-load-balancer/blob/master/INSTALL_HTTPS.txt.

• Start requesting certificate
  • sudo docker exec -it docker-c-certbot-1 sh
  • certbot certonly -m [email protected] --manual --agree-tos --preferred-challenges http -vvv -d api.openraadsinformatie.nl
• While the above command is hanging on Press Enter to Continue do the following to setup the challenge
  • sudo docker exec -it docker-c-nginx-load-balancer-1 sh
  • mkdir -p /usr/share/nginx/html/.well-known/acme-challenge
  • In this directory create the file with contents as described by the certbot output.
• For the challenge to succeed, port 80 needs to be accessible and serving the right content. So in the nginx-load-balancer project edit the file default.conf and in the server block for api.openraadsinformatie.nl:

  • Comment out the line return 301 ...

  • Add the following:

      location ^~ /.well-known/acme-challenge/ {
         root /usr/share/nginx/html/;
      }
    

  • sudo docker exec docker-c-nginx-load-balancer-1 nginx -t

  • sudo docker exec docker-c-nginx-load-balancer-1 nginx -s reload

• In the terminal hanging on Press Enter to Continue press Enter. The certificate will be downloaded to /etc/letsencrypt/live.
• Finally, undo the above changes to default.conf, add the new SSL certificate in the right server block and test/restart nginx.

Storage

Using HDDs for the Elasticsearch index was found to be too slow for the production database. In september 2025 we switched to a server having 2 SSDs (3.5TB/RAID 1) and 2HDDs (14.6T/RAID 1). The mounts are as follows:

• /: SSDs
• /data: HDDs

The Docker volumes, containing the Elasticsearch index and the PostgreSQL database, are placed on the SSDs. The backend and loader Docker containers, which before also used a Docker volume, were connected to /data, the HDDs, for mass storage of the PDFs.

Import a municipality in development:

The following commands build and start the Docker containers, empty the PostgreSQL and Redis databases and Elastic Search index for a fresh start and then import a municipality for a certain date range. Change the start_date, end_date and source_path as desired.

• docker compose --compatibility -f docker-compose.yml -f docker-compose.dev.yml up --build -d
• docker exec ori_backend_1 bin/purge_dbs.sh
• docker exec ori_redis_1 redis-cli -n 1 set _all.start_date "2025-01-14"
• docker exec ori_redis_1 redis-cli -n 1 set _all.end_date "2025-01-15"
• docker exec ori_backend_1 ./manage.py extract load_redis 'all daily monthly'
• docker exec ori_backend_1 ./manage.py extract process all --source_path=ori.notubiz.haarlem

Testing

The next lines were copied from the Github workflow (which never actually ran):

• docker compose --compatibility -f docker-compose.yml -f docker-compose.test.yml up --build -d
• docker exec ori_backend_1 bin/run_tests.sh 2>&1
• docker exec ori_backend_1 pylint ocd_backend -E -sy To run a single test, e.g.
• docker exec ori_backend_1 bin/run_tests.sh tests.ocd_backend.utils.test_document_storage

Getting data in development

See script manual_retrieval.py (WIP) See also the Troubleshooting section in the maintenance_guide.

To get data from iBabs in development you need to use a proxy:

• edit /etc/hosts and add a line linking your IP address to host.docker.internal, e.g.: 192.168.121.174 host.docker.internal
• start proxy with ssh -gD 8090 ori-octopus
• PROXY_HOST and PROXY_PORT are always set in development (docker-compose-dev.yml)

Supported Sources

Data extraction support is available for the following source systems:

• GO GemeenteOplossingen
• Notubiz
• iBabs
• Parlaeus
• Tweede Kamer

How to add your municipality

Get in touch with Sander Bakker from VNG Realisatie. Your griffie (municipality clerk) needs to formally agree that the data becomes open data, and the source system might need some configuration.

Related repositories

• openbesluitvorming, the new search interface.

Contributing

Please read through our contributing guidelines. Included are directions for opening issues, coding standards, and notes on development.

Bugs and feature requests

Have a bug or a feature request? Please first read through and search for existing and closed issues. If your problem or idea is not addressed yet, please open a new issue.

Authors and contributors

The Open Raadsinformatie API was originally based on the Open Cultuur Data API.

Authors and contributors of both projects are:

• Bart de Goede (@bartdegoede)
• Justin van Wees (@justin_v_w)
• Breyten Ernsting (@breyten)
• Sicco van Sas (@siccovansas)
• Jurrian Tromp (@jurrian, @ontola)
• Jorrit van Belzen (@jorritb, @ontola)
• Joep Meindertsma (@joepio)
• Rob van Dijk (@robvandijk)

Contributors:

• DutchCoders
• Benno Kruit

Copyright and license

The Open Raadsinformatie project is distributed under the MIT license.