ONS service that imports uploaded images and adds them to a private bucket
- Run
make debug
The service runs in the background consuming messages from Kafka. The messages are produced by the
image API and an example image can be created using the helper script,
make produce.
- Requires running…
- No further dependencies other than those defined in
go.mod
| Environment variable | Default | Description |
|---|---|---|
| BIND_ADDR | :24800 | The host and port to bind to |
| SERVICE_AUTH_TOKEN | - | The service token for this app |
| AWS_REGION | eu-west-1 | The AWS region |
| GRACEFUL_SHUTDOWN_TIMEOUT | 5s | The graceful shutdown timeout in seconds (time.Duration format) |
| HEALTHCHECK_INTERVAL | 30s | Time between self-healthchecks (time.Duration format) |
| HEALTHCHECK_CRITICAL_TIMEOUT | 90s | Time to wait until an unhealthy dependent propagates its state to make this app unhealthy (time.Duration format) |
| IMAGE_API_URL | http://localhost:24700 | The image api url |
| KAFKA_ADDR | localhost:9092 |
The address of Kafka brokers (comma-separated values) |
| KAFKA_VERSION | 1.0.2 |
The version of Kafka |
| KAFKA_SEC_PROTO | unset (only TLS) |
if set to TLS, kafka connections will use TLS |
| KAFKA_SEC_CLIENT_KEY | unset | PEM [2] for the client key (optional, used for client auth) [1] |
| KAFKA_SEC_CLIENT_CERT | unset | PEM [2] for the client certificate (optional, used for client auth) [1] |
| KAFKA_SEC_CA_CERTS | unset | PEM [2] of CA cert chain if using private CA for the server cert [1] |
| KAFKA_SEC_SKIP_VERIFY | false | ignore server certificate issues if set to true [1] |
| KAFKA_CONSUMER_WORKERS | 1 | The maximum number of parallel kafka consumers |
| IMAGE_UPLOADED_GROUP | dp-image-importer | The consumer group this application to consume ImageUploaded messages |
| IMAGE_UPLOADED_TOPIC | image-uploaded | The name of the topic to consume messages from |
| S3_PRIVATE_BUCKET_NAME | csv-exported | Name of the S3 bucket used to store generated images |
| S3_UPLOADED_BUCKET_NAME | dp-frontend-florence-file-uploads | Name of the S3 bucket used to read original images from |
| DOWNLOAD_SERVICE_URL | http://localhost:23600 | The public address of the download service |
Notes:
- For more info, see the kafka TLS examples documentation
The /health endpoint returns the current status of the service. Dependent services are health checked on an interval defined by the HEALTHCHECK_INTERVAL environment variable.
On a development machine a request to the health check endpoint can be made by:
curl localhost:24800/health
See CONTRIBUTING for details.
Copyright © 2021, Office for National Statistics (https://www.ons.gov.uk)
Released under MIT license, see LICENSE for details.