feat(extractor): stream MusicBrainz .tar.xz directly from network to β¦ #1183
Annotations
4 warnings and 2 notices
|
Complete job
Node.js 20 actions are deprecated. The following actions are running on Node.js 20 and may not work as expected: actions/setup-python@v5, astral-sh/setup-uv@e92bafb6253dcd438e0484186d7669ea7a8ca1cc. Actions will be forced to run with Node.js 24 by default starting June 2nd, 2026. Node.js 20 will be removed from the runner on September 16th, 2026. Please check if updated versions of these actions are available that support Node.js 24. To opt into Node.js 24 now, set the FORCE_JAVASCRIPT_ACTIONS_TO_NODE24=true environment variable on the runner or in your workflow file. Once Node.js 24 becomes the default, you can temporarily opt out by setting ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
π’ Send notification to Discord
embed field value must be shorter than 1024, got 1043
[`c60eef7`](https://github.com/SimplicityGuy/discogsography/commit/c60eef78acc396aefa61c39b6e2d4e1712ea3452) feat(extractor): stream MusicBrainz .tar.xz directly from network to .jsonl.xz
Replace the two-phase download-to-disk + extract pipeline with a single
streaming pass: reqwest bytes_stream β SyncIoBridge β HashingReader β
XzDecoder β tar β XzEncoder β disk. Source .tar.xz files never touch
disk, eliminating ~1 GB of I/O for the release tarball alone. Download
and decompression now run concurrently instead of sequentially.
Key changes:
- Add tokio-util (io-util feature) for StreamReader + SyncIoBridge
- Add HashingReader<R> to tee SHA256 through the byte stream
- Factor extract_entity_from_tarball into generic extract_entity_from_reader<R>
- Replace download_file + disk extract with stream_download_verify_extract
- Update tests for streaming error paths and no-tarball-on-disk assertions
Also includes Dockerfile base image updates and dependency bumps.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
|
π Build and push Docker image to GitHub Container Registry - discogsography/api:
api/Dockerfile#L113
SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ENV "JWT_SECRET_KEY")
More info: https://docs.docker.com/go/dockerfile/rule/secrets-used-in-arg-or-env/
|
|
π Build and push Docker image to GitHub Container Registry - discogsography/api:
api/Dockerfile#L113
SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ENV "POSTGRES_PASSWORD")
More info: https://docs.docker.com/go/dockerfile/rule/secrets-used-in-arg-or-env/
|
|
π Collect metrics
Docker cache hit for api
|
|
π Collect metrics
Service: api, Duration: 119s, Cache Used: true
|
background
wait
wait-all
cancel
Loading