Pipelines
- Pubsub data producer, 1 message per second
- Pubsub data consumer, 10s sliding window, for every 1s log a sample message per window + count number of messages per window
For load tests, use this data producer to dump messages to Pubsub
Simply change parameters defined in this makefile, for example, project id, pubsub topic name, pubsub subscription name etc.
then make df will launch the dataflow job, or make up will update the current running job
