Skip to content

Test data from producer not shown in event stream #213

@chimosky

Description

@chimosky

Problem Description
For some reason, osprey ui doesn't show the event stream when test-data is being produced, this data is being consumed as expected, but the event stream shows No data when a query is submitted.

This behaviour is inconsistent and I haven't been able to figure out why, some days ago it ran without issues on kafka-python v1.4.7, and also ran the first time I updated to kafka-python v2.3.0.

To Reproduce
Steps to reproduce the behavior:

  1. Run ./start.sh.
  2. Run docker compose --profile test-data up osprey-kafka-test-data-producer -d.
  3. Input query EventType == "create_post".
  4. Event stream shows no data.

Expected behavior
Results of the query is show in the event stream.

Screenshots/Recordings/Logs

On debugging, I found that the produced events are being consumed but not shown in the query result;

[appuser@osprey-kafka ~]$ kafka-console-consumer --bootstrap-server localhost:9092 --topic osprey.actions_input --from-beginning --group debug-producer-group
{"send_time": "2026-04-08T22:35:06.739915563Z","data": {"action_id": "3","action_name": "create_post","data": {"user_id": "user_8030","ip_address": "192.168.1.89","event_type": "create_post","post": {"text": "over fast cat runs brown."}}}}
{"send_time": "2026-04-08T22:35:08.902957765Z","data": {"action_id": "4","action_name": "create_post","data": {"user_id": "user_3960","ip_address": "192.168.1.71","event_type": "create_post","post": {"text": "brown cat fox lazy fast."}}}}
{"send_time": "2026-04-08T22:35:11.256141871Z","data": {"action_id": "5","action_name": "create_post","data": {"user_id": "user_5487","ip_address": "192.168.1.225","event_type": "create_post","post": {"text": "the the over over lazy."}}}}
{"send_time": "2026-04-08T22:35:13.659420008Z","data": {"action_id": "6","action_name": "create_post","data": {"user_id": "user_9756","ip_address": "192.168.1.74","event_type": "create_post","post": {"text": "jumps brown fast fast hello."}}}}
{"send_time": "2026-04-08T22:35:28.940382155Z","data": {"action_id": "12","action_name": "create_post","data": {"user_id": "user_2249","ip_address": "192.168.1.223","event_type": "create_post","post": {"text": "the and cat runs and."}}}}
{"send_time": "2026-04-08T22:35:31.497586666Z","data": {"action_id": "13","action_name": "create_post","data": {"user_id": "user_5736","ip_address": "192.168.1.93","event_type": "create_post","post": {"text": "dog dog fast quick lazy."}}}}
{"send_time": "2026-04-08T22:35:38.709120867Z","data": {"action_id": "16","action_name": "create_post","data": {"user_id": "user_6796","ip_address": "192.168.1.207","event_type": "create_post","post": {"text": "dog dog lazy fast lazy."}}}}
{"send_time": "2026-04-08T22:35:49.175876784Z","data": {"action_id": "20","action_name": "create_post","data": {"user_id": "user_6303","ip_address": "192.168.1.60","event_type": "create_post","post": {"text": "dog quick the and fox."}}}}
{"send_time": "2026-04-08T22:36:00.324318363Z","data": {"action_id": "25","action_name": "create_post","data": {"user_id": "user_7434","ip_address": "192.168.1.50","event_type": "create_post","post": {"text": "over brown hello lazy fast."}}}}
{"send_time": "2026-04-08T22:36:15.845490906Z","data": {"action_id": "32","action_name": "create_post","data": {"user_id": "user_5571","ip_address": "192.168.1.131","event_type": "create_post","post": {"text": "lazy hello over runs lazy."}}}}
...

The above is from the consumer, and shows that the events are consumed.

The producer logs also show the events being sent;

$ docker compose logs osprey-kafka-test-data-producer 
osprey-kafka-test-data-producer  | Generating actions every second to Kafka topic 'osprey.actions_input'...
osprey-kafka-test-data-producer  | Kafka broker: osprey-kafka:29092
osprey-kafka-test-data-producer  | Press Ctrl+C to stop...
osprey-kafka-test-data-producer  | 
osprey-kafka-test-data-producer  | Sending {"send_time": "2026-04-08T22:35:01.994270480Z","data": {"action_id": "1","action_name": "create_post","data": {"user_id": "user_816","ip_address": "192.168.1.207","event_type": "create_post","post": {"text": "hello the fox lazy quick."}}}}
osprey-kafka-test-data-producer  | Sending {"send_time": "2026-04-08T22:35:04.557031845Z","data": {"action_id": "2","action_name": "create_post","data": {"user_id": "user_1416","ip_address": "192.168.1.70","event_type": "create_post","post": {"text": "runs hello lazy fast hello."}}}}
osprey-kafka-test-data-producer  | Sending {"send_time": "2026-04-08T22:35:06.739915563Z","data": {"action_id": "3","action_name": "create_post","data": {"user_id": "user_8030","ip_address": "192.168.1.89","event_type": "create_post","post": {"text": "over fast cat runs brown."}}}}
osprey-kafka-test-data-producer  | Sending {"send_time": "2026-04-08T22:35:08.902957765Z","data": {"action_id": "4","action_name": "create_post","data": {"user_id": "user_3960","ip_address": "192.168.1.71","event_type": "create_post","post": {"text": "brown cat fox lazy fast."}}}}
...

Context/Environment (please complete the following information):

  • OS: Fedora 42
  • Browser: Firefox
  • Version 149.0

Additional context
@julietshen mentioning you as this is the issue I said I'll open.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions