Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Time series use case #3502

Merged
merged 17 commits into from
Mar 17, 2025
Merged

Time series use case #3502

merged 17 commits into from
Mar 17, 2025

Conversation

mneedham
Copy link
Contributor

Time-series use case

@mneedham mneedham requested a review from a team as a code owner March 14, 2025 11:01
@vercel Vercel
Copy link

vercel bot commented Mar 14, 2025

@mneedham is attempting to deploy a commit to the ClickHouse Team on Vercel.

A member of the Team first needs to authorize it.

@vercel Vercel
Copy link

vercel bot commented Mar 14, 2025 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for Git ↗︎

2 Skipped Deployments
Name Status Preview Comments Updated (UTC)
clickhouse-docs-ru ⬜️ Ignored (Inspect) Visit Preview Mar 17, 2025 10:15am
clickhouse-docs-zh ⬜️ Ignored (Inspect) Visit Preview Mar 17, 2025 10:15am

@mneedham mneedham requested a review from Blargian March 15, 2025 07:51
Blargian
Blargian approved these changes Mar 15, 2025
View reviewed changes
Copy link
Member

@Blargian Blargian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Left a few comments.

docs/use-cases/time-series/basic-operations.md
Comment on lines +151 to +172
┌────────────────hour─┬─sum(hits)─┐
│ 2015-07-01 00:00:00 │ 3 │ <- missing values
│ 2015-07-01 02:00:00 │ 1 │ <- missing values
│ 2015-07-01 04:00:00 │ 1 │
│ 2015-07-01 05:00:00 │ 2 │
│ 2015-07-01 06:00:00 │ 1 │
│ 2015-07-01 07:00:00 │ 1 │
│ 2015-07-01 08:00:00 │ 3 │
│ 2015-07-01 09:00:00 │ 2 │ <- missing values
│ 2015-07-01 12:00:00 │ 2 │
│ 2015-07-01 13:00:00 │ 4 │
│ 2015-07-01 14:00:00 │ 2 │
│ 2015-07-01 15:00:00 │ 2 │
│ 2015-07-01 16:00:00 │ 2 │
│ 2015-07-01 17:00:00 │ 1 │
│ 2015-07-01 18:00:00 │ 5 │
│ 2015-07-01 19:00:00 │ 5 │
│ 2015-07-01 20:00:00 │ 4 │
│ 2015-07-01 21:00:00 │ 4 │
│ 2015-07-01 22:00:00 │ 2 │
│ 2015-07-01 23:00:00 │ 2 │
└─────────────────────┴───────────┘
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get a slightly different result when I run this query:

Suggested change
┌────────────────hour─┬─sum(hits)─┐
│ 2015-07-01 00:00:00 │ 3 │ <- missing values
│ 2015-07-01 02:00:00 │ 1 │ <- missing values
│ 2015-07-01 04:00:00 │ 1 │
│ 2015-07-01 05:00:00 │ 2 │
│ 2015-07-01 06:00:00 │ 1 │
│ 2015-07-01 07:00:00 │ 1 │
│ 2015-07-01 08:00:00 │ 3 │
│ 2015-07-01 09:00:00 │ 2 │ <- missing values
│ 2015-07-01 12:00:00 │ 2 │
│ 2015-07-01 13:00:00 │ 4 │
│ 2015-07-01 14:00:00 │ 2 │
│ 2015-07-01 15:00:00 │ 2 │
│ 2015-07-01 16:00:00 │ 2 │
│ 2015-07-01 17:00:00 │ 1 │
│ 2015-07-01 18:00:00 │ 5 │
│ 2015-07-01 19:00:00 │ 5 │
│ 2015-07-01 20:00:00 │ 4 │
│ 2015-07-01 21:00:00 │ 4 │
│ 2015-07-01 22:00:00 │ 2 │
│ 2015-07-01 23:00:00 │ 2 │
└─────────────────────┴───────────┘
┌────────────────hour─┬─sum(hits)─┐
│ 2015-07-01 00:00:00 │ 4 │
│ 2015-07-01 01:00:00 │ 2 │ <- missing 02:00:00
│ 2015-07-01 03:00:00 │ 1 │ <- missing 04:00:00
│ 2015-07-01 05:00:00 │ 1 │
│ 2015-07-01 06:00:00 │ 2 │ <- missing 07:00:00
│ 2015-07-01 08:00:00 │ 1 │
│ 2015-07-01 09:00:00 │ 3 │
│ 2015-07-01 10:00:00 │ 2 │ <- missing 11:00:00, 12:00:00
│ 2015-07-01 13:00:00 │ 1 │
│ 2015-07-01 14:00:00 │ 4 │
│ 2015-07-01 15:00:00 │ 2 │
│ 2015-07-01 16:00:00 │ 2 │
│ 2015-07-01 17:00:00 │ 2 │
│ 2015-07-01 18:00:00 │ 1 │
│ 2015-07-01 19:00:00 │ 5 │
│ 2015-07-01 20:00:00 │ 5 │
│ 2015-07-01 21:00:00 │ 4 │
│ 2015-07-01 22:00:00 │ 4 │
│ 2015-07-01 23:00:00 │ 1 │
└─────────────────────┴───────────┘

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeh I don't really know how it's picking up the 1 billion rows, it seems to be different each time! I'm not sure how to make it reproducible as there are so many records per date, but we need to include multiple days for some of the queries to return interesting results

docs/use-cases/time-series/basic-operations.md
Comment on lines +188 to +213
┌────────────────hour─┬─sum(hits)─┐
│ 2015-07-01 00:00:00 │ 3 │
│ 2015-07-01 01:00:00 │ 0 │ <- new value
│ 2015-07-01 02:00:00 │ 1 │
│ 2015-07-01 03:00:00 │ 0 │ <- new value
│ 2015-07-01 04:00:00 │ 1 │
│ 2015-07-01 05:00:00 │ 2 │
│ 2015-07-01 06:00:00 │ 1 │
│ 2015-07-01 07:00:00 │ 1 │
│ 2015-07-01 08:00:00 │ 3 │
│ 2015-07-01 09:00:00 │ 2 │
│ 2015-07-01 10:00:00 │ 0 │ <- new value
│ 2015-07-01 11:00:00 │ 0 │ <- new value
│ 2015-07-01 12:00:00 │ 2 │
│ 2015-07-01 13:00:00 │ 4 │
│ 2015-07-01 14:00:00 │ 2 │
│ 2015-07-01 15:00:00 │ 2 │
│ 2015-07-01 16:00:00 │ 2 │
│ 2015-07-01 17:00:00 │ 1 │
│ 2015-07-01 18:00:00 │ 5 │
│ 2015-07-01 19:00:00 │ 5 │
│ 2015-07-01 20:00:00 │ 4 │
│ 2015-07-01 21:00:00 │ 4 │
│ 2015-07-01 22:00:00 │ 2 │
│ 2015-07-01 23:00:00 │ 2 │
└─────────────────────┴───────────┘
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
┌────────────────hour─┬─sum(hits)─┐
│ 2015-07-01 00:00:00 │ 3 │
│ 2015-07-01 01:00:00 │ 0 │ <- new value
│ 2015-07-01 02:00:00 │ 1 │
│ 2015-07-01 03:00:00 │ 0 │ <- new value
│ 2015-07-01 04:00:00 │ 1 │
│ 2015-07-01 05:00:00 │ 2 │
│ 2015-07-01 06:00:00 │ 1 │
│ 2015-07-01 07:00:00 │ 1 │
│ 2015-07-01 08:00:00 │ 3 │
│ 2015-07-01 09:00:00 │ 2 │
│ 2015-07-01 10:00:00 │ 0 │ <- new value
│ 2015-07-01 11:00:00 │ 0 │ <- new value
│ 2015-07-01 12:00:00 │ 2 │
│ 2015-07-01 13:00:00 │ 4 │
│ 2015-07-01 14:00:00 │ 2 │
│ 2015-07-01 15:00:00 │ 2 │
│ 2015-07-01 16:00:00 │ 2 │
│ 2015-07-01 17:00:00 │ 1 │
│ 2015-07-01 18:00:00 │ 5 │
│ 2015-07-01 19:00:00 │ 5 │
│ 2015-07-01 20:00:00 │ 4 │
│ 2015-07-01 21:00:00 │ 4 │
│ 2015-07-01 22:00:00 │ 2 │
│ 2015-07-01 23:00:00 │ 2 │
└─────────────────────┴───────────┘
┌────────────────hour─┬─sum(hits)─┐
│ 2015-07-01 00:00:00 │ 4 │
│ 2015-07-01 01:00:00 │ 2 │
│ 2015-07-01 02:00:00 │ 0 │ <- filled value
│ 2015-07-01 03:00:00 │ 1 │
│ 2015-07-01 04:00:00 │ 0 │ <- filled value
│ 2015-07-01 05:00:00 │ 1 │
│ 2015-07-01 06:00:00 │ 2 │
│ 2015-07-01 07:00:00 │ 0 │ <- filled value
│ 2015-07-01 08:00:00 │ 1 │
│ 2015-07-01 09:00:00 │ 3 │
│ 2015-07-01 10:00:00 │ 2 │
│ 2015-07-01 11:00:00 │ 0 │ <- filled value
│ 2015-07-01 12:00:00 │ 0 │ <- filled value
│ 2015-07-01 13:00:00 │ 1 │
│ 2015-07-01 14:00:00 │ 4 │
│ 2015-07-01 15:00:00 │ 2 │
│ 2015-07-01 16:00:00 │ 2 │
│ 2015-07-01 17:00:00 │ 2 │
│ 2015-07-01 18:00:00 │ 1 │
│ 2015-07-01 19:00:00 │ 5 │
│ 2015-07-01 20:00:00 │ 5 │
│ 2015-07-01 21:00:00 │ 4 │
│ 2015-07-01 22:00:00 │ 4 │
│ 2015-07-01 23:00:00 │ 1 │
└─────────────────────┴───────────┘

mneedham and others added 9 commits March 17, 2025 17:25
@mneedham @Blargian
Update docs/use-cases/time-series/analysis-functions.md
d179ed4 
Co-authored-by: Shaun Struwig <[email protected]>
@mneedham @Blargian
Update docs/use-cases/time-series/basic-operations.md
7274837 
Co-authored-by: Shaun Struwig <[email protected]>
@mneedham @Blargian
Update docs/use-cases/time-series/date-time-data-types.md
58fd27a 
Co-authored-by: Shaun Struwig <[email protected]>
@mneedham @Blargian
Update docs/use-cases/time-series/index.md
85aa0f4 
Co-authored-by: Shaun Struwig <[email protected]>
@mneedham @Blargian
Update docs/use-cases/time-series/query-performance.md
8f6fc0f 
Co-authored-by: Shaun Struwig <[email protected]>
@mneedham @Blargian
Update docs/use-cases/time-series/basic-operations.md
c81c4af 
Co-authored-by: Shaun Struwig <[email protected]>
@mneedham @Blargian
Update docs/use-cases/time-series/basic-operations.md
0566cbe 
Co-authored-by: Shaun Struwig <[email protected]>
@mneedham @Blargian
Update docs/use-cases/time-series/basic-operations.md
81ac0fd 
Co-authored-by: Shaun Struwig <[email protected]>
@Blargian
Update docs/use-cases/time-series/storage-efficiency.md
af1838a
@mneedham mneedham merged commit 3a57e07 into ClickHouse:main Mar 17, 2025
11 checks passed
@Blargian Blargian linked an issue Mar 19, 2025 that may be closed by this pull request
Handling Time series data, optimizing for time series + functions #3024
Closed
@Blargian Blargian mentioned this pull request Mar 19, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Blargian Blargian Blargian approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Handling Time series data, optimizing for time series + functions
2 participants
@mneedham @Blargian