Skip to content

feat: entropy metrics #119

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ishanarya0 wants to merge 132 commits into from
Closed

feat: entropy metrics #119

ishanarya0 wants to merge 132 commits into from

Conversation

@ishanarya0
Copy link
Member

@ishanarya0 ishanarya0 commented Mar 14, 2025

No description provided.

spy16 and others added 30 commits March 2, 2023 19:27
@spy16
fix: kube deployment char limit
ecf98ee
@spy16
feat: add deployment_id feature
d48ba77
@spy16
Merge pull request #1 from goto/fix/bugs
5d3d7ad 
fix: kube deployment name-limit handling
@ishanarya0
refactor: rename to goto (#2)
4867aa2 
chore: change odpf references to goto
@ishanarya0
fix: golanci config (#3)
3b338aa 
* fix: golanci config

* fix: gci lint issues

* fix: lint issues

* fix: remove nosnakecase lint from disable list
@spy16
fix: kube config handling (#4)
4d909e6
@spy16
fix: deployment id name (#5)
e96dcf9 
* fix: deployment id name
@spy16
Fix/generic (#6)
202e4ad 
* chore: add better error description

* refactor: merge firehose module files
@ishanarya0
fix: change odpf reference to goto (#7)
e758108
@spy16
generic refactors (#8)
26d5226 
refactor: reduce firehose package
@spy16
fix: firehose image name (#9)
ab5b2dc
@spy16
refactor: re-implement firehose module (#10)
022f7a4 
* feat: implement create, update, reset
* feat: implement upgrade
* feat: implement start, stop, scale
* test: add lots of tests
* feat: implement reset-sync
* refactor: simplify kafka reset flow
* feat: implement log
* refactor: separate client & server CLI
* feat: fix entropy client cli
@spy16
fix: use previous telegraf conf (#11)
4b6a238 
* fix: use previous telegraf conf
* refactor: remove old firehose module
@spy16
fix: length validation issue (#12)
152a191
@ishanarya0
fix: goreleaser commit author (#14)
f66d8bf 
fix: change commit author to bot
@spy16
feat: add stop flag support (#13)
fade8c7
@spy16
feat: add stop flag support (#13) (#15)
226fd7f
@spy16
fix: revision listing db query (#16)
cffb11f
@spy16
fix: expose chart values type (#17)
b93cbb8
@spy16
feat: strip trailing colon in image tag, add telegraf config template
b063bba 
* fix: strip trailing colon in image tag
* feat: add telegraf config templating
@spy16
fix: telegraf config format (#19)
fb2202d
@spy16
feat: add support for dynamic telegraf labels (#20)
eacb30f
@spy16
fix: use go map for telegraf image spec (#21)
9a37814
@spy16
fix: use map to fix yaml marshal (#22)
52671a6
@spy16
fix: set deployment-id length limit as 63 (#23)
2b799d7
@spy16
fix: revert 63 limit to 53 due helm limitation (#24)
3a368d2 
This reverts commit 2b799d7.
@spy16
feat: refactor release name to be consistent with old format (#25)
d3e42c0
@spy16
fix: helm release name limit (#26)
cbc5e88
@spy16
feat: implement deployment label templating (#27)
dc4b7ca
@spy16
fix: expose name in firehose module (#28)
199def5
FemiNoviaLina and others added 29 commits August 27, 2024 16:56
@FemiNoviaLina
fix: change worker config to map (#106)
d947041
@FemiNoviaLina
feat: enable opentelemetry for worker (#108)
417f10e
@FemiNoviaLina
feat: add kafka resource type (#109)
80d5637
@FemiNoviaLina
feat: kafka module config merge (#111)
eb750e2
@ishanarya0
feat: add dry run (#116)
bac23e0 
* feat: add dry run

* feat: add swagger

* test: add dry run tests

* refactor: use option pattern

* tests: fix call

* tests: fix params

---------

Co-authored-by: Ishan Arya <[email protected]>
@FemiNoviaLina
feat: allow maxcompute sink (#118)
46a524e
@ishanarya0
feat: add dagger (#115)
37c2a81 
* feat: add flink module (#110)

* rebase

* test: add e2e-test

* test: fix TearDown

* test: fix test

* feat: change var name

* feat: add dagger module create & update action (#107)

* feat: add dagger module

* feat: add flink dep

* feat: add transformations

* fix: read stream from config root

* feat: add Plan implementation

* fix: chart values

* fix: resolve TODOs and refactored

* fix: source sink base handling

* feat: Output to have CR details

* feat: handle status

* refactor: seperate contants by type

* refactor: kubeGetCRD function

* feat: add dagger update action

* fix: add Update action

* chore: change var name to sink_kafka_stream

* feat: merge consumer group ID if sink is same

---------

Co-authored-by: Ishan Arya <[email protected]>

* feat: dagger actions (#114)

* feat: add dagger module

* feat: add flink dep

* feat: add transformations

* fix: read stream from config root

* feat: add Plan implementation

* fix: chart values

* fix: resolve TODOs and refactored

* fix: source sink base handling

* feat: Output to have CR details

* feat: handle status

* refactor: seperate contants by type

* refactor: kubeGetCRD function

* feat: add dagger update action

* fix: add Update action

* chore: change var name to sink_kafka_stream

* feat: merge consumer group ID if sink is same

* feat: add start, stop and reset actions (#112)

* feat: add start & update action

* feat: add reset action

---------

Co-authored-by: Ishan Arya <[email protected]>

---------

Co-authored-by: Ishan Arya <[email protected]>

* feat: add extra streams

---------

Co-authored-by: Ishan Arya <[email protected]>

* feat: add logs

* feat: stop time in dagger

* fix: kafka source overriding

* fix: flink config check

* feat: add job_state as label

* fix: char len limit

* feat: add SINK_CONNECTOR_SCHEMA_PROTO_MESSAGE_CLASS

* feat: action and labels

* feat: add gcp_credentials

* feat: use deployment_id for chart name

* feat: state control in create and update

* feat: handle empty state fields

* fix: BuildResourceName

* fix: FLINK_JOB_ID

* fix: taskmanager resources

* feat: add flink module (#110)

* rebase

* test: add e2e-test

* test: fix TearDown

* test: fix test

* feat: change var name

* feat: add dagger module create & update action (#107)

* feat: add dagger module

* feat: add flink dep

* feat: add transformations

* fix: read stream from config root

* feat: add Plan implementation

* fix: chart values

* fix: resolve TODOs and refactored

* fix: source sink base handling

* feat: Output to have CR details

* feat: handle status

* refactor: seperate contants by type

* refactor: kubeGetCRD function

* feat: add dagger update action

* fix: add Update action

* chore: change var name to sink_kafka_stream

* feat: merge consumer group ID if sink is same

---------

Co-authored-by: Ishan Arya <[email protected]>

* feat: dagger actions (#114)

* feat: add dagger module

* feat: add flink dep

* feat: add transformations

* fix: read stream from config root

* feat: add Plan implementation

* fix: chart values

* fix: resolve TODOs and refactored

* fix: source sink base handling

* feat: Output to have CR details

* feat: handle status

* refactor: seperate contants by type

* refactor: kubeGetCRD function

* feat: add dagger update action

* fix: add Update action

* chore: change var name to sink_kafka_stream

* feat: merge consumer group ID if sink is same

* feat: add start, stop and reset actions (#112)

* feat: add start & update action

* feat: add reset action

---------

Co-authored-by: Ishan Arya <[email protected]>

---------

Co-authored-by: Ishan Arya <[email protected]>

* feat: add extra streams

---------

Co-authored-by: Ishan Arya <[email protected]>

* feat: add logs

* feat: stop time in dagger

* fix: kafka source overriding

* fix: flink config check

* feat: add job_state as label

* fix: char len limit

* feat: add SINK_CONNECTOR_SCHEMA_PROTO_MESSAGE_CLASS

* feat: action and labels

* feat: add gcp_credentials

* feat: use deployment_id for chart name

* feat: state control in create and update

* feat: handle empty state fields

* fix: BuildResourceName

* fix: FLINK_JOB_ID

* fix: taskmanager resources

* feat: send lowercase sink type to helm

* fix: preserve \n and \ in env vars

* feat: allow \t

* fix: use %q to quote

* feat: output shall not return internal server error (#119)

* feat: output shall not return internal server error

* refactor

---------

Co-authored-by: Ishan Arya <[email protected]>

* fix: change Error to string type

* fix: default handling for each key

* feat: pass urn to helm chart

* feat: chart will upgrade on start and update (#120)

Co-authored-by: Ishan Arya <[email protected]>

* fix: chart update

* fix: consumer offset update and reset

* fix: base case

* feat: add state in output

* fix: empty error in case of success

* feat: jaruri, image & chart version always update

* feat: reset to update jaruri and chart values

* fix: remove keys from env vars

---------

Co-authored-by: Ishan Arya <[email protected]>
@mabdh
feat: kube resource support namespace (#121)
c124014
@ishanarya0
fix: flink parallelism update (#122)
77eba57 
* fix: flink parallelism env var should be updated when replicas are updated

* fix: type conversion

---------

Co-authored-by: Ishan Arya <[email protected]>
@ishanarya0
Flink parallelism (#123)
6828b8d 
* fix: flink parallelism env var should be updated when replicas are updated

* fix: type conversion

* fix: env vars on create

---------

Co-authored-by: Ishan Arya <[email protected]>
@ishanarya0
fix: fix key to SCHEMA_REGISTRY_STENCIL_URLS (#124)
953741b 
Co-authored-by: Ishan Arya <[email protected]>
poc
6026b4d
feat: add metrics for res status change
4855c41
chore: remove unused variables
462e63c
refactor: otel setup
717f233
chore: clear opencencus
a0e0ce9
fix: add runtime metrics
c4e43b0
test: fix
34644ad
feat: add mod file
943d2e4
@ishanarya0
feat: add metrics for res status change (#126)
5862a9e 
* poc

* feat: add metrics for res status change

* chore: remove unused variables

* refactor: otel setup

* chore: clear opencencus

* fix: add runtime metrics

* test: fix

* feat: add mod file

---------

Co-authored-by: Ishan Arya <[email protected]>
chore: add debug logs
cbd3626
merge
ac966ae
@ishanarya0
chore: add debug logs (#128)
0fa074c 
* poc

* feat: add metrics for res status change

* chore: remove unused variables

* refactor: otel setup

* chore: clear opencencus

* fix: add runtime metrics

* test: fix

* feat: add mod file

* chore: add debug logs

---------

Co-authored-by: Ishan Arya <[email protected]>
chore: change log type
a30c6bd
merge
b6b2f39
@ishanarya0
chore: counter logs (#129)
163ee15 
* poc

* feat: add metrics for res status change

* chore: remove unused variables

* refactor: otel setup

* chore: clear opencencus

* fix: add runtime metrics

* test: fix

* feat: add mod file

* chore: add debug logs

* chore: change log type

---------

Co-authored-by: Ishan Arya <[email protected]>
@FemiNoviaLina
feat(dagger): add dagger bucket url config (#127)
eab3919
Merge branch 'main' into entropy-metrics
96f6840
chore: add res attributes to metric
c411af5
@ishanarya0 ishanarya0 closed this Mar 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@ishanarya0 @spy16 @mabdh @lavkesh @manishdangi98 @abhishekv24 @FemiNoviaLina