Skip to content

Latest commit

 

History

History
1281 lines (1104 loc) · 115 KB

CHANGELOG.md

File metadata and controls

1281 lines (1104 loc) · 115 KB

Changelog

v1.1.7 (2024-03-15)

  • d4e0bd28 Update manifests to v1.1.7
  • e53f8584 fix: initialize inflightAcks channel to not nil channel (#1548)
  • ae1ddaac chore(deps): bump github.com/go-jose/go-jose/v3 from 3.0.1 to 3.0.3 (#1549)

Contributors

  • Antonino Fugazzotto
  • Derek Wang
  • dependabot[bot]

v1.1.6 (2024-02-27)

  • 9613573e Update manifests to v1.1.6
  • b8bb2336 fix(controller): vertex template metadata nil check (#1527)
  • c0b3bdac feat: add support for SASL SCRAM 256 and 512 for Kafka (#1518)
  • fe26f91a fix panic inside reduce after getting sigterm
  • bdac5a75 fix: incorrect json schema for tls config (#1520)
  • 53cfae97 fix: podSpec incorrectly configured in case template exist (#1516)
  • 915c09aa fix: unknown for ISB details in pipeline card (#1497)
  • c71b2aba feat: configure standardResources via controller configmap (#1490)
  • ce0a3155 fix: add idle handler offset nil check (#1489)
  • 8d16d499 feat: terminate reduce vertex pods when pausing pipeline (#1481)

Contributors

  • Damien RAYMOND
  • Derek Wang
  • Dillen Padhiar
  • Juanlu Yu
  • Nishchith Shetty
  • Vedant Gupta
  • Yashash H L

v1.1.5 (2024-01-23)

  • e5bcf32e Update manifests to v1.1.5
  • 266cb227 fix(controller): incorrect cpu/mem resources calculation (#1477)

Contributors

  • Derek Wang

v1.1.4 (2024-01-20)

  • 7ffb521b Update manifests to v1.1.4
  • de780b95 fix: bug in late message handling for sliding window (#1472)

Contributors

  • Derek Wang
  • Yashash H L

v1.1.3 (2024-01-14)

  • 0b96acf9 Update manifests to v1.1.3
  • 907949be fix: GetDownstreamEdges is not cycle safe (#1447)
  • 1d83b51e chore(deps): bump follow-redirects from 1.15.3 to 1.15.4 in /ui (#1448)
  • 855672dd fix: UI Filter by status for pipelines doesn't work as expected (#1444)
  • c06de95e fix: Kafka source reads duplicated messages (#1438)
  • 17c9c0e2 feat: enhance autoscaling peeking logic (#1432)

Contributors

  • Derek Wang
  • Juanlu Yu
  • Nishchith Shetty
  • akash khamkar
  • dependabot[bot]

v1.1.2 (2024-01-01)

  • ac716ec4 Update manifests to v1.1.2
  • af17d8ce fix: server-secrets-init container restart (#1433)

Contributors

  • Derek Wang
  • Vedant Gupta

v1.1.1 (2023-12-21)

  • 5ff77fe0 Update manifests to v1.1.1
  • 5fd20ad9 chore(deps): bump golang.org/x/crypto from 0.14.0 to 0.17.0 (#1424)
  • da32632c fix: configmap const name (#1423)

Contributors

  • Derek Wang
  • dependabot[bot]

v1.1.0 (2023-12-18)

  • 07d46ca9 Update manifests to v1.1.0
  • 41b8dffc feat: local user support for Numaflow (#1416)
  • 818be4f2 fix: consider lastPublishedIdleWm when computed watermark is -1 (#1415)
  • 263263b3 fix: access path for auth endpoints (#1403)
  • 0db9cd19 feat: Generate Idle Watermark if the source is idling (#1385)
  • 6eb25c25 fix: include dropped messages in source watermark calculation (#1404)
  • 1eee1942 chore(deps): bump @adobe/css-tools from 4.3.1 to 4.3.2 in /ui (#1400)
  • 0a2ff566 fix: updated access path config for root path (#1397)
  • 024597d5 feat: improve numaflow k8s events (#1393)
  • 83fb9068 fix: update numaflow-go version (#1387)
  • 9b0fadb6 fix: access path for api/v1 route (#1388)
  • 96cfa555 fix: dropped messages should not be considered for watermark propagation (#1386)
  • 5bf63076 chore(deps): bump github.com/go-jose/go-jose/v3 from 3.0.0 to 3.0.1 (#1383)
  • 0c82ee0f refactor: move udf forwarder to the right dir (#1381)
  • 3a77ed42 fix: add pipeline update validation checks (#1379)
  • 44a38f4a fix: disallow updating an existing isbsvc's persistence strategy (#1376)
  • 82538b6a fix: add more checks to isbsvc validation (#1358)
  • 118c309d fix: non-ack failed offsets (#1370)
  • 3456f714 feat: validate patched data for pipelines (#1349)
  • e75e9581 Add Atlan into USERS.md (#1351)
  • 483c0185 Unit tests UI (#1348)
  • d0ae1484 fix(SERVER): remove unknown filter (#1346)
  • 7ac77521 Upstreammain (#1345)
  • f1af1f02 fix: rc-4 bug bash bug fixes (#1343)
  • f6edd5ae fix(UI): rc-0.4 fixes (#1342)
  • ecbd489f fix(SERVER): fix styles for ISB cards
  • d4836e9e fix(SERVER): fix styles for ISB cards
  • 953feb67 fix(SERVER): fix for pagination issue
  • d484393d fix(SERVER): namespace inout filtering space alignment
  • e9360127 fix(SERVER): pipeline card style fix
  • c24b91ce fix: block pipeline load post update (#1333)
  • e928be34 fix: full isb spec in edit (#1331)
  • d5c3b07a RC2.0 UI fixes (#1329)
  • f4211fb1 feat: container for generator vertices (#1321)
  • f4354af3 fix: create isb should move to isb tab (#1323)
  • 24afc5e1 fix: max lag (#1319)
  • 42f81df4 feat: cache daemon client for each pipeline (#1276)
  • a7270426 fix: rc2 UI fixes (#1317)
  • 01bf1854 feat: added filtering based on number, status and health of pipelines… (#1312)
  • 91b0effa fix: user identity cookie max age (#1316)
  • 5c999b63 feat: add scopes to authorization (#1288)
  • e119a0ee fix: logout fix (#1310)
  • 449dfd3a fix: Split cookie to meet the cookie length requirement (#1305)
  • 87c4c1e1 chore(deps): bump github.com/nats-io/nats-server/v2 from 2.10.3 to 2.10.4 (#1307)
  • ea577452 add tooltips (#1289)
  • 93eec963 fix: fixed the timer not clearing issue (#1303)
  • c7bdbda5 feat: k8s events filtering and cluster summary card fixes (#1297)
  • a558b229 fix: graph overflow with large height (#1301)
  • 73ffaa86 doc: need metrics server (#1296)
  • 32416bf2 refactor: unified metrics names for forwarders (#1290)
  • 17b7b313 feat: added separate colors for sideInput and dynamic legend (#1292)
  • 6efab64e feat: add tabs to display pipelines and isb services (#1293)
  • 86df4a84 fix: more sidebar testing (#1287)
  • d7ae1d36 refactor: create interfaces for AuthN and AuthZ (#1286)
  • 65aca23f fix: tests for utils (#1283)

Contributors

  • Bradley Behnke
  • Chandan Kumar
  • Darshan Simha
  • Derek Wang
  • Dillen Padhiar
  • Juanlu Yu
  • Keran Yang
  • Madusudanan.B.N
  • Shakira M
  • Sidhant Kohli
  • Vedant Gupta
  • Vigith Maurice
  • Yashash H L
  • dependabot[bot]
  • mshakira

v1.0.0 (2023-11-03)

  • 78134e8f Update manifests to v1.0.0
  • 660ff501 fix: rc-4 bug bash bug fixes (#1343)
  • ceed5def fix(UI): rc-0.4 fixes (#1342)

Contributors

  • Darshan Simha
  • Derek Wang
  • mshakira

v1.0.0-rc4 (2023-11-03)

  • e94b563f Update manifests to v1.0.0-rc4
  • 14757afc fix(SERVER): fix styles for ISB cards
  • fc90920b fix(SERVER): fix styles for ISB cards
  • 8bf275bd fix(SERVER): fix for pagination issue
  • 2e8cff7a fix(SERVER): namespace inout filtering space alignment
  • c77fb0fc fix(SERVER): pipeline card style fix
  • 52d9370f fix: block pipeline load post update (#1333)
  • 2307660f fix: full isb spec in edit (#1331)
  • a70c77b9 RC2.0 UI fixes (#1329)
  • 02281f38 feat: container for generator vertices (#1321)
  • feb4977a fix: create isb should move to isb tab (#1323)
  • e05132ef fix: max lag (#1319)

Contributors

  • Bradley Behnke
  • Darshan Simha
  • Derek Wang
  • Juanlu Yu
  • Shakira M
  • Vedant Gupta
  • mshakira

v1.0.0-rc3 (2023-11-01)

  • 6ab96b18 Update manifests to v1.0.0-rc3
  • 4ffda383 feat: cache daemon client for each pipeline (#1276)
  • 5a7d739e fix: rc2 UI fixes (#1317)
  • cbd810bc feat: added filtering based on number, status and health of pipelines… (#1312)
  • 491c8786 fix: user identity cookie max age (#1316)

Contributors

  • Darshan Simha
  • Derek Wang
  • Dillen Padhiar
  • Juanlu Yu
  • mshakira

v1.0.0-rc2 (2023-11-01)

  • 8a7dc592 Update manifests to v1.0.0-rc2
  • 8ed52b39 feat: add scopes to authorization (#1288)
  • eebe623f fix: logout fix (#1310)
  • 9c34c27f fix: Split cookie to meet the cookie length requirement (#1305)
  • c6073b17 chore(deps): bump github.com/nats-io/nats-server/v2 from 2.10.3 to 2.10.4 (#1307)
  • b1bb6575 add tooltips (#1289)
  • 40ca1705 fix: fixed the timer not clearing issue (#1303)
  • 2a91668d feat: k8s events filtering and cluster summary card fixes (#1297)
  • 4929a1e4 fix: graph overflow with large height (#1301)
  • 8cc04a77 doc: need metrics server (#1296)
  • 05b19400 refactor: unified metrics names for forwarders (#1290)
  • 5475e682 feat: added separate colors for sideInput and dynamic legend (#1292)
  • 052201e0 feat: add tabs to display pipelines and isb services (#1293)
  • 9e1ea4f0 fix: more sidebar testing (#1287)
  • d018af50 refactor: create interfaces for AuthN and AuthZ (#1286)
  • ada5ea4d fix: tests for utils (#1283)

Contributors

  • Bradley Behnke
  • Darshan Simha
  • Derek Wang
  • Juanlu Yu
  • Keran Yang
  • Sidhant Kohli
  • Vedant Gupta
  • Vigith Maurice
  • dependabot[bot]
  • mshakira

v1.0.0-rc1 (2023-10-26)

  • 0ff1f58f Update manifests to v1.0.0-rc1
  • 16c3fc3c fix: incorrect image version for namespaced numaflow-server (#1282)
  • 18d62935 fix: ISBCreate test and fetch mock setup (#1279)
  • 9325140c fix: update dex to work with basehref (#1278)
  • 36610a5a feat: AuthN/AuthZ for Numaflow UI (#1234)
  • 730552e8 fix(doc): hpa api version (#1274)
  • c103427b fix: updating example.md (#1262)
  • c1725b18 Feat/side input tests (#1257)
  • 5554bd65 chore(deps): bump google.golang.org/grpc from 1.57.0 to 1.57.1 (#1268)
  • b86b2254 fix(UI): pod selection fix (#1266)
  • 07a7b8ea doc: Numaflow high level security (#1264)
  • 308acf2f fix: fixed the ns-summary page to allow creation of pipeline when no … (#1263)
  • ce737732 Unit tests graph page (#1250)
  • 639a9364 feat: Add e2e test for map sideinput, Fixes #1192 (#1211)
  • 7b32af34 feat: get current status of ISB service (#1199)
  • 5c7fc90e Summary view fixes (#1253)
  • 53aed68d chore(deps): bump github.com/nats-io/nats-server/v2 from 2.9.19 to 2.9.23 (#1232)
  • 30382995 fix: updated div's with box and removed unwanted css (#1236)
  • 8b50f36c chore(deps): bump @babel/traverse from 7.23.0 to 7.23.2 in /ui (#1221)
  • 7a3ca76a Update kafka.md (#1218)
  • e49a1811 Update generator.md (#1217)
  • 9785eb07 Update map.md (#1219)
  • a874478a feat: UI 1.0 CRUD (#1181)
  • d34fcc47 fix: get isbsvc kind apiversion (#1220)
  • 74f4d980 Update kafka.md (#1215)
  • 7d5fe51f Update generator.md (#1214)
  • b6adac15 Update overview.md (#1213)
  • 0748449d feat: add udsource python e2e (#1204)
  • fea29657 doc: roadmap (#1208)
  • d7630553 Namespace card status bar changes 0.11 (#1206)
  • 19a523aa feat: added unit tests for PipelineCard component (#1205)
  • 6a50f130 pods component error fix (#1203)
  • a07e78f7 fix: updated image styles to a class (#1202)
  • 71c048eb feat: Changed the status bar component to an icon based component (#1198)
  • 13264e3f fix(SERVER): restructure pod details component (#1189)
  • 1bbdcad8 feat: updated the legend to a collapsible one on the top left (#1196)
  • 910243bd chore(deps): bump golang.org/x/net from 0.12.0 to 0.17.0 (#1190)
  • 57af20e5 feat: API Delete ISBSVC validation (#1182)
  • 2e1fd701 added BCubed to the user list (#1184)
  • b962f8e3 feat: add timeout for pausing pipeline. Fixes #992 (#1138)
  • ef62c5ca feat: Jetstream support for replica of 1 Fixes #944 (#1177)
  • 80294989 doc: minor clean up of JOIN doc (#1175)
  • d4b5f1b2 fix: incorrect side inputs watch logic (#1164)
  • 00e6b6ae fix: not considered as back pressured when onFull is discardLatest (#1153)
  • 8b0d8cef fix(SERVER): handle states when status is unknown (#1154)
  • 31b1aacd fix(SERVER): fix key warning (#1152)

Contributors

  • Bradley Behnke
  • Caroline Dikibo
  • Chandan Kumar
  • Darshan Simha
  • Darshan Simha U
  • Dennis Sosa
  • Derek Wang
  • Dillen Padhiar
  • Joel Millage
  • Jorvaulx
  • Juanlu Yu
  • Kayla Nussbaum
  • Shubham Dixit
  • Sidhant Kohli
  • TASNEEM KOUSHAR
  • Vigith Maurice
  • aruwanip
  • bpcarey01
  • dependabot[bot]
  • mshakira

v0.11.0 (2023-10-13)

  • fbf51b2d Update manifests to v0.11.0
  • f33d614f Namespace card status bar changes 0.11 (#1206)
  • 391b75dc feat: added unit tests for PipelineCard component (#1205)
  • 7bcc1f27 pods component error fix (#1203)
  • df28f937 fix: updated image styles to a class (#1202)
  • bbb7db77 feat: Changed the status bar component to an icon based component (#1198)
  • f6f20d6c fix(SERVER): restructure pod details component (#1189)
  • 96a6002e feat: updated the legend to a collapsible one on the top left (#1196)
  • 575605b4 chore(deps): bump golang.org/x/net from 0.12.0 to 0.17.0 (#1190)
  • bfec4fc5 added BCubed to the user list (#1184)
  • 1e0b25fd feat: add timeout for pausing pipeline. Fixes #992 (#1138)
  • ae232764 feat: Jetstream support for replica of 1 Fixes #944 (#1177)
  • 1e405b37 doc: minor clean up of JOIN doc (#1175)
  • 77c01811 fix: incorrect side inputs watch logic (#1164)

Contributors

  • Darshan Simha U
  • Derek Wang
  • Dillen Padhiar
  • Joel Millage
  • Vigith Maurice
  • dependabot[bot]
  • mshakira

v0.11.0-rc2 (2023-10-03)

  • 8ae28af3 Update manifests to v0.11.0-rc2
  • f74bde39 fix: not considered as back pressured when onFull is discardLatest (#1153)
  • 689aa295 fix(SERVER): handle states when status is unknown (#1154)
  • f83ea00a fix(SERVER): fix key warning (#1152)

Contributors

  • Derek Wang
  • mshakira

v0.11.0-rc1 (2023-10-03)

  • 1d89a12b Update manifests to v0.11.0-rc1
  • 882bcef3 feat: Numaflow UI 1.0 (#1077)
  • a7adee1e fix: treat ALL user-defined source vertices as scalable (#1132)
  • 408ff389 feat: add doc link checker (#1130)
  • dae11a7f extracting redis streams source (#1113)
  • 9d65e229 refactor: shared kubeconfig util (#1095)
  • bb6f29b2 fix: wrong api group in webhook rbac settings (#1086)
  • 4150eb28 feat: set correct number of replices for some types of vertices after resuming pipeline (#1085)
  • 2182b83b chore(deps): bump graphql from 16.6.0 to 16.8.1 in /ui (#1078)
  • b2a377ca feat: add forest validation for pipelines. Fixes #1002 (#1063)
  • 9035ba8b feat: implement pipeline validation for unsupported states (#1043)
  • 4369d65f fix: message count read in forwarder (#1030)
  • 8d52d78b fix(ci): Wrong test pod image tag used for running CI on release branch (#1041)
  • 6ed2c381 fix: allow udsource pending api to return negative count to indicate PendingNotAvailable (#1040)
  • c12e9094 refactor: re-arrange some of the rater implementations (#1036)
  • abe332e6 fix(docs): file names and links for side inputs (#1037)
  • 20fb7cc2 doc: Add side input docs (#1029)
  • b373406c doc: kill-switch when buffer is full (#1034)
  • 9f5127f0 fix: Idle handler refactor (#1021)
  • b1f8b026 fix: calculate processing rate for sink vertices (#1025)
  • c01a1406 feat: colored logs for UI with toggle for logs order (#1022)

Contributors

  • Derek Wang
  • Dillen Padhiar
  • Juanlu Yu
  • Julie Vogelman
  • Keran Yang
  • Sidhant Kohli
  • Tianchu Zhao
  • Vedant Gupta
  • Vigith Maurice
  • dependabot[bot]

v0.10.1 (2023-09-14)

  • e5e2b619 Update manifests to v0.10.1
  • 4702849c feat: implement pipeline validation for unsupported states (#1043)
  • e36b3c6c fix: message count read in forwarder (#1030)
  • 01d9abee fix: allow udsource pending api to return negative count to indicate PendingNotAvailable (#1040)
  • 8cbb4d67 refactor: re-arrange some of the rater implementations (#1036)
  • b511effc fix(docs): file names and links for side inputs (#1037)
  • 5b083c2d doc: Add side input docs (#1029)
  • 8d5c56f8 doc: kill-switch when buffer is full (#1034)
  • 73db23a3 fix: Idle handler refactor (#1021)
  • 7119ed96 fix: calculate processing rate for sink vertices (#1025)
  • 6408137d feat: colored logs for UI with toggle for logs order (#1022)
  • 2311260c fix(ci): Wrong test pod image tag used for running CI on release branch (#1041)

Contributors

  • Derek Wang
  • Dillen Padhiar
  • Juanlu Yu
  • Keran Yang
  • Sidhant Kohli
  • Vedant Gupta
  • Vigith Maurice

v0.10.0 (2023-09-05)

  • 10d1cfde Update manifests to v0.10.0
  • 6f7c1f4a fix: seg fault inside controller (#1016)
  • c2fdef16 fix: reconcile headless services before pods (#1014)
  • 7d8b9087 fix: print version info when starting (#1013)
  • 247b89ed feat: join vertex UI support (#1010)
  • aabb8af0 feat: scaleUpCooldownSeconds and scaleDownCooldownSeconds to replace cooldownSeconds (#1008)
  • ad647ab7 chore(deps): bump @adobe/css-tools from 4.2.0 to 4.3.1 in /ui (#1005)
  • 92fbf7f1 fix: avoid unwanted watcher creation and reduce being stuck with udf is restarted (#999)
  • bac06df0 fix: missing edges on UI (#998)
  • f90d4fe7 feat: Add side input sdkclient and grpc (#953)
  • d99480a8 feat: implement user-defined source (#980)
  • 70685902 fix: send keys for udsink (#979)
  • 8f32b9a3 fix bulleted list (#977)
  • 1f33bf8b refactor: build wmstore and wmstorewatcher directly, and remove some unnecessary fields (#970)
  • 4cea3444 feat: add vertex template to pipeline spec (#947)
  • 4a4ed927 feat: Add side-input initializer and synchronizer (#912)
  • d10f36e6 fix: npe when the ctx is canceled inside kv watcher (#942)
  • 6b1b3337 fix: retry logic for fetching last updated kv time (#939)
  • e3da4a3e fix: close the watermark fetcher and publishers after all the forwarders exit (#921)
  • 2d6112bf Pipelines with Cycles: e2e testing, and pipeline validation (#920)
  • 5e0bf77e docs quick fixes (#919)
  • 0f8f7a17 docs updates (#917)
  • b55566b8 feat: watermark delay in tooltip (#910)
  • 667ada75 fix: removing WIP tag (#914)
  • 872aa864 feat: emit k8s events for controller messages. Fixes #856 (#901)
  • 0fbdb7ab fix: avoid potential deadlocks when operating UniqueStringList (#905)
  • 2c85ec43 refactor: avoid exposing internal data structures of pod tracker to the rater (#902)
  • 7e86306b feat: Join Vertex (#875)
  • 85360f65 fix: stabilize nats connection (#889)
  • d4f8f594 doc: Update multi partition doc (#898)
  • 404672d6 fix: Reduce idle WM unit test fix (#897)
  • a1bbdedf updated default version of Redis used for e2e (#891)
  • 85ee4b0d fix TestBuiltinEventTimeExtractor (#885)
  • f3e1044e chore(deps): bump word-wrap from 1.2.3 to 1.2.4 in /ui (#881)
  • a02f29a7 fix: remove retry when the processor is not found. (#868)
  • cfdeaa8a refactor: create a new data forwarder dedicated for source (#874)
  • 6d14998a feat: controller changes for Side Inputs support (#866)
  • 92db62a9 fix: highlight edge when buffer is full (#869)
  • 9c4e83c0 fix: minor ui bugs (#861)
  • b970b4cc fix: release script for validating webhook (#860)
  • 7684aada fix: use windower to fetch next window yet to be closed (#850)
  • 609d8b3c feat: implement optional validation webhook. Fixes #817. (#832)
  • 3ae1cedb chore(deps): bump semver from 6.3.0 to 6.3.1 in /ui (#845)

Contributors

  • Derek Wang
  • Dillen Padhiar
  • Jason Zesheng Chen
  • Juanlu Yu
  • Julie Vogelman
  • Keran Yang
  • RohanAshar
  • Sidhant Kohli
  • Vedant Gupta
  • Vigith Maurice
  • Yashash H L
  • dependabot[bot]

v0.9.3 (2023-09-05)

  • 6141719f Update manifests to v0.9.3
  • 022f8bfa fix: seg fault inside controller (#1016)

Contributors

  • Derek Wang
  • Yashash H L

v0.9.2 (2023-08-23)

  • 6f81361f Update manifests to v0.9.2
  • 66c32197 fix: error when kv_watch with no keys (#981)

Contributors

  • Derek Wang

v0.9.1 (2023-08-11)

  • 4cbd729c Update manifests to v0.9.1
  • aa5e8ae3 fix: npe when the ctx is canceled inside kv watcher (#942)
  • e5a5cd6c feat: watermark delay in tooltip (#910)

Contributors

  • Derek Wang
  • Vedant Gupta
  • Yashash H L

v0.9.0 (2023-08-02)

  • 8e4b6ca1 Update manifests to v0.9.0
  • 7424ae50 feat: emit k8s events for controller messages. Fixes #856 (#901)
  • d0bfac6d fix: avoid potential deadlocks when operating UniqueStringList (#905)
  • 75c7f975 fix: stabilize nats connection (#889)
  • 0db1238d fix: Reduce idle WM unit test fix (#897)
  • 5073f1c8 fix TestBuiltinEventTimeExtractor (#885)
  • 33b7d1d0 fix: remove retry when the processor is not found. (#868)
  • 89b2d1c4 fix: highlight edge when buffer is full (#869)
  • 8d49c0f6 fix: minor ui bugs (#861)
  • 9478e302 fix: release script for validating webhook (#860)

Contributors

  • Derek Wang
  • Dillen Padhiar
  • Juanlu Yu
  • Keran Yang
  • Vedant Gupta
  • Yashash H L

v0.9.0-rc2 (2023-07-13)

  • d0df669a Update manifests to v0.9.0-rc2
  • c8aaeff8 fix: use windower to fetch next window yet to be closed (#850)
  • bcda8dcf feat: implement optional validation webhook. Fixes #817. (#832)
  • e605504d chore(deps): bump semver from 6.3.0 to 6.3.1 in /ui (#845)

Contributors

  • Derek Wang
  • Dillen Padhiar
  • Yashash H L
  • dependabot[bot]

v0.9.0-rc1 (2023-07-11)

  • 40f45410 Update manifests to v0.9.0-rc1
  • f5276dbb fix: pod tracker logic for calculating processing rate (#838)
  • db06e7e4 chore(deps): bump tough-cookie from 4.1.2 to 4.1.3 in /ui (#839)
  • b660b6d9 fix: resource leak inside daemon server (#837)
  • 1f19a742 feat: capability to increase max message size (#835)
  • c61ce319 doc: update roadmap (#830)
  • aca1c9bf feat: add stragglers (late data) into the window is window is open (#824)
  • 0155b4a5 fix(docs): fixed some incorrect docs and renamed a timeExtractionFilter arg (#814)
  • dd060cb8 feat: rater changes to track processing rate per partition (#805)
  • 541ceb20 fix: metric to track watermark bug was wrongly tagged (#809)
  • cf473151 feat: autoscaling changes to support multi partition (#806)
  • 6a5ee1a5 fix: segmentation fault in daemon server (#804)
  • 2ce0ac90 fix: Intermittent failure from Kafka to get consumer offsets (#803)
  • 32be7fc5 feat: support UI for multipartition edges (#789)
  • 97db1984 refactor: remove redundant delta calculations for rater (#795)
  • 3406a130 fix: select pods not in evicted status (#786)
  • a9204fbc feat: combine built-in UDTransformers for filter and eventTime assignment (#783)
  • 85955e30 feat: support multi-partitioned edges (#751)
  • 5ce6936d fix: duplicate ui served from gin Router (#781)
  • e9ea7d85 fix: unexpected high processing rates (#780)
  • f60b8ab9 chore(deps): bump github.com/gin-gonic/gin from 1.9.0 to 1.9.1 (#772)
  • 466e3804 feat: gRPC error handling (#744)
  • 9aff2bd8 feat: forwardAChunk to support multi partitioned edges (#757)
  • f3273170 fix: pipeline view fix (#755)
  • 037c9a61 fix: toVertexPartitions for reduce was incorrectly populated to 1 (#756)
  • 099b914a feat: use metrics to calculate vertex processing rate (#743)
  • 59880e97 feat: enable streaming message to next vertex when batch size is 1 (#709)
  • 160b9414 fix: use int32 for message length (#750)
  • e383ee2f feat: using one bucket for partitioned reduce watermark propagation (#742)
  • ba1f493d fix(test): flakey test (#738)
  • f0c83291 refactor: buffer, edge, bucket (#733)
  • eb9a7c4c feat: change baseHref for Numaflow UI. Fixes #375. (#698)
  • b1f639e7 fix: let kafka source crash and restart when there is any server side error (#735)
  • c8aafc7f feat: Autoscale for Redis Streams Source (#726)
  • d8074ec1 feat: Redis7 as an ISB svc (#717)
  • 3dd2a31d chore(deps): bump github.com/gin-gonic/gin from 1.8.1 to 1.9.0 (#724)
  • 15a229b5 docs(proposal): edges, buffers and buckets (#704)
  • 714c8036 chore(doc): update README with demo (#718)
  • 431778de doc: add overview (#713)
  • f518d99c feat: allowedLateness to support late data ingestion (#703)
  • 1efac426 fix: allow late message as long as window is not closed (#696)
  • 3da84fa5 fix: add wal dir x permission (#689)
  • 3a91cc52 chore(doc): refactor doc struct (#685)

Contributors

  • Derek Wang
  • Dillen Padhiar
  • Juanlu Yu
  • Julie Vogelman
  • Keran Yang
  • Vedant Gupta
  • Vigith Maurice
  • Yashash H L
  • dependabot[bot]
  • xdevxy

v0.8.1 (2023-05-30)

  • 4b119387 Update manifests to v0.8.1
  • 67277b79 fix: pipeline view fix (#755)
  • 7cb399e9 fix: toVertexPartitions for reduce was incorrectly populated to 1 (#756)
  • 16067af2 feat: use metrics to calculate vertex processing rate (#743)
  • 11cd8e9f feat: enable streaming message to next vertex when batch size is 1 (#709)
  • a5058840 fix: use int32 for message length (#750)
  • c602a520 feat: using one bucket for partitioned reduce watermark propagation (#742)
  • ccf79c6d fix(test): flakey test (#738)
  • af8e3346 refactor: buffer, edge, bucket (#733)
  • d57bfed4 feat: change baseHref for Numaflow UI. Fixes #375. (#698)
  • 37dfae58 fix: let kafka source crash and restart when there is any server side error (#735)
  • b7a0dda5 feat: Autoscale for Redis Streams Source (#726)
  • 5654e0af feat: Redis7 as an ISB svc (#717)
  • 9e8d8cc4 chore(deps): bump github.com/gin-gonic/gin from 1.8.1 to 1.9.0 (#724)

Contributors

  • Derek Wang
  • Dillen Padhiar
  • Julie Vogelman
  • Keran Yang
  • Vedant Gupta
  • Yashash H L
  • dependabot[bot]
  • xdevxy

v0.8.0 (2023-04-26)

  • e57ca739 Update manifests to v0.8.0
  • 652be8d6 feat: allowedLateness to support late data ingestion (#703)
  • 8e7e3b61 fix: allow late message as long as window is not closed (#696)
  • fcaed47d fix: add wal dir x permission (#689)
  • aae08fa1 chore(doc): refactor doc struct (#685)

Contributors

  • Derek Wang
  • Vigith Maurice

v0.8.0-rc1 (2023-04-14)

  • ca88313d Update manifests to v0.8.0-rc1
  • b83525df feat: introducing tags for conditional forwarding (#668)
  • a6e81746 feat: expose cpu/mem info to sidecar containers (#678)
  • c7b853aa feat: Redis Streams source fixes (#669)
  • 2f73b5b8 fix: skip empty Kafka partitions when calculating pending count (#666)
  • eeb37d8b feat: support for multi keys (#658)
  • 91b516b8 feat: Adds SASL (plain and gssapi) support for kafka sink (#656)
  • 196f887d fix: vertex overlapping watermark (#660)
  • 0db3248d feat: incremental search and namespace preview in search bar (#654)
  • 46bb8750 feat: integrate serde WAL (#650)
  • b0560876 fix: unit test (#653)
  • 1b0ea088 feat: handle idle watermark for reduce vertex (#627)
  • 33882628 feat: Redis streams source (#628)
  • d85bf93f feat: Adds SASL (plain and gssapi) support for kafka source (#643)
  • 60bb2bb9 feat: namespace scope api and disable namespace search on UI (#638)
  • 38b5a9ec fix: GetHeadWatermark Logic (#636)
  • 927b95cd feat: enable edge-level kill switch to drop messages when buffer is full, for the non-reduce forwarder (#634)
  • 0c79113c fix: IdleWatermark unit test (#640)
  • 924ad33e fix: desired replicas should not be greater than pending (#639)
  • 150c5c23 fix: add timeout to the test (#618)
  • 2f112fb2 feat: kustomize integration (#637)
  • 5062aac6 fix: exclude ack pending messages (#631)
  • e533ba35 feat: UI error component (#613)
  • 20aaca9d fix: do not update status.replicas until pod operation succeeds (#620)
  • cc62c81c feat: track and expose oldest work yet to be done to the reduce loop (#617)
  • 4bbe80bb feat: handle watermark barrier for map vertex (#607)
  • f9f05442 fix: corrected reduce vertex replica number. Fixes #593 (#616)
  • c9815132 feat: add API for pipeline status check. Fixes #407. (#599)
  • 5282766d chore(deps): bump webpack from 5.74.0 to 5.76.1 in /ui (#610)
  • 927bfc04 feat: use randomized shuffle using vertex name as the seed (#601)
  • d667d799 fix: ack the dropped messages as well (#603)
  • 64e17d88 feat: enable controller HA (#602)
  • 20ba722f feat: expose dnspolicy and dnsconfig to pod template (#598)
  • abfdd78f Chore: tickgen changes to test reduce pipelines (#587)
  • b2f8a12a feat: bidirectional streaming (#553)
  • 148663e7 feat: use customized binary serde for nats message payload (#585)
  • 8d339b68 fix: Idle watermark fix for read batch size > 0 and partial idle outgoing edges (#575)
  • 87ab1e3d feat: implement watermark propagation for source data transformer (#557)
  • d561867f feat: namespace search (#559)
  • 1b4800af fix: refine log for buffer validation. Fixes #185 (#573)
  • 4eb27eff feat: add readiness and liveness check for daemon server. Fixes #543 (#571)
  • 8ae2116d feat: marshal/unmarshal binary for read message (#565)
  • a0505e67 chore(deps): bump golang.org/x/net from 0.0.0-20220722155237-a158d28d115b to 0.7.0 (#568)
  • fbf36894 chore(deps): bump golang.org/x/text from 0.3.7 to 0.3.8 (#567)
  • 92c8009d feat: expose image pull policy to user defined containers (#563)
  • 88a41c2b fix: typos in reduce examples (#556)
  • 97567f3c feat: edge-watermark (#537)
  • 1e06ba2f feat: enable envFrom for user defined containers (#554)
  • 0dc85f69 feat: remove secret watch privilege dependency (#542)
  • 8b7e397e fix: Use a copied object to update (#541)
  • 943e7bd8 chore(deps): bump github.com/emicklei/go-restful from 2.9.5+incompatible to 2.16.0+incompatible (#539)
  • 93753c15 feat: improve reduce performance (#501)
  • b502fa93 feat: Offset time idle watermark put (#529)
  • 4551505c fix: securityContext not applied to container templates (#528)
  • 2727e62a feat: idle watermark v0 (#520)
  • 1e34e315 feat: Reduce UI Support (#500)
  • 39ecae42 feat: enable RuntimeClassName for vertex pod (#519)
  • b965318d feat: add builtin filter and event time extractor for source transformer (#517)
  • 077771ce chore(deps): bump ua-parser-js from 0.7.32 to 0.7.33 in /ui (#507)
  • 58b12ec3 Add an e2e test for source data transformer (#505)
  • 2af91933 feat: Implement source data transformer and apply to all existing sources (#487)
  • a3024f4e fix: -ve metrics and return early if isLate (#495)
  • 22986153 fix: JetStream context KV store/watch fix (#460)
  • 2177d621 doc: reduce persistent store (#458)
  • 3c621207 doc: reduce documentation (#448)
  • 784fe15c chore(deps): bump json5 from 1.0.1 to 1.0.2 in /ui (#454)
  • 659a98b5 refactor: simplify http request construction in test cases (#444)
  • cc9c194b refactor: use exact matching instead of regex to perform e2e data validation. (#443)
  • f7f712b3 doc: windowing fixed and sliding (#439)
  • 9ad504b5 refactor: move redis sink resources creation to E2ESuite (#437)
  • 0148258d refactor: a prototype for enhancing E2E test framework (#424)
  • 8579dc67 feat: pipeline watermark (#416)

Contributors

  • Alex Ang HH
  • Derek Wang
  • Dillen Padhiar
  • Juanlu Yu
  • Julie Vogelman
  • Keran Yang
  • Vedant Gupta
  • Vigith Maurice
  • Yashash H L
  • ashwinidulams
  • dependabot[bot]

v0.7.3 (2023-04-02)

  • 68a14793 Update manifests to v0.7.3
  • a17a41df feat: integrate serde WAL (#650)
  • 096c6acf fix: unit test (#653)
  • cce50ffd feat: handle idle watermark for reduce vertex (#627)
  • 968cc5f5 feat: Redis streams source (#628)
  • 38baae8a feat: Adds SASL (plain and gssapi) support for kafka source (#643)
  • 563b85b1 feat: namespace scope api and disable namespace search on UI (#638)
  • c8194690 fix: GetHeadWatermark Logic (#636)
  • d37f4db5 feat: enable edge-level kill switch to drop messages when buffer is full, for the non-reduce forwarder (#634)
  • a0dce69a fix: IdleWatermark unit test (#640)
  • f840e1c3 fix: desired replicas should not be greater than pending (#639)
  • 0ca5630e fix: add timeout to the test (#618)
  • b049f0b4 feat: kustomize integration (#637)
  • 01305ea3 fix: exclude ack pending messages (#631)
  • 4ec4b3d7 feat: UI error component (#613)
  • 90ca505b fix: do not update status.replicas until pod operation succeeds (#620)
  • 339db43b feat: track and expose oldest work yet to be done to the reduce loop (#617)
  • 1ce4c383 feat: handle watermark barrier for map vertex (#607)
  • 90dbe1fa fix: corrected reduce vertex replica number. Fixes #593 (#616)
  • a155f2af feat: add API for pipeline status check. Fixes #407. (#599)
  • d9e3a56f chore(deps): bump webpack from 5.74.0 to 5.76.1 in /ui (#610)

Contributors

  • Alex Ang HH
  • Derek Wang
  • Dillen Padhiar
  • Juanlu Yu
  • Julie Vogelman
  • Keran Yang
  • Vedant Gupta
  • Vigith Maurice
  • Yashash H L
  • dependabot[bot]

v0.7.2 (2023-03-13)

  • 1196a24b Update manifests to v0.7.2
  • a6f64f8d feat: use randomized shuffle using vertex name as the seed (#601)
  • 16b20a35 fix: ack the dropped messages as well (#603)
  • 4cdef174 feat: enable controller HA (#602)
  • d2e0513c feat: expose dnspolicy and dnsconfig to pod template (#598)
  • 6002c829 Chore: tickgen changes to test reduce pipelines (#587)
  • b1aee945 feat: bidirectional streaming (#553)
  • 67fc688a feat: use customized binary serde for nats message payload (#585)
  • cc104199 fix: Idle watermark fix for read batch size > 0 and partial idle outgoing edges (#575)
  • df1574da feat: implement watermark propagation for source data transformer (#557)
  • 45d5c396 feat: namespace search (#559)
  • b14d470f fix: refine log for buffer validation. Fixes #185 (#573)
  • a8e8bb15 feat: add readiness and liveness check for daemon server. Fixes #543 (#571)
  • fd6acb6d feat: marshal/unmarshal binary for read message (#565)
  • d1032b4c chore(deps): bump golang.org/x/net from 0.0.0-20220722155237-a158d28d115b to 0.7.0 (#568)
  • fd00ebdf chore(deps): bump golang.org/x/text from 0.3.7 to 0.3.8 (#567)
  • 05ec77f5 feat: expose image pull policy to user defined containers (#563)
  • ecbe3a00 fix: typos in reduce examples (#556)
  • 0dc3f5c6 feat: edge-watermark (#537)
  • 77298c85 feat: enable envFrom for user defined containers (#554)

Contributors

  • Derek Wang
  • Dillen Padhiar
  • Juanlu Yu
  • Julie Vogelman
  • Keran Yang
  • Vedant Gupta
  • Vigith Maurice
  • Yashash H L
  • ashwinidulams
  • dependabot[bot]

v0.7.1 (2023-02-14)

  • 92925c15 Update manifests to v0.7.1
  • 2f8e147a feat: remove secret watch privilege dependency (#542)
  • f8e7daae fix: Use a copied object to update (#541)
  • 98de2459 chore(deps): bump github.com/emicklei/go-restful from 2.9.5+incompatible to 2.16.0+incompatible (#539)
  • 0df812c1 feat: improve reduce performance (#501)
  • ab49de68 feat: Offset time idle watermark put (#529)
  • c0aa7c1e fix: securityContext not applied to container templates (#528)
  • ac33fb02 feat: idle watermark v0 (#520)
  • 2844cfb6 feat: Reduce UI Support (#500)
  • e701180d feat: enable RuntimeClassName for vertex pod (#519)
  • bb94f631 feat: add builtin filter and event time extractor for source transformer (#517)
  • 4562196d chore(deps): bump ua-parser-js from 0.7.32 to 0.7.33 in /ui (#507)
  • 764cefda Add an e2e test for source data transformer (#505)
  • 7665d6ce feat: Implement source data transformer and apply to all existing sources (#487)
  • d0226084 fix: -ve metrics and return early if isLate (#495)

Contributors

  • Derek Wang
  • Juanlu Yu
  • Keran Yang
  • Vedant Gupta
  • Vigith Maurice
  • ashwinidulams
  • dependabot[bot]

v0.7.0 (2023-01-13)

  • 734e5d3b Update manifests to v0.7.0
  • 5d6c5336 fix: JetStream context KV store/watch fix (#460)
  • d6152e77 doc: reduce persistent store (#458)
  • ac77656d doc: reduce documentation (#448)
  • 257356af chore(deps): bump json5 from 1.0.1 to 1.0.2 in /ui (#454)
  • 7752db4b refactor: simplify http request construction in test cases (#444)
  • 1a10af4c refactor: use exact matching instead of regex to perform e2e data validation. (#443)
  • 2777e27a doc: windowing fixed and sliding (#439)
  • 70fc008f refactor: move redis sink resources creation to E2ESuite (#437)
  • 6c078b42 refactor: a prototype for enhancing E2E test framework (#424)
  • e7021c9a feat: pipeline watermark (#416)

Contributors

  • Derek Wang
  • Juanlu Yu
  • Keran Yang
  • Vedant Gupta
  • Vigith Maurice
  • dependabot[bot]

v0.7.0-rc1 (2022-12-16)

  • 71887db5 Update manifests to v0.7.0-rc1
  • dda4835d feat: reduce metrics. Closes #313 (#414)
  • 85dbe4d7 feat: udsink grpc stream (#421)
  • fa07587f chore(doc): scope UDF under a dir (#426)
  • 0a911da9 feat: sliding window. closes #339 (#354)
  • a46fb964 refactor: nats/jetstream testing (#418)
  • 13d95c48 feat: nats as source (#411)
  • f1e7c737 fix: adding lock while discovering partitions, Closes #412 (#413)
  • 3b64d674 fix(test): e2e-api-pod can not start on M1 mac (#410)
  • 6504a562 fix: getWatermark to return-1 if any processor returns -1 (#402)
  • d4d22041 fix: e2e testing for PBQ WAL with reduce pipeline (#393)
  • 80e97850 feat: add Grafana instruction and a dashboard template. Closes #287 (#381)
  • 2f94a915 fix: unit tests for replay. Closes #373 (#377)
  • 8f367ab2 chore(docs): update docs (#380)
  • efe4d41c fix: best effort processing during SIGTERM. Closes #371 (#372)
  • 7e041d87 feat(wal): First pass to implement WAL and hook to PBQ store. (#344)
  • 256e66b3 feat: watermark otwatcher enhancement (#364)
  • f8170577 refactor(docs): group docs in categories (#362)
  • 1a5d424f chore(deps): bump loader-utils from 2.0.3 to 2.0.4 in /ui (#356)
  • 6c8f03f2 fix(controller): vertex nil check for edge listing. Fixes #352 (#353)
  • f254c28a fix: data race in pbq manager. Closes #348 (#349)
  • bc359457 Chore: Windower interface. closes #234 (#340)
  • 3206bd12 feat: add minikube, kind and podman support (#206)
  • d40ecdaa refactor: Close watermark fetcher and publisher correctly (#336)
  • 0d8f659e passing window information inside the context (#341)
  • 89516277 feat: timestamp in UI to display milliseconds. closes #280 (#337)
  • 5c43f5aa Simple reduce pipeline. Fixes #289 (#317)
  • 7f5d86c3 feat: add blackhole sink. Closes #329 (#330)
  • 10f355c3 fix: move watermark based on the head of the read batch (#332)
  • b2b975f3 feat: configurable jetstream storage (#328)
  • 3fcf637c feat: support adding sidecars in vertex pods. Closes #323 (#325)
  • 6eab1b5b feat: populate watermark settings to vertex spec. Closes #320 (#321)
  • 2355978b doc: add few use cases (#318)
  • bfc1eb60 Chore: run in sdks-e2e tests, python-udsink log check before go-udsink (#315)
  • cda41eca fix: jetstream build watermark progressors bug (#316)
  • bfab8f1d feat: update watermark offset bucket implementation (#307)
  • 1d86aa5f feat: shuffling support (#306)
  • b817920a feat: customize init-container resources. Closes #303 (#304)
  • 0548d4d3 feat: watermark - remove non-share OT bucket option (#302)
  • cc44875b feat: customization for batch jobs. Closes #259 (#300)
  • d16015f3 refactor: abstract pod template (#296)
  • 4550f459 feat: customization for daemon deployment. Closes #223 (#290)
  • d61377a5 feat: add pvc support for reduce vertex PBQ (#292)
  • b0e3f944 fix(doc): hyperlink for security doc (#288)
  • 6c61728d feat: support adding init containers to vertices. Closes #284 (#285)
  • 88cf272c fix: retry when getting EOF error at E2E test (#281)
  • 1436071c feat: Watermark millisecond. Fixes #201 (#278)
  • 7a7e7945 feat: add pipeline node counts and age to printcolumn. Closes #267 (#282)
  • 5883e973 feat: introduce reduce UDF. Closes #246 (#262)
  • a0dc17f8 feat: add pandoc to required tools development doc. Closes #276 (#277)
  • 284be2d6 feat: add isbsvc type and age to printcolumn. Closes #268 (#275)
  • 7bb689bc fix: watermark consumer fix (#273)
  • 8ff9e28e refactor: generalize watermark fetching as an interface of ISB service. Fixes #252 (#263)
  • 8e038d1e fix: set default property values for minimal CRD installation (#264)
  • 57df392f fix: validate only one isbsvc implementation type is defined. Fixes #269 (#271)
  • 21378a36 fix: main branch make build failure: math.MaxInt64 for int type (#265)
  • 3d9997d6 fix: nil pointer deref when running example with minimal CRD. Fixes #260 (#261)
  • 4b0cbc37 fix: retry only the failed offsets (#255)
  • 27e6a875 fix: re-enable build constraint on processor manager test. Fixes #256 (#257)
  • 98b3ec4d fix: container resource for jetstream isbsvc. Fixes #253 (#254)
  • e615e16e fix: update vertex watermark fetching logic. Fixes: #134 (#245)
  • 30c734bd fix: watermark watcher leak (#242)
  • 3d29f79d fix(docs): fix a typo (#241)
  • 0370fd6c feat: Support running UX server with namespace scope. Fixes #248 (#249)
  • 29f15d57 fix(manifests): Include ServiceAccount in namespace scoped install (#240)
  • 6870d2a4 fix: Watermark close fix and removed the nil check (#238)
  • 998e3988 fix: skip publishing watermarks to unexpected vertices. Fixes #235 (#236)
  • fff05f32 fix: update default watermark to -1. Fixes #133 (#218)
  • a23e3592 feat: support disabling TLS and changing port for UX server (#228)
  • 5a4387c7 feat: reducer for stream aggregation without fault tolerance (#208)
  • fc2ba4e9 feat: in-memory watermark store for better testing (#216)
  • c89aef31 Add USERS.md (#221)
  • 2377c4c6 fix(watermark): generator should not publish wm for every message (#217)

Contributors

  • David Seapy
  • Derek Wang
  • Ed Lee
  • Juanlu Yu
  • Keran Yang
  • Shay Dratler
  • SianLoong
  • Vedant Gupta
  • Vigith Maurice
  • Yashash H L
  • ashwinidulams
  • dependabot[bot]
  • xdevxy

v0.6.5 (2022-12-07)

  • 845c9594 Update manifests to v0.6.5
  • 676ea1c6 fix: adding lock while discovering partitions, Closes #412 (#413)
  • c439a6a1 fix(test): e2e-api-pod can not start on M1 mac (#410)
  • 115a69d6 fix: getWatermark to return-1 if any processor returns -1 (#402)
  • e6e24eef fix: e2e testing for PBQ WAL with reduce pipeline (#393)
  • 7ef3d47c feat: add Grafana instruction and a dashboard template. Closes #287 (#381)
  • 13ce4d27 fix: unit tests for replay. Closes #373 (#377)

Contributors

  • Derek Wang
  • Keran Yang
  • Yashash H L
  • xdevxy

v0.6.4 (2022-11-28)

  • ad9719a6 Update manifests to v0.6.4
  • c5e82176 chore(docs): update docs (#380)
  • 1b244c1a fix: best effort processing during SIGTERM. Closes #371 (#372)
  • 9bb8ebd5 feat(wal): First pass to implement WAL and hook to PBQ store. (#344)

Contributors

  • Derek Wang
  • Vigith Maurice
  • xdevxy

v0.6.3 (2022-11-18)

  • 3cf391b1 Update manifests to v0.6.3
  • bec020b7 feat: watermark otwatcher enhancement (#364)
  • 2b5478fc refactor(docs): group docs in categories (#362)
  • 6d9e129b chore(deps): bump loader-utils from 2.0.3 to 2.0.4 in /ui (#356)
  • 77364a4d fix(controller): vertex nil check for edge listing. Fixes #352 (#353)
  • 7db4fe56 fix: data race in pbq manager. Closes #348 (#349)
  • 6f5e83a7 Chore: Windower interface. closes #234 (#340)
  • 24ba5157 feat: add minikube, kind and podman support (#206)
  • 12c6ca52 refactor: Close watermark fetcher and publisher correctly (#336)
  • 2a8b97e1 passing window information inside the context (#341)
  • d52a5a75 feat: timestamp in UI to display milliseconds. closes #280 (#337)
  • de9059cf Simple reduce pipeline. Fixes #289 (#317)
  • 3d936a50 feat: add blackhole sink. Closes #329 (#330)
  • 45905475 fix: move watermark based on the head of the read batch (#332)
  • 049e5c66 feat: configurable jetstream storage (#328)
  • ee5cd642 feat: support adding sidecars in vertex pods. Closes #323 (#325)

Contributors

  • David Seapy
  • Derek Wang
  • Juanlu Yu
  • Shay Dratler
  • Vedant Gupta
  • Vigith Maurice
  • Yashash H L
  • ashwinidulams
  • dependabot[bot]

v0.6.2 (2022-11-07)

  • 99be6c08 Update manifests to v0.6.2
  • dc733da1 feat: populate watermark settings to vertex spec. Closes #320 (#321)
  • 2b247cad doc: add few use cases (#318)
  • 07ffa168 Chore: run in sdks-e2e tests, python-udsink log check before go-udsink (#315)
  • 7b3285b9 fix: jetstream build watermark progressors bug (#316)
  • 1198a609 feat: update watermark offset bucket implementation (#307)
  • 34a6d709 feat: shuffling support (#306)
  • 448127ff feat: customize init-container resources. Closes #303 (#304)
  • 61cf2272 feat: watermark - remove non-share OT bucket option (#302)
  • 51c9ff42 feat: customization for batch jobs. Closes #259 (#300)
  • afbe2557 refactor: abstract pod template (#296)
  • 24089460 feat: customization for daemon deployment. Closes #223 (#290)
  • bedf567c feat: add pvc support for reduce vertex PBQ (#292)
  • 2341614b fix(doc): hyperlink for security doc (#288)
  • 6c05190d feat: support adding init containers to vertices. Closes #284 (#285)
  • dc96b872 fix: retry when getting EOF error at E2E test (#281)
  • f5db937c feat: Watermark millisecond. Fixes #201 (#278)
  • c1535365 feat: add pipeline node counts and age to printcolumn. Closes #267 (#282)

Contributors

  • David Seapy
  • Derek Wang
  • Juanlu Yu
  • Keran Yang
  • Vigith Maurice

v0.6.1 (2022-10-26)

  • 32b284f6 Update manifests to v0.6.1
  • 9684e161 fix(manifests): Include ServiceAccount in namespace scoped install (#240)
  • fe83918a fix(docs): fix a typo (#241)
  • f2094b4b feat: introduce reduce UDF. Closes #246 (#262)
  • e19a1e7d feat: add pandoc to required tools development doc. Closes #276 (#277)
  • 9a937118 feat: add isbsvc type and age to printcolumn. Closes #268 (#275)
  • f25e303e fix: watermark consumer fix (#273)
  • d2a3d908 refactor: generalize watermark fetching as an interface of ISB service. Fixes #252 (#263)
  • 5ffcadcc fix: set default property values for minimal CRD installation (#264)
  • 17a99564 fix: validate only one isbsvc implementation type is defined. Fixes #269 (#271)
  • 2272a1fc fix: main branch make build failure: math.MaxInt64 for int type (#265)
  • 02c31d27 fix: nil pointer deref when running example with minimal CRD. Fixes #260 (#261)
  • 391b53e1 fix: retry only the failed offsets (#255)
  • 7b42dc80 fix: re-enable build constraint on processor manager test. Fixes #256 (#257)
  • 34360490 fix: container resource for jetstream isbsvc. Fixes #253 (#254)
  • 33ce7422 fix: update vertex watermark fetching logic. Fixes: #134 (#245)
  • fd219a5c fix: watermark watcher leak (#242)
  • 979a3a3f feat: Support running UX server with namespace scope. Fixes #248 (#249)
  • 5e9d1c1c fix: Watermark close fix and removed the nil check (#238)
  • 340bd820 fix: skip publishing watermarks to unexpected vertices. Fixes #235 (#236)
  • 904b2cde fix: update default watermark to -1. Fixes #133 (#218)
  • 321e285f feat: support disabling TLS and changing port for UX server (#228)
  • d0d74e19 feat: reducer for stream aggregation without fault tolerance (#208)
  • 06a9b58a feat: in-memory watermark store for better testing (#216)
  • f25cc58e Add USERS.md (#221)
  • a37cece9 fix(watermark): generator should not publish wm for every message (#217)

Contributors

  • David Seapy
  • Derek Wang
  • Ed Lee
  • Juanlu Yu
  • Keran Yang
  • SianLoong
  • Yashash H L
  • ashwinidulams

v0.6.0 (2022-10-12)

  • 48aad5fc Update manifests to v0.6.0
  • 09ce54f1 fix(autoscaling): Ack pending should be included in total pending calculation (#212)
  • 9922787c fix(autoscaling): Skip autoscaling if vertex is not in running phase (#207)
  • bc2380a7 feat: ISBSVC add support for redis cluster mode (#195)
  • 72a96a58 refactor: move controllers package to pkg/reconciler (#192)
  • b1b78faa fix: update udf fetchWatermark and publishWatermark initial values (#193)
  • d4912600 fix(docs): readme for UI development (#181)
  • 6b121c6e feat: grpc udsink (#174)
  • 567da7b0 fix: numaflow-go udf example & docs (#177)
  • 4652f808 fix: use scale.max if it is set (#179)
  • 900314bc fix broken path (#176)
  • 3b02f2a6 feat: Shuffle implementation (#169)
  • 021bb9df feat: windowing operations (#157)
  • 7d411294 feat: watermark for sources (#159)
  • 5f5b2dfd fix: daemon service client memory leak (#161)
  • bfe96695 pbq implementation (#155)
  • 8dfedd83 feat: check if udf is running in liveness probe (#156)
  • 81e76d82 feat: Add udf grpc support Fixes #145 (#146)
  • 511faffc refactor: some refactor on watermark (#149)
  • 7fe40c42 fix: Fixed JS bug (#144)
  • 24a16a04 bug: watermark needs nil check
  • f4ed831b fix: pipeline UI broken when vertex scaling down to 0 (#132)
  • 0ae0377f feat: JetStream auto-reconnection (#127)
  • 2fc04eb3 feat: Add watermark for sink vertex (#124)
  • d958ee6d feat: autoscaling with back pressure factor (#123)
  • b1f77682 feat: add watermark to UI (#122)
  • 7feeaa87 feat: add processing rate to UI (#121)
  • 43fae931 feat: Expose watermark over HTTP (#120)
  • ec02304a fix: daemon service rest api not working (#119)
  • f3da56d3 chore(deps): bump terser from 5.14.1 to 5.14.2 in /ui (#117)
  • e2e63c84 feat: Numaflow autoscaling (#115)
  • e5da3f54 feat: watermark for headoffset (#116)
  • a45b2eed feat: support namespace scope installation (#112)
  • ce39199e feat: Expose ReadTimeoutSeconds on Vertex (#110)
  • 18ad1c5f fix: imagepullpocily for local testing (#113)
  • 469849b5 feat: removed udfWorkers from limits and added some docs (#103)
  • 3fada667 feat: Add icon and other minor changes (#94)
  • a81838d7 feat: end to end tickgen watermark validation (#98)
  • d7d93175 fix: Broken hyperlink (#96)
  • a2e07926 add no-op KV Store (#91)
  • 45c8cb69 feat: no operation watermark progressor (#90)
  • 448c229a feat: kafka source pending messages (#80)
  • 1aa39300 feat: Interface for Watermark (#82)
  • be78c523 feat: expose pending messages and processing rate (#79)
  • df30f2a8 feat: Added the right way to decipher from and to vertex (#78)
  • 639c459a feat: define buffer limits on edges (#70)
  • 41fdd38b feat: Merge UX server code (#67)
  • ced99079 feat: auto-scaling (part 1) (#59)
  • fd5b3741 Added name to service spec (#58)
  • dc2badfd feat: introduce source buffer and sink buffer (#53)
  • 4ed83a2a feat: async publishing for kafka sink (#51)
  • 9f9f5ba7 fix spelling errors (#48)
  • f423002e feat: controller to create buckets (#47)
  • 8328739c turn on watermark only if ENV value is true (#46)
  • 46f72e23 minimal end to end line-graph watermark integration (#43)
  • 1f8203f4 Fixed spelling error (#44)
  • f1e99eae Exponential buckets (#42)
  • dfcfdeba fix: different behavior for time.After in go 1.18 (#39)

Contributors

  • Chrome
  • Derek Wang
  • Juanlu Yu
  • Krithika3
  • Qianbo Huai
  • Saravanan Balasubramanian
  • Sidhant Kohli
  • Vigith Maurice
  • Yashash H L
  • dependabot[bot]

v0.5.6 (2022-09-19)

  • ac15d229 Update manifests to v0.5.6
  • f2363757 feat: grpc udsink (#174)
  • 2650c2de fix: numaflow-go udf example & docs (#177)
  • c44f733f fix: use scale.max if it is set (#179)
  • 39e92d06 fix broken path (#176)
  • 46ce0f87 feat: Shuffle implementation (#169)
  • 71ca00a1 feat: windowing operations (#157)
  • ca00b78f feat: watermark for sources (#159)
  • 91e21cee pbq implementation (#155)
  • 65424004 feat: check if udf is running in liveness probe (#156)
  • 79dce0b3 feat: Add udf grpc support Fixes #145 (#146)

Contributors

  • Chrome
  • Derek Wang
  • Juanlu Yu
  • Vigith Maurice
  • Yashash H L

v0.5.5 (2022-09-07)

  • 9aae638c Update manifests to v0.5.5
  • 32414325 fix: daemon service client memory leak (#161)
  • be47a26a refactor: some refactor on watermark (#149)
  • 857cce75 fix: Fixed JS bug (#144)
  • da16abc7 bug: watermark needs nil check
  • c9998a1c fix: pipeline UI broken when vertex scaling down to 0 (#132)

Contributors

  • Derek Wang
  • Krithika3
  • Vigith Maurice

v0.5.4 (2022-08-05)

  • 57513b40 Update manifests to v0.5.4
  • 94cdb82f feat: JetStream auto-reconnection (#127)
  • 8d835408 feat: Add watermark for sink vertex (#124)
  • 228ba321 feat: autoscaling with back pressure factor (#123)
  • 9833efdf feat: add watermark to UI (#122)
  • 0dab55d8 feat: add processing rate to UI (#121)
  • ffd38a15 feat: Expose watermark over HTTP (#120)
  • c09502a2 fix: daemon service rest api not working (#119)
  • ebc10f41 chore(deps): bump terser from 5.14.1 to 5.14.2 in /ui (#117)
  • 84490ca8 feat: Numaflow autoscaling (#115)
  • 32b98486 feat: watermark for headoffset (#116)
  • 283dae90 feat: support namespace scope installation (#112)
  • 8e612b1f feat: Expose ReadTimeoutSeconds on Vertex (#110)
  • d95d41bd fix: imagepullpocily for local testing (#113)

Contributors

  • Derek Wang
  • Krithika3
  • Saravanan Balasubramanian
  • Sidhant Kohli
  • Vigith Maurice
  • dependabot[bot]

v0.5.3 (2022-07-08)

  • efee5442 Update manifests to v0.5.3
  • 5895facd feat: removed udfWorkers from limits and added some docs (#103)
  • 0b75495f feat: Add icon and other minor changes (#94)
  • 7eb08f58 feat: end to end tickgen watermark validation (#98)
  • 3338e658 fix: Broken hyperlink (#96)
  • e3112229 add no-op KV Store (#91)
  • 5d2f90ed feat: no operation watermark progressor (#90)
  • f58d0f49 feat: kafka source pending messages (#80)
  • cbb16ca2 feat: Interface for Watermark (#82)
  • 5592bb1b feat: expose pending messages and processing rate (#79)
  • 06a3df2d feat: Added the right way to decipher from and to vertex (#78)
  • a0908ad4 feat: define buffer limits on edges (#70)
  • a1d36395 feat: Merge UX server code (#67)
  • 571c48eb feat: auto-scaling (part 1) (#59)
  • 1e0384ba Added name to service spec (#58)

Contributors

  • Derek Wang
  • Krithika3
  • Sidhant Kohli
  • Vigith Maurice

v0.5.2 (2022-06-13)

  • 2f2d10ce Update manifests to v0.5.2
  • cedd0d1f feat: introduce source buffer and sink buffer (#53)
  • d3301aa9 feat: async publishing for kafka sink (#51)
  • 2474eb8e fix spelling errors (#48)
  • c4a12f87 feat: controller to create buckets (#47)
  • eb97dc3b turn on watermark only if ENV value is true (#46)
  • f189ba30 minimal end to end line-graph watermark integration (#43)

Contributors

  • Derek Wang
  • Qianbo Huai
  • Vigith Maurice

v0.5.1 (2022-06-02)

  • bb9be807 Update manifests to v0.5.1
  • 912747eb Fixed spelling error (#44)
  • 3aeb33a8 Exponential buckets (#42)
  • 1d656829 fix: different behavior for time.After in go 1.18 (#39)

Contributors

  • Derek Wang
  • Krithika3