Numaflow Versions Save

Kubernetes-native platform to run massively parallel data/streaming jobs

v1.2.1

1 week ago

v1.2.0

2 weeks ago

Major Features 🔥

Introducing Session Window and Streaming Reduce Capability
Ability to pass headers from Source to Sink
Fallback Sink

🚨 Breaking Change 🚨

SDKs have to be updated to the latest version
Python SDK has interface changes.

New Features 🔥

feat: health status implementation by @kohlisid in https://github.com/numaproj/numaflow/pull/1406
feat: Session Window and Reduce Streaming by @yhl25 in https://github.com/numaproj/numaflow/pull/1384
feat: update tcp client connections by @kohlisid in https://github.com/numaproj/numaflow/pull/1429
feat: enhance autoscaling peeking logic by @akashjkhamkar in https://github.com/numaproj/numaflow/pull/1432
feat: pipeline health status for UI by @veds-g in https://github.com/numaproj/numaflow/pull/1460
feat: configure standardResources via controller configmap by @inishchith in https://github.com/numaproj/numaflow/pull/1490
feat: add support for SASL SCRAM 256 and 512 for Kafka by @Dnomyar in https://github.com/numaproj/numaflow/pull/1518
feat: noop persistence store for reduce by @yhl25 in https://github.com/numaproj/numaflow/pull/1532
feat: unaligned wal by @yhl25 in https://github.com/numaproj/numaflow/pull/1511
feat: numaflow package by @veds-g in https://github.com/numaproj/numaflow/pull/1579
feat: support headers for message by @yhl25 in https://github.com/numaproj/numaflow/pull/1578
feat: Fallback Sink by @yhl25 in https://github.com/numaproj/numaflow/pull/1669

Major Fixes 🐞

fix: terminate reduce vertex pods when pausing pipeline by @dpadhiar in https://github.com/numaproj/numaflow/pull/1481
fix: configmap const name by @whynowy in https://github.com/numaproj/numaflow/pull/1423
fix: server-secrets-init container restart by @veds-g in https://github.com/numaproj/numaflow/pull/1433
fix: Kafka source reads duplicated messages by @jy4096 in https://github.com/numaproj/numaflow/pull/1438
fix: UI Filter by status for pipelines doesn't work as expected by @inishchith in https://github.com/numaproj/numaflow/pull/1444
fix: GetDownstreamEdges is not cycle safe by @inishchith in https://github.com/numaproj/numaflow/pull/1447
fix: memory leak inside session windower by @yhl25 in https://github.com/numaproj/numaflow/pull/1445
fix: bug where dashed line is treated as permalink + spelling/formatting by @ayildirim21 in https://github.com/numaproj/numaflow/pull/1467
fix(controller): incorrect cpu/mem resources calculation by @whynowy in https://github.com/numaproj/numaflow/pull/1477
fix: add idle handler offset nil check by @jy4096 in https://github.com/numaproj/numaflow/pull/1489
fix: unknown for ISB details in pipeline card by @veds-g in https://github.com/numaproj/numaflow/pull/1497
fix: avoid panic when ctx is canceled by @yhl25 in https://github.com/numaproj/numaflow/pull/1515
fix: idle manager refactor for multi partitions by @jy4096 in https://github.com/numaproj/numaflow/pull/1512
fix: podSpec incorrectly configured in case template exist by @inishchith in https://github.com/numaproj/numaflow/pull/1516
fix: incorrect json schema for tls config by @whynowy in https://github.com/numaproj/numaflow/pull/1520
fix(controller): vertex template metadata nil check by @inishchith in https://github.com/numaproj/numaflow/pull/1527
fix: break from retry loop when key is not found by @yhl25 in https://github.com/numaproj/numaflow/pull/1535
fix: initialize inflightAcks channel to not nil channel by @afugazzotto in https://github.com/numaproj/numaflow/pull/1548
fix(codegen): protobuf upgrade by @whynowy in https://github.com/numaproj/numaflow/pull/1558
fix: error message that cause Buttons overflow in the UI by @a49ibrahim in https://github.com/numaproj/numaflow/pull/1591
fix: flaky e2e tests by @vigith in https://github.com/numaproj/numaflow/pull/1590
fix: dedup in user-defined source #1613
fix: allow pipeline to start with redis isbsvc (Fixes: #1513) #1567
fix: watermark progression during pods creation/deletion #1619
fix: numaflow package style fixes in #1622
fix: bug in late message handling for sliding window #1471
fix: race condition while publishing wm inside reduce #1599
fix: read-only view for UI #1628

New Contributors ❤️

@akashjkhamkar made their first contribution in https://github.com/numaproj/numaflow/pull/1432
@inishchith made their first contribution in https://github.com/numaproj/numaflow/pull/1444
@ayildirim21 made their first contribution in https://github.com/numaproj/numaflow/pull/1467
@syayi made their first contribution in https://github.com/numaproj/numaflow/pull/1510
@Dnomyar made their first contribution in https://github.com/numaproj/numaflow/pull/1518
@afugazzotto made their first contribution in https://github.com/numaproj/numaflow/pull/1548
@a49ibrahim made their first contribution in https://github.com/numaproj/numaflow/pull/1591
@QuentinFAIDIDE made their first contribution in https://github.com/numaproj/numaflow/pull/1567

Installation

kubectl create namespace numaflow-system

# Install numaflow controller and UI
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.2.0/install.yaml

# Install validating webhook (Optional)
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.2.0/validating-webhook-install.yaml

Full Changelog: https://github.com/numaproj/numaflow/compare/v1.1.0...v1.2.0

v1.2.0-rc5

3 weeks ago

Major Changes

Skip sdk compatibility check if server info does not contain sdk version (#1703)

Pull Requests

https://github.com/numaproj/numaflow/compare/v1.2.0-rc4...v1.2.0-rc5

Installation

kubectl create namespace numaflow-system

# Install numaflow controller and UI
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.2.0-rc5/install.yaml

# Install validating webhook (Optional)
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.2.0-rc5/validating-webhook-install.yaml

v1.2.0-rc4

1 month ago

Major Features 🔥

Fallback Sink https://github.com/numaproj/numaflow/issues/1660

🚨 Breaking Change 🚨

SDKs have to be updated to the latest version, user-defined sink has a breaking change.

Other changes

feat: expose controller leader election duration and renew opts #1657
chore: remove pods/exec from UX server RBAC #1659
chore: do not create init-secrets when auth is disabled #1661
fix: routing fixes #1671
fix: version downgrade for monaco-editor #1673
chore: add version compatibility check #1683

Full Changelog: https://github.com/numaproj/numaflow/compare/v1.2.0-rc3...v1.2.0-rc4

v1.2.0-rc3

1 month ago

Major Changes

Fix: pass headers to custom UDSink (#1653)
Fix: pass headers to Source Transformer (#1651)
Fix: avoid publishing watermarks for duplicate messages (#1649)
Fix: Dedup not working for multi-partitioned edge (#1639)
Fix: readonly view (#1640)

Pull Requests

https://github.com/numaproj/numaflow/compare/v1.2.0-rc2...v1.2.0-rc3

Installation

kubectl create namespace numaflow-system

# Install numaflow controller and UI
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.2.0-rc3/install.yaml

# Install validating webhook (Optional)
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.2.0-rc3/validating-webhook-install.yaml

v1.2.0-rc2

1 month ago

Major Changes

Fix: dedup in user-defined source #1613
Fix: allow pipeline to start with redis isbsvc (Fixes: #1513) #1567
Fix: watermark progression during pods creation/deletion #1619
Fix: numaflow package style fixes in #1622
Fix: bug in late message handling for sliding window #1471
Fix: race condition while publishing wm inside reduce #1599
Feat: read-only view for UI #1628

Pull Requests

https://github.com/numaproj/numaflow/compare/v1.2.0-rc1...v1.2.0-rc2

Installation

kubectl create namespace numaflow-system

# Install numaflow controller and UI
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.2.0-rc2/install.yaml

# Install validating webhook (Optional)
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.2.0-rc2/validating-webhook-install.yaml

v1.2.0-rc1

1 month ago

Major Features 🔥

Introducing Session Window and Streaming Reduce Capability
Ability to pass headers from Source to Sink

🚨 Breaking Change 🚨

SDKs have to be updated to the latest version
Python SDK has interface changes.

New Features 🔥

feat: health status implementation by @kohlisid in https://github.com/numaproj/numaflow/pull/1406
feat: Session Window and Reduce Streaming by @yhl25 in https://github.com/numaproj/numaflow/pull/1384
feat: update tcp client connections by @kohlisid in https://github.com/numaproj/numaflow/pull/1429
feat: enhance autoscaling peeking logic by @akashjkhamkar in https://github.com/numaproj/numaflow/pull/1432
feat: pipeline health status for UI by @veds-g in https://github.com/numaproj/numaflow/pull/1460
feat: terminate reduce vertex pods when pausing pipeline by @dpadhiar in https://github.com/numaproj/numaflow/pull/1481
feat: configure standardResources via controller configmap by @inishchith in https://github.com/numaproj/numaflow/pull/1490
feat: add support for SASL SCRAM 256 and 512 for Kafka by @Dnomyar in https://github.com/numaproj/numaflow/pull/1518
feat: noop persistence store for reduce by @yhl25 in https://github.com/numaproj/numaflow/pull/1532
feat: unaligned wal by @yhl25 in https://github.com/numaproj/numaflow/pull/1511
feat: numaflow package by @veds-g in https://github.com/numaproj/numaflow/pull/1579
feat: support headers for message by @yhl25 in https://github.com/numaproj/numaflow/pull/1578

Major Fixes 🐞

fix: configmap const name by @whynowy in https://github.com/numaproj/numaflow/pull/1423
fix: server-secrets-init container restart by @veds-g in https://github.com/numaproj/numaflow/pull/1433
fix: Kafka source reads duplicated messages by @jy4096 in https://github.com/numaproj/numaflow/pull/1438
fix: UI Filter by status for pipelines doesn't work as expected by @inishchith in https://github.com/numaproj/numaflow/pull/1444
fix: GetDownstreamEdges is not cycle safe by @inishchith in https://github.com/numaproj/numaflow/pull/1447
fix: memory leak inside session windower by @yhl25 in https://github.com/numaproj/numaflow/pull/1445
fix: bug where dashed line is treated as permalink + spelling/formatting by @ayildirim21 in https://github.com/numaproj/numaflow/pull/1467
fix(controller): incorrect cpu/mem resources calculation by @whynowy in https://github.com/numaproj/numaflow/pull/1477
fix: add idle handler offset nil check by @jy4096 in https://github.com/numaproj/numaflow/pull/1489
fix: unknown for ISB details in pipeline card by @veds-g in https://github.com/numaproj/numaflow/pull/1497
fix: avoid panic when ctx is canceled by @yhl25 in https://github.com/numaproj/numaflow/pull/1515
fix: idle manager refactor for multi partitions by @jy4096 in https://github.com/numaproj/numaflow/pull/1512
fix: podSpec incorrectly configured in case template exist by @inishchith in https://github.com/numaproj/numaflow/pull/1516
fix: incorrect json schema for tls config by @whynowy in https://github.com/numaproj/numaflow/pull/1520
fix(controller): vertex template metadata nil check by @inishchith in https://github.com/numaproj/numaflow/pull/1527
fix: break from retry loop when key is not found by @yhl25 in https://github.com/numaproj/numaflow/pull/1535
fix: initialize inflightAcks channel to not nil channel by @afugazzotto in https://github.com/numaproj/numaflow/pull/1548
fix(codegen): protobuf upgrade by @whynowy in https://github.com/numaproj/numaflow/pull/1558
fix: error message that cause Buttons overflow in the UI by @a49ibrahim in https://github.com/numaproj/numaflow/pull/1591
fix: flaky e2e tests by @vigith in https://github.com/numaproj/numaflow/pull/1590

New Contributors ❤️

@akashjkhamkar made their first contribution in https://github.com/numaproj/numaflow/pull/1432
@inishchith made their first contribution in https://github.com/numaproj/numaflow/pull/1444
@ayildirim21 made their first contribution in https://github.com/numaproj/numaflow/pull/1467
@syayi made their first contribution in https://github.com/numaproj/numaflow/pull/1510
@Dnomyar made their first contribution in https://github.com/numaproj/numaflow/pull/1518
@afugazzotto made their first contribution in https://github.com/numaproj/numaflow/pull/1548
@a49ibrahim made their first contribution in https://github.com/numaproj/numaflow/pull/1591

Installation

kubectl create namespace numaflow-system

# Install numaflow controller and UI
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.2.0-rc1/install.yaml

# Install validating webhook (Optional)
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.2.0-rc1/validating-webhook-install.yaml

Full Changelog: https://github.com/numaproj/numaflow/compare/v1.1.0...v1.2.0-rc1

v1.1.7

2 months ago

Major Changes

Fixed a Kafka source issue to avoid potential dead lock (#1548)
A temporary solution to support including message headers for Kafka source (#1569) - Be aware this might be changed later on.

Pull Requests

https://github.com/numaproj/numaflow/compare/v1.1.6...v1.1.7

Installation

kubectl create namespace numaflow-system

# Install numaflow controller and UI
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.1.7/install.yaml

# Install validating webhook (Optional)
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.1.7/validating-webhook-install.yaml

v1.1.6

2 months ago

Major Changes

Fixed controller config concurrent loding issue [#1499]
Limit scale.max has to be > 0 [#1500]
Fixed a controller bug about applying pod template [#1516]
Fixed incorrect json tag for TLS configuration [#1520]
Fixed a bug in reduce code logic after getting sigterm
Fixed a nil check bug in the controller about vertex template metadata [#1527]
Added SASL SCRAM 256 and 512 support for Kafka source [#1518]
Upgrade sarama lib version to 1.43 [#1522]

Pull Requests

https://github.com/numaproj/numaflow/compare/v1.1.5...v1.1.6

Installation

kubectl create namespace numaflow-system

# Install numaflow controller and UI
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.1.6/install.yaml

# Install validating webhook (Optional)
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.1.6/validating-webhook-install.yaml

v1.1.5

3 months ago

Major Changes

Fixed cpu/mem resources calculation issue [#1477]

Pull Requests

https://github.com/numaproj/numaflow/compare/v1.1.4...v1.1.5

Installation

kubectl create namespace numaflow-system

# Install numaflow controller and UI
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.1.5/install.yaml

# Install validating webhook (Optional)
kubectl apply -n numaflow-system -f https://github.com/numaproj/numaflow/releases/download/v1.1.5/validating-webhook-install.yaml