Data On Eks Versions Save

DoEKS is a tool to build, deploy and scale Data & ML Platforms on Amazon EKS

v1.0.2

9 months ago

What's Changed

remove deprecated format and increaes volume timeout to 60 seconds by @nabuskey in https://github.com/awslabs/data-on-eks/pull/119
EMR on EKS with Karpenter example with V5 EKS Blueprints by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/121
feat: EMR on EKS Fargate pattern with new approach by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/102
Update Adopters and Add policy of SSM to karpenter role by @srikaanthpenugonda in https://github.com/awslabs/data-on-eks/pull/123
Readme update by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/124
Data on EKS workshop module by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/125
Apache NiFi on EKS by @codesometech in https://github.com/awslabs/data-on-eks/pull/99
Cleanup scripts added by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/126
add workshop util by @lmouhib in https://github.com/awslabs/data-on-eks/pull/129
modifying job submission by changing to input variables rather than hardcoding by @dalbhanj in https://github.com/awslabs/data-on-eks/pull/130
Workshop content update by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/133
fix for karpenter provisioner by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/134
Bump webpack from 5.74.0 to 5.76.1 in /website by @dependabot in https://github.com/awslabs/data-on-eks/pull/135
adding ebs volume example by @dalbhanj in https://github.com/awslabs/data-on-eks/pull/132
Updated all the addons to use v4.25.0 by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/136
minor bug in compute provisioner script by @dalbhanj in https://github.com/awslabs/data-on-eks/pull/137
Correct URL in README.md by @mfjackson in https://github.com/awslabs/data-on-eks/pull/141
Fsx for lustre added as module to Karpenter example by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/139
Removing link from Adopters page by @dalbhanj in https://github.com/awslabs/data-on-eks/pull/142
added preserve field to true to avoid destroy issues by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/144
simplifying deployment experience by @dalbhanj in https://github.com/awslabs/data-on-eks/pull/143
chore: Remove accidental commits by @askulkarni2 in https://github.com/awslabs/data-on-eks/pull/145
feat: Updates to the Ray blueprint by @askulkarni2 in https://github.com/awslabs/data-on-eks/pull/111
doc: Add Kubeflow on AWS page by @askulkarni2 in https://github.com/awslabs/data-on-eks/pull/147
Improve guide by @dalbhanj in https://github.com/awslabs/data-on-eks/pull/148
EMR EKS Workshop module updates by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/146
RAID-0 config for NVMe based SSD EC2 instances with Karpenter Provisioners by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/156
Add CloudNativePG operator. by @jihed in https://github.com/awslabs/data-on-eks/pull/127
CloudNativePG by @jihed in https://github.com/awslabs/data-on-eks/pull/159
initialize terraform in install scripts by @nabuskey in https://github.com/awslabs/data-on-eks/pull/161
update secrets name by @jihed in https://github.com/awslabs/data-on-eks/pull/160
Fix documentation error in ray pytorch example by @elyall in https://github.com/awslabs/data-on-eks/pull/165
Spark operator refactor by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/170
fix pre-commit run by @jihed in https://github.com/awslabs/data-on-eks/pull/173
EMR EKS Blueprints leveraging the sub terraform modules by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/174
Updated broken links on Readme by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/177
Removed the cluster name variable by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/178
Bump node-fetch and docusaurus in /website by @dependabot in https://github.com/awslabs/data-on-eks/pull/168
Flink Operator on EKS by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/180
Security critical fixes for docusaurus by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/181
Bump xml2js and @aws-quickstart/eks-blueprints in /analytics/cdk/emr-eks by @dependabot in https://github.com/awslabs/data-on-eks/pull/183
add output for amp by @lmouhib in https://github.com/awslabs/data-on-eks/pull/184
delta lake example by @rajarshighosal in https://github.com/awslabs/data-on-eks/pull/182
updated website doc for delta lake example by @rajarshighosal in https://github.com/awslabs/data-on-eks/pull/188
Bump yaml from 2.1.3 to 2.2.2 in /analytics/cdk/emr-eks by @dependabot in https://github.com/awslabs/data-on-eks/pull/195
Dependabot security fix for GOT and TRIM packages by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/196
Monitoring ML Workflows using Kubeflow on Amazon EKS by @elamaran11 in https://github.com/awslabs/data-on-eks/pull/202
Fix region mismatch in XGBoost example by @raykrueger in https://github.com/awslabs/data-on-eks/pull/204
Spark Operator blueprint upgrade to EKS 1.26 by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/205
Fix region mismatch in pytorch example by @raykrueger in https://github.com/awslabs/data-on-eks/pull/207
Fix ray example docs by @raykrueger in https://github.com/awslabs/data-on-eks/pull/208
EMR on EKS with Karpenter updated to Kubernetes 1.26 by @mihneaspirescu in https://github.com/awslabs/data-on-eks/pull/210
Fixed intallation scripts and removed default region from variables.tf by @lusoal in https://github.com/awslabs/data-on-eks/pull/211
fixed broken link in spark observability module by @lusoal in https://github.com/awslabs/data-on-eks/pull/214
remove acl using bucket policy by @hitsub2 in https://github.com/awslabs/data-on-eks/pull/218
Website restructure and Dynamic EBS volumes for karpenter userdata by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/221
Fixed links for EMR EKS Inro page by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/222
EMR NVIDIA Spark Rapids Blueprint added by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/224
Modify Python script and execution shell script to customize Docker i… by @tanmatth in https://github.com/awslabs/data-on-eks/pull/225
Feat/emr eks spark operator by @lmouhib in https://github.com/awslabs/data-on-eks/pull/220
DataHub on EKS by @bbgu1 in https://github.com/awslabs/data-on-eks/pull/213
Kafka on EKS - EKS Bluprints V5 Mitigation by @AlyIbrahim in https://github.com/awslabs/data-on-eks/pull/169
pre-commit updates and vpc module version uplifted by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/227
EMR spark rapids website and XGBoost example update by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/229
Bump aws-cdk-lib from 2.51.1 to 2.80.0 in /analytics/cdk/emr-eks by @dependabot in https://github.com/awslabs/data-on-eks/pull/226
Reduce the number of times we run apply by @raykrueger in https://github.com/awslabs/data-on-eks/pull/233
Upgrade Airflow to 2.5.3 and EKS to 1.26 by @jagpk in https://github.com/awslabs/data-on-eks/pull/230
Jupyterhub on EKS by @rajarshighosal in https://github.com/awslabs/data-on-eks/pull/228
Updated readme with correct links by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/234
feat: Refactor/update kafka v5 by @ovaleanu in https://github.com/awslabs/data-on-eks/pull/231
feat: Docs update for Jupyterhub and Spark rapids by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/236
fix: Bump cdk and remove package-lock.json by @lmouhib in https://github.com/awslabs/data-on-eks/pull/238
Bump stylelint from 14.16.1 to 15.10.1 in /website by @dependabot in https://github.com/awslabs/data-on-eks/pull/239
chore: Fix linting errors to pass CI checks by @bryantbiggs in https://github.com/awslabs/data-on-eks/pull/242
chore: Change Prometheus with for kube-prometheus-stack by @ovaleanu in https://github.com/awslabs/data-on-eks/pull/241
chore: Update module versions to standardize and remove deprecation errors/warnings by @bryantbiggs in https://github.com/awslabs/data-on-eks/pull/243
Bump semver from 5.7.1 to 5.7.2 in /website by @dependabot in https://github.com/awslabs/data-on-eks/pull/245
chore: Remove VPC CNI IRSA role; utilize permissions provided on EC2 node IAM role by @bryantbiggs in https://github.com/awslabs/data-on-eks/pull/244
chore: Change Prometheus to kube-prometheus-stack by @ovaleanu in https://github.com/awslabs/data-on-eks/pull/246
chore: Update Spark operator example to use new bootstrap flag for instance store volume RAID0 configuration by @bryantbiggs in https://github.com/awslabs/data-on-eks/pull/237
chore: Update Karpenter provisioner codeblock for self-managed Airflow by @bryantbiggs in https://github.com/awslabs/data-on-eks/pull/247
fix: Update Spark install script to remove target apply on addons modules by @bryantbiggs in https://github.com/awslabs/data-on-eks/pull/250
chore: Replace local EMR on EKS module with virtual cluster sub-module from terraform-aws-emr by @bryantbiggs in https://github.com/awslabs/data-on-eks/pull/249
chore: Emr-eks-karpenter blueprint updates by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/252
feat: Trainium blueprint Part1 by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/251
fix: Added a fix for Grafana to use sigv4 authentication by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/253
chore: README Update with logo by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/254
chore: Add remote AMP for Grafana by @ovaleanu in https://github.com/awslabs/data-on-eks/pull/256
Bump word-wrap from 1.2.3 to 1.2.4 in /website by @dependabot in https://github.com/awslabs/data-on-eks/pull/258
chore: EMR Spark Rapids upgrade by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/259
feat: EMR on EKS with Karpenter Managed Endpoint by @youngjeong46 in https://github.com/awslabs/data-on-eks/pull/261
mwaa v5 migration by @bbgu1 in https://github.com/awslabs/data-on-eks/pull/248

New Contributors

@nabuskey made their first contribution in https://github.com/awslabs/data-on-eks/pull/119
@srikaanthpenugonda made their first contribution in https://github.com/awslabs/data-on-eks/pull/123
@codesometech made their first contribution in https://github.com/awslabs/data-on-eks/pull/99
@mfjackson made their first contribution in https://github.com/awslabs/data-on-eks/pull/141
@askulkarni2 made their first contribution in https://github.com/awslabs/data-on-eks/pull/145
@jihed made their first contribution in https://github.com/awslabs/data-on-eks/pull/127
@elyall made their first contribution in https://github.com/awslabs/data-on-eks/pull/165
@rajarshighosal made their first contribution in https://github.com/awslabs/data-on-eks/pull/182
@elamaran11 made their first contribution in https://github.com/awslabs/data-on-eks/pull/202
@raykrueger made their first contribution in https://github.com/awslabs/data-on-eks/pull/204
@mihneaspirescu made their first contribution in https://github.com/awslabs/data-on-eks/pull/210
@lusoal made their first contribution in https://github.com/awslabs/data-on-eks/pull/211
@hitsub2 made their first contribution in https://github.com/awslabs/data-on-eks/pull/218
@tanmatth made their first contribution in https://github.com/awslabs/data-on-eks/pull/225
@bbgu1 made their first contribution in https://github.com/awslabs/data-on-eks/pull/213
@jagpk made their first contribution in https://github.com/awslabs/data-on-eks/pull/230
@ovaleanu made their first contribution in https://github.com/awslabs/data-on-eks/pull/231
@bryantbiggs made their first contribution in https://github.com/awslabs/data-on-eks/pull/242
@youngjeong46 made their first contribution in https://github.com/awslabs/data-on-eks/pull/261

Full Changelog: https://github.com/awslabs/data-on-eks/compare/v1.0.1...v1.0.2

v1.0.1

1 year ago

NOTE: This is the final release for Data on EKS Version1 blueprints(Final Tag v1.0.1). We will be updating the DoEKS blueprints with major changes and it will be published under Version2 onwards

What's Changed

Bump loader-utils from 2.0.3 to 2.0.4 in /website by @dependabot in https://github.com/awslabs/data-on-eks/pull/62
fix broken link after folder structure change by @season1946 in https://github.com/awslabs/data-on-eks/pull/63
Feat: Apache Kafka using Strimzi on EKS by @AlyIbrahim in https://github.com/awslabs/data-on-eks/pull/47
Fix link to Kafka docs. by @brianhammons in https://github.com/awslabs/data-on-eks/pull/64
Better positioning EMR on EKS by @dalbhanj in https://github.com/awslabs/data-on-eks/pull/72
add cdk blueprint for emr-eks by @lmouhib in https://github.com/awslabs/data-on-eks/pull/45
Architecture diagram for home page by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/73
Update import for emr on eks blueprint by @lmouhib in https://github.com/awslabs/data-on-eks/pull/74
Add analytics reference architecture blog by @lmouhib in https://github.com/awslabs/data-on-eks/pull/75
Fix the aws cli command by @lmouhib in https://github.com/awslabs/data-on-eks/pull/78
VPC endpoints added;s3 state for e2e tests by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/77
Add storage considerations and managed alternatives sections to Kafka… by @brianhammons in https://github.com/awslabs/data-on-eks/pull/79
[feat] ACK EMR on EKS Controller pattern and example by @season1946 in https://github.com/awslabs/data-on-eks/pull/81
[enhancement] Change argo workflow example to create spark app in a new ns by @season1946 in https://github.com/awslabs/data-on-eks/pull/84
Feat: Observability on Spark on EKS by @season1946 in https://github.com/awslabs/data-on-eks/pull/85
EMR on EKS Doc improvement by @dalbhanj in https://github.com/awslabs/data-on-eks/pull/83
EMR on EKS Karpenter with YuniKorn updates by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/87
fixing bug in CloudWatch Logs autocreate in helm bcoz terraform alrea… by @dalbhanj in https://github.com/awslabs/data-on-eks/pull/88
Add Troubleshooting page by @dalbhanj in https://github.com/awslabs/data-on-eks/pull/90
[enhancement] update argo workflows based on blog comments by @season1946 in https://github.com/awslabs/data-on-eks/pull/89
Bump json5 from 2.2.1 to 2.2.3 in /website by @dependabot in https://github.com/awslabs/data-on-eks/pull/91
Bump json5 from 2.2.1 to 2.2.3 in /analytics/cdk/emr-eks by @dependabot in https://github.com/awslabs/data-on-eks/pull/92
Bump ua-parser-js from 0.7.31 to 0.7.33 in /website by @dependabot in https://github.com/awslabs/data-on-eks/pull/96
Bump http-cache-semantics from 4.1.0 to 4.1.1 in /website by @dependabot in https://github.com/awslabs/data-on-eks/pull/100
Bump @sideway/formula from 3.0.0 to 3.0.1 in /website by @dependabot in https://github.com/awslabs/data-on-eks/pull/103
Website update to [email protected] by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/106
Bump node-fetch and docusaurus in /website by @dependabot in https://github.com/awslabs/data-on-eks/pull/108
Data on EKS Logo added with footer icons by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/109
Removing the Robot font by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/110
Added EMR cluster Name to output.tf so instructions work by @rbarcia in https://github.com/awslabs/data-on-eks/pull/113
Spark Live UI added by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/114

New Contributors

@AlyIbrahim made their first contribution in https://github.com/awslabs/data-on-eks/pull/47
@brianhammons made their first contribution in https://github.com/awslabs/data-on-eks/pull/64
@dalbhanj made their first contribution in https://github.com/awslabs/data-on-eks/pull/72
@lmouhib made their first contribution in https://github.com/awslabs/data-on-eks/pull/45
@rbarcia made their first contribution in https://github.com/awslabs/data-on-eks/pull/113

Full Changelog: https://github.com/awslabs/data-on-eks/compare/v1.0.0...v1.0.1

v1.0.0

1 year ago

What's Changed

🚀 Initial commit for DoEKS by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/1
removed package-lock.json file by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/2
Adding package-lock.json file by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/3
Fixing the git node package dependa alert by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/4
Fix to github pages deployment by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/5
Updated Test environment by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/6
removed deployment branch as main for website by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/7
changed publish dire to ./website by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/8
baseurl for the site changed by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/9
Searchbar added to website by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/32
Karpenter Graviton Provisioner support for EMR on EKS by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/37
Doc updates for Airflow and fixes to spark operator by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/38
Trigger EMR on EKS jobs with Apache Airflow by @season1946 in https://github.com/awslabs/data-on-eks/pull/39
Website updates by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/40
🦄 Apache YuniKorn batch scheduler with EMR on EKS by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/42
Updates to github workflows by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/44
EMR on EKS Gang scheduling example by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/49
Added FluentBit config to write logs to S3 by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/50
MWAA doc fix by @season1946 in https://github.com/awslabs/data-on-eks/pull/48
Argo workflows on EKS example and doc by @season1946 in https://github.com/awslabs/data-on-eks/pull/43
KubeCost addon for EMR on EKS Karpenter by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/52
argo workflow enhancement by @season1946 in https://github.com/awslabs/data-on-eks/pull/53
fix: Fixed the karpenter example hard-coded the provisioner region by @hustshawn in https://github.com/awslabs/data-on-eks/pull/55
Bump loader-utils from 2.0.2 to 2.0.3 in /website by @dependabot in https://github.com/awslabs/data-on-eks/pull/57
Bump minimatch, recursive-readdir and serve-handler in /website by @dependabot in https://github.com/awslabs/data-on-eks/pull/58
Folder structure and major updates to docs by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/59
Github workflow security updates by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/60
daily tag gh flow added by @vara-bonthu in https://github.com/awslabs/data-on-eks/pull/61

New Contributors

@vara-bonthu made their first contribution in https://github.com/awslabs/data-on-eks/pull/1
@season1946 made their first contribution in https://github.com/awslabs/data-on-eks/pull/39
@hustshawn made their first contribution in https://github.com/awslabs/data-on-eks/pull/55
@dependabot made their first contribution in https://github.com/awslabs/data-on-eks/pull/57

Full Changelog: https://github.com/awslabs/data-on-eks/commits/v0.1