πͺ΄ Project Activity
π©βπ Introduction
A personal research and development (R&D) lab that facilitates the sharing of knowledge.
π Architecture
The diagram illustrates the repository's architecture, which is considered overly complex. It is essential to thoroughly understand the tradeoffs associated with before onboarding any technology into your project.
(The diagram here may take a moment to load. Please wait patiently.)
π¦ Setup
make kubernetes-set-up
make kubernetes-clean
𧱠Tech Stack
Web
-
React - Web framework
-
Redux - State container
-
React Query - Hooks for fetching, caching and updating asynchronous data
-
redux-observable - Side effects
-
RxJS - Asynchronous programming with observable streams
-
graphql-tag - GraphQL query parsing
-
Bulma - CSS framework
-
PurgeCSS - Unused CSS removing
-
Jest - Unit testing, snapshot testing
-
React Testing Library - React component testing
-
Storybook - Visual testing
-
rxjs/testing - Marble testing
-
Cypress - End-to-end testing
-
Lighthouse CI - Performance, accessibility, search engine optimization (SEO), progressive web app (PWA) analysis
-
Sentry - Error tracking
-
Report URI - Security reporting
-
Google Tag Manager - Tag management
-
Google Analytics - Web analytics
-
FullStory - Experience analytics, session replay, heatmaps
-
Namecheap - Domain
-
Cloudflare - DNS, DDoS protection, CDN
-
HTTP/3 Check - HTTP/3 checking
-
hstspreload.org - HSTS checking
-
Mozilla Observatory - Security monitoring
-
UptimeRobot - Uptime monitoring
Mobile - iOS
-
SwiftUI - UI framework
-
XCTest - Unit testing, performance testing
-
Slather - Code coverage reports generating
Mobile - Android
-
AndroidX - Android Jetpack
-
JUnit - Unit testing, instrumented testing
Mobile - React Native
-
Expo - Universal native apps making platform
-
React Native - Mobile application framework
-
UI Kitten - UI library
-
React Native Testing Library - React Native component testing
Desktop - Qt
-
Qt Quick - Cross-platform application development framework
-
QML - Qt modeling language
API Server - Go
-
Gin - Web framework
-
gRPC - Remote procedure call (RPC) framework
-
graphql-go - GraphQL
-
jwt-go - JSON Web Token (JWT)
-
gin-contrib/cors - Cross-Origin Resource Sharing (CORS)
-
opa - Open Policy Agent
-
dgo - Dgraph client
-
minio-go - MinIO client
-
go-redis - Redis client
-
pgx - PostgreSQL driver
-
Resty - HTTP client
-
Squirrel - SQL query builder
-
apm-agent-go - Application performance monitoring (APM) agent
-
OpenTelemetry Go - OpenTelemetry
-
Prometheus Go - Prometheus
-
Testify - Unit testing
-
GoDotEnv - Environment variables loading
-
jsonparser - JSON parser
-
zerolog - Logging
API Server - Python
-
Flask - Web framework
-
Flask-CORS - Cross-Origin Resource Sharing (CORS)
-
confluent-kafka - Kafka client
-
Gunicorn - Python web server gateway interface (WSGI) HTTP server
-
asyncpg - PostgreSQL client
-
Tenacity - General-purpose retrying library
-
pytest - Unit testing
-
pydantic - Data validation
-
HTTPX - HTTP client
-
pypdf - PDF library
-
Poetry - Python package management
-
Poe the Poet - Poetry task runner
API Server - Node.js
-
Express - Web framework
-
GraphQL.js, express-graphql - GraphQL
-
graphql-ws, graphql-subscriptions - GraphQL subscriptions
-
graphql-upload - GraphQL upload
-
graphql-shield - GraphQL permissions
-
graphql-depth-limit - GraphQL depth limit
-
graphql-query-complexity - GraphQL query complexity analysis
-
DataLoader - Batching and caching
-
Knex.js - SQL query builder
-
node-postgres - PostgreSQL client
-
ioredis - Redis client
-
rate-limiter-flexible - Rate limiting
-
expressjs/cors - Cross-Origin Resource Sharing (CORS)
-
csurf - CSRF protection
-
jsonwebtoken, express-jwt - JSON Web Token (JWT)
-
bcrypt - Password hashing
-
axios - HTTP client
-
Helmet - HTTP header
Content-Security-Policy
, Referrer-Policy
, Strict-Transport-Security
, X-Content-Type-Options
, X-DNS-Prefetch-Control
, X-Download-Options
, X-Frame-Options
, X-Permitted-Cross-Domain-Policies
, X-XSS-Protection
-
Report To - HTTP header
Report-To
-
Network Error Logging - HTTP header
NEL
-
express-request-id - HTTP header
X-Request-ID
-
response-time - HTTP header
X-Response-Time
-
connect-timeout - Request timeout
-
Terminus - Health check and graceful shutdown
-
pino - Logging
-
dotenv-flow - Environment variables loading
-
Stryker - Mutation testing
-
SuperTest - HTTP testing
-
autocannon - HTTP benchmarking
-
Clinic.js - Performance profiling
-
Node.js - JavaScript runtime
-
npm - JavaScript package management
Cloud Native
-
Hasura - GraphQL Engine
-
hasura-metric-adapter - Hasura GraphQL Engine metric adapter
-
Linkerd - Service mesh
-
Caddy - Web server, reverse proxy, load balancer
-
Traefik - Web server, reverse proxy, load balancer
-
nginx - Web server, reverse proxy, load balancer
-
Elastic APM - Application performance monitoring
-
OpenTelemetry - Observability framework
-
Jaeger - Distributed tracing system
-
Netdata - Distributed monitoring platform
-
Telegraf - Plugin-driven server agent
-
Thanos - Highly available Prometheus setup with long term storage capabilities
-
Pixie - Observability tool for Kubernetes applications
-
Docker - Container
-
Skaffold - Continuous development for Kubernetes applications
-
Multipass - VM management
-
Locust - Load testing
-
Cloudflare Tunnel - Tunneling
-
Kubernetes - Container-orchestration system
-
Vertical Pod Autoscaler - Kubernetes vertical pod autoscaler
-
K3s - Lightweight Kubernetes
-
containerd - Container runtime
Data
Database, Data Warehouse, Data Lakehouse
-
Trino - Distributed SQL query engine
-
PostgreSQL - Object-relational database
-
Postgres Operator - PostgreSQL high-availability (HA) template
-
Postgres Operator - PostgreSQL cluster provisioning
-
pgAdmin - PostgreSQL management tool
-
MySQL - Relational database
-
Hydra - Column-oriented SQL database
-
ClickHouse - Column-oriented SQL database
-
YugabyteDB - Distributed SQL database
-
TimescaleDB - Time-series SQL database
-
InfluxDB - Time-series database
-
InfluxDB Enterprise - Distributed time-series database
-
Prometheus - Time-series database
-
Loki - Log aggregation system
-
Apache Cassandra - Distributed wide-column NoSQL database
-
Qdrant - Distributed vector database
-
Chroma - Distributed vector database
-
Dgraph - Distributed graph database
-
Elasticsearch - Distributed document-oriented search engine
-
Kibana - Elasticsearch visualization
-
Redis - Distributed in-memory keyβvalue database
-
KeyDB - Multithreaded fork of Redis
-
MinIO - Object storage
-
Apache ZooKeeper - Distributed coordination system
-
Apache Hadoop - Software utility collection
-
Apache Hadoop HDFS (Distributed File System) - Distributed file system
-
Apache Hadoop YARN (Yet Another Resource Negotiator) - Resource management and job scheduling framework
-
Apache Hadoop MapReduce - Data processing framework
-
Apache Hive - Distributed data warehousing and SQL-like query language system built on top of Apache Hadoop
-
Delta Lake - Data lakehouse
-
Snowflake - Data warehouse
-
golang-migrate/migrate - Database migrations
Data Ingestion
-
Airbyte - Data integration
-
Vector - Log collector
-
Fluent Bit - Log collector
Data Orchestration
-
Prefect - Orchestration platform
-
Apache Airflow - Orchestration platform
-
Temporal - Orchestration platform
Data Processing
-
Apache Spark - Data processing framework
-
Spark ML - Spark machine learning
-
pyspark - Spark API library
-
Delight - Spark UI and history server
-
Apache Sedona - Spatial data processing framework
-
Apache Flink - Data processing framework
-
flink-streaming-java - Flink
-
flink-connector-twitter - Flink Twitter connector
-
flink-connector-jdbc - Flink JDBC Connector
-
flink-connector-redis - Flink Redis connector
-
Apache Kafka - Distributed event streaming platform
- Schema registries
-
Confluent Schema Registry - Schema Registry
-
Apicurio Registry - Schema Registry
- Connectors
-
Debezium - Distributed change-data-capture (CDC) platform
-
kafka-connect-elasticsearch - Elasticsearch sink connector
-
confluentinc-kafka-connect-jdbc - JDBC source and sink connector
-
debezium-connector-postgres - PostgreSQL CDC source connector
-
http-connector-for-apache-kafka - HTTP sink connector
-
kafka-connect-avro-converter - Confluent Avro converter
-
apicurio-registry-distro-connect-converter - Apicurio Avro converter
- Management tools
-
Redpanda Console - Kafka management tool
-
AKHQ - Kafka management tool
-
UI for Apache Kafka - Kafka management tool
-
dbt - Data transformation
Data Visualization
-
Grafana - Data visualization
-
Metabase - Data visualization
-
Apache Superset - Data visualization
-
Tableau - Data visualization
Data Analytics
-
NumPy - Scientific computing library
-
pandas - Data analysis library
-
GeoPandas - Geographic data library
-
AWS SDK for pandas - pandas on AWS
-
Modin - pandas workflows scaling
-
JupyterLab - Web-based interactive computing platform
-
nb-clean - Jupyter notebook cleaning
-
Databricks - Unified data analytics platform
-
Palantir - Data integration and analysis platform
Machine Learning (ML)
-
PyTorch - Machine learning framework
-
PyTorch Geometric - PyTorch geometric deep learning extension
-
TorchServe - PyTorch models serving
-
Gradio - Machine learning web application building
-
Streamlit - Data web application building
-
Lightning - Deep Learning framework
-
LangChain - Large language model (LLM) framework
-
NeuralForecast - Neural forecasting
-
GPT4All - Large language models
-
Transformers - Machine learning models
-
OGB - Open graph benchmark
-
Rasa - Machine learning framework for automated text and voice-based conversations
-
CML - Continuous machine learning
-
DVC - Data version control
-
Feast - Feature store
-
Kubeflow - Machine learning platform
-
MLflow - Machine learning experiment tracking
-
Weights & Biases - Machine learning experiment tracking
Computer Vision
-
OpenCV - Computer vision library
-
supervision - Computer vision library
-
Ultralytics YOLOv8 - Object detection model
-
Open3D - 3D data processing
-
PyVista - 3D plotting and mesh analysis
-
Visualization Toolkit (VTK) - Image processing, 3D graphics, volume rendering and visualization
Computing
Parallel Computing
-
CUDA - Parallel computing
-
AWS ParallelCluster - High performance computing (HPC) cluster management
-
AWS Batch - Batch computing
-
Open MPI - High-performance computing (HPC) library
-
Slurm - Workload management
Cloud Computing
-
Amazon EC2 - Cloud computing
-
Ray - Distributed computing framework
-
SkyPilot - Sky computing
Quantum Computing
-
Qiskit - Quantum computing
Cloud
-
Amazon Web Services
-
Amazon Athena - Serverless query service
-
Amazon CloudTrail - Data governance, data compliance, data auditing
-
Amazon EBS - Block storage
-
Amazon EC2 - Cloud computing
-
Amazon ECR - Container registry
-
Amazon EKS - Kubernetes
-
Amazon EMR - Big data platform
-
Amazon EventBridge - Serverless event bus
-
Amazon MSK - Kafka
-
Amazon RDS - Relational database service
-
Amazon Route 53 - Domain Name System (DNS) web service
-
Amazon S3 - Object storage
-
Amazon Redshift - Data warehouse
-
Amazon SageMaker - Machine learning platform
-
Amazon SQS - Queue
-
AWS Batch - Batch computing
-
AWS CloudFormation - Infrastructure as code (IaC)
-
AWS CodeCommit - Version control
-
AWS Glue - Serverless data integration
-
AWS Glue Crawler - Data source discovery
-
AWS Glue Data Catalog - Data catalog
-
AWS Glue DataBrew - Data cleaning
-
AWS IAM - Identity and access management
-
AWS IoT Core - Internet of Things (IoT)
-
AWS Lake Formation - Data lake governance
-
AWS ParallelCluster - High performance computing (HPC) cluster management
-
AWS Secrets Manager - Password management
-
Google Cloud
-
BigQuery - Data warehouse
-
BigQuery ML - BigQuery machine learning
-
Dataprep - Data cleaning
-
Looker Studio - Data visualization
-
Vertex AI - Machine learning platform
Cloud Infrastructure
-
Terraform - Infrastructure as code (IaC)
-
Pulumi - Infrastructure as code (IaC)
-
Karpenter - Kubernetes node autoscaler
Cloud Security
-
Prowler - Cloud security assessments
Cloud Cost
-
Komiser - Cloud cost monitoring
Ops
-
Argo CD - Declarative GitOps CD for Kubernetes
-
Rancher - Kubernetes container management platform
-
Goldilocks - Kubernetes resource requests recommendation
-
Polaris - Kubernetes best practices validation
-
Sloop - Kubernetes history visualization
-
OpenCost - Kubernetes cost monitoring
-
Kubecost - Kubernetes cost monitoring
-
Diun - Container image update notifier
-
Vagrant - Development environments building and distributing
-
Ansible - IT automation system
-
Discord - ChatOps
-
Opsgenie - Incident management platform
-
GitHub Actions - Continuous integration
Authentication, Authorization, Security
-
Apache Ranger - Authorization, auditing
-
Ory Hydra - OAuth 2.0 and OpenID Connect server
-
Open Policy Agent (OPA) - Policy-based control
-
OPAL - Open-policy administration layer
-
CodeQL - Variant analysis
-
Gitleaks - Git secret scanning
-
GitGuardian - Git secret scanning
-
xxHash - Hash algorithm
Simulation
-
AnyLogic - Simulation modeling tool
-
NI LabVIEW - Graphical programming environment
-
NI VeriStand - Real-time testing and simulation
-
niveristand - NI VeriStand API library
-
npTDMS - TDMS files reading and writing
-
PyVISA - Virtual instrument software architecture (VISA) API library
-
MATLAB - Programming and numeric computing platform
-
5G Toolbox - 5G communications systems simulation, analysis, and testing
-
Aerospace Toolbox - Aerospace vehicle motion analysis and visualization
-
Automated Driving Toolbox - ADAS and autonomous driving systems design, simulation, and testing
-
Bioinformatics Toolbox - Genomic and proteomic data analysis and visualization
-
Computer Vision Toolbox - Computer vision, 3D vision, and video processing systems design and testing
-
Database Toolbox - Relational and NoSQL databases interacting
-
Lidar Toolbox - Lidar processing systems design, analysis, and testing
-
Navigation Toolbox - Autonomous navigation algorithms design, simulation, and deployment
-
Satellite Communications Toolbox - Satellite communications systems simulation
-
Signal Processing Toolbox - Signal processing and analysis
-
Simulink - Simulation and model-based designing
-
Simscape - Multidomain physical systems simulation
-
SimScale - Computational fluid dynamics (CFD), finite element analysis (FEA), thermal simulation
-
CoppeliaSim - Robot simulation
Embedded, IoT, Hardware
-
VHDL - Very High Speed Integrated Circuits Program (VHSIC) hardware description language
-
pySerial - Serial communication library
-
cantools - Controller Area Network (CAN) bus tools
-
python-can - Controller Area Network (CAN) bus library
-
Valgrind - Memory debugging and profiling
-
hexedit - File viewing and editing in hexadecimal and ASCII
-
Yocto Project - Linux distribution creating
-
ROS - Robot operating system
-
FreeRTOS - Real-time operating system
-
ASTERIOS - Real-time, safety-critical applications development
-
PX4 - Flight control software
-
RTI Connext - Real-time, distributed systems framework
-
RTI Connext DDS - Data distribution service (DDS)
-
Rclone - Sync program
-
restic - Backup program
-
OpenSCAD - 3D CAD Modeller
-
Arduino Uno - Microcontroller board
-
BeagleBone Black - Microcontroller board
-
Raspberry Pi 4 Model B - Single-board computer (SBC)
-
Jetson Nano - Single-board computer (SBC)
-
Jetson TX2 - Single-board computer (SBC)
-
CubeSat - Miniaturized satellite
-
QGroundControl - Ground control station (GCS) for unmanned aerial vehicles (UAVs)
Ethereum
-
Solidity - Contract-oriented programming language
-
solc-js - JavaScript bindings for the Solidity compiler
Code
-
ansible-lint - Ansible linter
-
ClangFormat - C/C++ code formatter
-
CMakeLint - CMake linter
-
Prettier - Code formatter
-
commitlint - Commit message linter
-
Stylelint - CSS linter
-
hadolint - Dockerfile linter
-
gofmt - Go code formatter
-
golangci-lint - Go linter
-
ESLint - JavaScript linter
-
Ktlint - Kotlin code formatter and linter
-
detekt - Kotlin static type checker
-
Kubeconform - Kubernetes manifest linter
-
markdownlint-cli2 - Markdown linter
-
MISS_HIT - MATLAB code formatter
-
Buf - Protocol Buffers linter
-
Black - Python code formatter
-
autoflake, isort, Ruff - Python linter
-
Mypy - Python static type checker
-
qmllint - QML linter
-
opa - Rego code formatter
-
RuboCop - Ruby code formatter and linter
-
rustfmt - Rust code formatter
-
Clippy - Rust linter
-
Scalafmt - Scala code formatter
-
Scalafix - Scala linter
-
ShellCheck - Shell linter
-
solhint - Solidity linter
-
SQLFluff - SQL code formatter and linter
-
terraform - Terraform code formatter
-
tsc - TypeScript static type checker
-
VHDL Style Guide (VSG) - VHDL code formatter
-
@prettier/plugin-xml - XML formatter
-
yamllint - YAML linter
-
GitHub - Version control
-
SonarCloud, Codacy, Code Climate - Code reviews and analytics
-
Codecov - Code coverage reports
-
Depfu - Dependency monitoring
-
FOSSA - License compliance
Design
-
Fusion 360 - Industrial design
-
Blender - 3D graphic design
-
Figma - UX design
Bots
-
Renovate - Dependency updating
-
CodeReview BOT - Code reviewing
-
Mergify - Automatically merging
-
Stale - Stale issues and pull requests closing
-
ImgBot - Image compression
-
semantic-release - Version management and package publishing
Testing
-
Unit testing
-
Snapshot testing
-
Visual testing
-
Instrumented testing
-
Smoke testing
-
Sanity testing
-
Compatibility testing
-
Integration testing
-
End-to-end testing
-
Contract testing
-
Mutation testing
-
Performance testing
-
Duck testing
-
Bus testing
-
Load testing
-
Durability testing
-
Fault injection testing
-
Parallel testing
-
Acceptance testing
-
Model-in-the-loop (MIL) testing
-
Hardware-in-the-loop (HIL) testing
-
Environmental testing
-
Vibration testing
-
Shock testing
-
Temperature testing
-
Humidity testing
-
Altitude testing
-
Icing testing
-
Rain testing
-
Fungus testing
-
Salt fog testing
-
Lightning testing
-
Structural testing
-
Dyno testing
-
Wind tunnel testing
-
Ground testing
-
Flight testing
Languages
-
C
-
CSS
-
Docker
-
Go
-
GraphQL
-
HCL
-
HTML
-
Java
-
JavaScript
-
Makefile
-
Markdown
-
MATLAB
-
Protocol Buffers
-
Python
-
QML
-
Rego
-
Ruby
-
Rust
-
Shell
-
Solidity
-
SQL
-
LogQL
-
PromQL
-
SedonaSQL
-
Spark SQL
-
Snowflake SQL
-
Swift
-
VHDL
-
XML
-
YAML
Communication Standards and Protocols
-
User Datagram Protocol (UDP)
-
Transmission Control Protocol (TCP)
-
The Internet Protocol (IP)
-
Hypertext Transfer Protocol (HTTP)
-
Hypertext Transfer Protocol Secure (HTTPS)
-
HTTP/1.1
-
HTTP/2
-
HTTP/3
-
WebSocket
-
Web Real-Time Communication (WebRTC)
-
Remote Procedure Call (RPC)
- RPC frameworks
-
Apache Avro
-
Apache Thrift
-
gRPC Remote Procedure Calls (gRPC)
- Binary data serialization formats
-
Avro
-
Thrift
-
Protocol Buffers (Protobuf)
- File transfer protocols
-
File Transfer Protocol (FTP)
-
Secure File Transfer Protocol (SFTP)
-
Server Message Block (SMB)
-
Web Distributed Authoring and Versioning (WebDAV)
- Email protocols
-
Post Office Protocol (POP)
-
Simple Mail Transfer Protocol (SMTP)
-
Internet Message Access Protocol (IMAP)
- Serial protocols
-
Inter-Integrated Circuit (IΒ²C)
-
Serial Peripheral Interface (SPI)
-
Controller Area Network (CAN)
- Time protocols
-
Network Time Protocol (NTP)
-
Precision Time Protocol (PTP)
-
IRIG-B Time Protocol
-
MQTT
Guidelines
-
Avionics Systems
-
DO-178C - Software considerations in airborne systems and equipment certification
-
DO-254 - Design assurance guidance for airborne electronic hardware
-
ARP4754A - Guidelines for development of civil aircraft and systems
-
DO-160G - Environmental conditions and test procedures for airborne equipment
-
DO-331 - Model-based development and verification supplement to DO-178C and DO-278A
-
DO-330 - Software tool qualification considerations
-
Drone Systems
-
Pixhawk standards - Hardware specifications and guidelines for drone systems development
-
Automotive Systems
-
ISO 26262 - Road vehicles β functional safety
-
AUTOSAR - Automotive open system architecture
-
Industrial Robot Systems
-
ISO 10218 - Robots and robotic devices β safety requirements for industrial robots
-
ANSI/RIA R15.06-2012 - Industrial robots and robot systems β safety requirements
π Highlights
Simulation
The following presents a model of a radar-based air defense system. Bombers are dispatched to destroy ground facilities, while the buildings are safeguarded by the air defense system, comprising two radars equipped with guided surface-to-air missiles.
NI LabVIEW - Graphical Programming Environment
NI VeriStand - Real-Time Testing and Simulation
CoppeliaSim - Robot Simulation
SimScale - Computational Fluid Dynamics (CFD), Finite Element Analysis (FEA), Thermal Simulation
MATLAB
Simscape - Multidomain Physical Systems Simulation
Embedded, IoT, Hardware
ASTERIOS - Real-Time, Safety-Critical Applications Development
RTI Connext - Real-Time, Distributed Systems Framework
ROS - Robot Operating System
PX4 - Flight Control Software
OpenSCAD - 3D CAD Modeller
The toroidal propeller allows a small multirotor aircraft to operate more quietly than the ones that use traditional propellers.
VHDL - Very High Speed Integrated Circuits Program (VHSIC) Hardware Description Language
The VHDL waveforms are displayed in GTKWave.
Poky - Reference Linux Distribution of the Yocto Project
Database, Data Warehouse, Data Lakehouse
Dgraph - Distributed Graph Database
Redis with RedisGraph Module
Data Cleaning
Dataprep - Data Cleaning
Data Orchestration
Data Processing
Flink - Data Processing
Machine Learning
Vertex AI - AutoML
Contextual AI assistant
Chatbot on Telegram powered by Rasa.
Weights & Biases - Machine Learning Experiment Tracking
Distributed hyperparameter optimization result by Weights & Biases.
Computer Vision
PyVista
Open3D
Computing
Ray - Distributed Computing
Cloud Native
Pixie - Kubernetes Application Observing
Linkerd - Service Mesh
Hasura - GraphQL Engine
Traefik - Reverse Proxy and Load Balancer
Jaeger - Distributed Tracing
Grafana - Data Visualization
Data Source: Prometheus - Time-Series Database
Data Source: Loki - Log Aggregation System
Kibana - Elasticsearch Visualization
Tableau - Data Visualization
Testing
Locust - Load Testing
Ops
Argo CD - GitOps
Discord - ChatOps
Rancher - Kubernetes Container Management
Kubecost - Kubernetes Cost Monitoring
Polaris - Kubernetes Best Practices Validation
Goldilocks - Kubernetes Resource Requests Recommendation
Web
HTTP/3
The website supports HTTP/3.
AVIF
Images on the website are using AVIF format.
The WebP is almost half the size of JPEG, and AVIF is under half the size of WebP.
Security
Below is the website security report generated by Mozilla Observatory.
Profiling
Profiling result by Clinic.js and autocannon.