Yildiz Graph Database

Thin graph database layer on top of Google Bigtable.

Intro

YildizDB acts as a highly scaleable HTTP layer in front of Google Bigtable. It helps you store billions of relations between nodes in edges and resolve them in milliseconds. A small access layer helps you manage multiple namespaces with ease. YildizDB scales to hundrets of Terabytes. YildizDB excells in $N:N$ non nested relationships.

Features

• simple namespacing (table separation) with prefixes
• simple access management through tokens
• multi-tenancy through table prefixes (with access management)
• high read and write performance
• fast read access under heavy write load
• able to handle billions of edges and nodes
• scales beyond Terabytes
• lightweight deployments (small Node.js footprint)
• ttl feature for all resources
• HTTP Open API
• Kubernetes HELM charts

Build for high throughput $N:N$

• Highly async API based on fastify
• Thin layer on top of Google's GRPC Bigtable API
• Hashing and translating all string identifiers into integer representations via murmurhash3 automatically

Fast random access to a node's edge data $1:N$

• Multiple complex caching layers
• Custom Bigtable cache table speed up by Google Memorystore (Redis)
• All Memorystore hits also speed up by In-Memory store
• Fetch job that keeps active nodes refreshed in cache

Available clients

• Node.js Client
• Any Http Client can be used to access the HTTP-Interface

Usage

• You will need a Google Cloud Project with a running Bigtable cluster
• Additionally YildizDB requires a Memorystore (or Redis) instance
• Configure ./config/bigtable.json accordingly
• Install and start: npm install -g yildiz and yildizdb -p 3058 -l ./config/bigtable.json
• A word on configuration can be found here
• Use right alongside your code
• Spawn server via http interface

Deployment

• YildizDB is designed to be deployed as simple platform service
• it requires Node.js > 9.x.x and Redis > 3.x
• It simply requires its npm module as well as a config file that describes the connections to Bigtable and Memorystore (Redis)
• It scales best with an HTTP load balancer in front of it e.g. NGINX
• We run and scale it very successfully in Google's Kubernetes Engine
• We also offer HELM charts
• Read some more about the config file here

Metrics & Monitoring

• Yildiz exposes Prometheus Metrics @ /admin/metrics
• Read more about it here

Developing YildizDB

Developing YildizDB with Google Bigtable backend

• Configure ./config/bigtable.json accordingly
• Run tests via yarn test
• Start via yarn http

Documentation

• yildiz means :star: in turkish
• Best practice
• Open API/Swagger JSON
• Access management
• Stats and metrics
• YildizDB Use-cases
• Why develop another Graph Database?
• How does it work?
• Storing node relations even faster
• The popular right node concept
• "Depth" or "Create" edge creation
• Configuration & Deployments

Disclaimer

• This project is not affiliated with Google
• License is MIT see