Gorse Versions Save

Gorse open source recommender system engine

v0.4.6

1 year ago

Features

  • Support dynamic goroutine allocation for multiple tasks (#513).
  • Support database prefixes postgresql:// and rediss:// (#499).
  • Support disable/enable model size searching (#502).
  • Support prefixes for database table names (#503).
  • Build windows arm64 release (#500).
  • Support HTTPS for ClickHouse connection: chhttps:// (#508).
  • Prune solely user labels and item labels (#512).

Fix

  • Fix unexpected database schema migration (#501).
  • Fix duplicate measurements (#505).
  • Stop the HTTP server and RPC server gracefully (#506).

v0.4.5

1 year ago

Features

  • Implement a playground based on the all-in-one server (#479).
  • Support Oracle as one of the cache stores (#482).
  • Support Oracle as one of the data stores (#484).

Fix

  • Remove tasks from disconnected worker nodes (#480).
  • Show progress for index building (#490).
  • Fix the isolation level (READ UNCOMMITTED) in MySQL (#481).
  • Fix the positive feedback display in the dashboard (#486, by @bradrisse).

v0.4.4

1 year ago

Features

  • Support SQLite as one of the data stores (all-in-one server only, #469).
  • Support SQLite as one of the cache stores (all-in-one server only, #467).
  • Implement the all-in-one server (#470).

Fix

  • Fix delete users and items (#471).
  • Fix import feedback CSV with quoted field (#472).

Performance

  • Remove redundant cache deletion (#475).

v0.4.3

1 year ago

Features

  • Track the memory usage of the master node and worker nodes (#461).
  • Calculate positive feedback rates during load dataset (#462).

Fix

  • Report metrics (precision, recall, etc.) once saved models are loaded (#457).
  • Redact username and password in DSN (#459).
  • Fix migration error with PostgresSQL and ClickHouse (#464).
  • Fix insert duplicate users/items/feedback (#465).

v0.4.2

2 years ago

Features

  • Display binary versions in the dashboard (#448).
  • Implement the request ID for RESTful APIs (#449).
  • Move measurements from the data store to the cache (#450).
  • Support database auto migrate by gorm (#453).
  • Reimplement auto similarity as hybrid similarity based on labels and users/items (#455).

Performance

  • Set MySQL and Postgres isolation level to READ UNCOMMITTED (#452).

v0.4.1

2 years ago

Features

  • Support session-based recommendation (#430).
  • Invalidate cache after the configuration change (#436).
  • Support cache garbage collection (#437).
  • Export metrics for new Gorse Grafana dashboard (#445).

Fix

  • Fix hidden items and deleted items visibility (#442).

Performance

  • Cache popular items in the server node (#438).
  • Cache hidden items in the server node (#439).

v0.4.0

2 years ago

Features

  • Support MongoDB as cache_store (#406).
  • Support MySQL as cache_store (#415).
  • Support PostgresSQL as cache_store (#414).

Fix

  • Fix typos in Swagger spec (#405) by @europaer.
  • Unify timezone in ClickHouse (#417).
  • Fix inserting duplicate cache rows in Postgres (#417).
  • Fix item timestamp modification (#417).

Performance

  • Optimize inserting users/items/feedbacks (#411).
Action Cache Data Before After
Insert 1,000 users Redis MySQL 75ms 38ms
Insert 1,000 items Redis MySQL 324ms 32ms
Insert 1,000 feedbacks Redis MySQL 248ms 42ms

BREAK CHANGES

  • New config file specification and default values are introduced in v0.4.0 (#418).
  • The write-back-delay parameter of GET /api/recommend/{user-id} requires time unit now. For example:
curl -X GET "http://172.18.0.3:8087/api/recommend/zhenghaoz?write-back-type=read&write-back-delay=10m&n=10" \
    -H "accept: application/json" \
    -H "X-API-Key: 19260817"

v0.3.4

2 years ago

Features

  • Support set check recommend period (#384).
  • Support replacement in the recommendation (#385).
  • Support DNS SRV record lookup for MongoDB (#386, authorized by @pranav377).
  • Support set clock error in the cluster (#397).

Fix

  • Import and export items with hidden flags and categories (#383).
  • Hide hidden items in the latest items, popular items, and item neighbors (#393).
  • Fix inaccurate representation of timestamps in float32 (#394).
  • Fix potential Redis memory leak (#395).

Upgrade Guide

  • Configuration: Replacement has been supported in the configuration, as well as check_recommend_period and epsilon_time.
[server]
# ...
epsilon_time = 5                # clock error in the cluster in seconds

[recommend]
# ...
# The time period to check recommendation for users (minutes). The default values is 1.
check_recommend_period = 1

# Replace historical items back to recommendations.
enable_replacement = false

# Decay the weights of replaced items from positive feedbacks.
positive_replacement_decay = 0.8

# Decay the weights of replaced items from read feedbacks.
read_replacement_decay = 0.6
  • Redis: Remove incompatible stale cache.
redis-cli KEYS "offline_recommend*" | xargs redis-cli DEL

v0.3.3

2 years ago

Feature

  • Support login authorization in Gorse dashboard (#376).
  • Support AVX2 for vector computation (#372).
  • Support Arm Neon for vector computation (#374).
  • Support feedback type-level "click-through rate" (#379).

Fix

  • Fix recommendations to untrained users (#380).
  • Create folders for log files and handle ignored errors (#378).
  • Return zero recall on empty ground truth (#381).

Upgrade Guide

  • Configuration: Set username and password for Gorse dashboard. If both dashboard_user_name and dashboard_password are empty, login authorization will be skipped.
dashboard_user_name = "admin"   # dashboard user name
dashboard_password = "password" # dashboard password
  • Redis: Remove incompatible stale cache.
redis-cli KEYS "item_neighbors*" | xargs redis-cli DEL

v0.3.2

2 years ago

Feature

  • Support modify configuration via environment variables (#359). There are 8 environment variables available:
Environment Variable Configuration
GORSE_CACHE_STORE cache_store database for caching.
GORSE_DATA_STORE data_store database for persist data.
GORSE_MASTER_PORT port master port
GORSE_MASTER_HOST host master host
GORSE_MASTER_HTTP_PORT http_port HTTP API port
GORSE_MASTER_HTTP_HOST http_host HTTP API host
GORSE_MASTER_JOBS n_jobs number of working jobs
GORSE_SERVER_API_KEY api_key secret key for RESTful APIs
  • (Experimental) Support IVF-based neighborhood searching (#363).
  • (Experimental) Support HNSW based recommended items searching (#368).

Fix

  • Split large Redis writes to batches (#352).
  • Allow empty JSON string in relational databases (#355).
  • Fix offset overflow in recommend API (#365).
  • Optimize dashboard overview page layout (#369).

Upgrade Guide

  • Configuration: IVF-based neighborhood searching and HNSW based recommended items searching are experimental features.
    • Use enable_xxx_index to enable experimental features.
    • xxx_index_recall is the expected recall of approximate searching.
    • xxx_index_fit_epoch is the number of epochs to adapt indices.

These approximate vector searching indices trade tolerable accuracy (recall) with larger throughput. The index builder tries to reach xxx_index_recall in xxx_index_fit_epoch. The building process will stop when xxx_index_recall or xxx_index_fit_epoch reached. Index-based searching costs more memory than brute force searching and its building process costs additional time, but they are negligible compared to the benefits.

# Enable approximate item neighbor searching using vector index.
enable_item_neighbor_index = false

# Minimal recall for approximate item neighbor searching.
item_neighbor_index_recall = 0.8

# Maximal number of fit epochs for approximate item neighbor searching vector index.
item_neighbor_index_fit_epoch = 3

# Enable approximate user neighbor searching using vector index.
enable_user_neighbor_index = false

# Minimal recall for approximate user neighbor searching.
user_neighbor_index_recall = 0.8

# Maximal number of fit epochs for approximate user neighbor searching vector index.
user_neighbor_index_fit_epoch = 3

# Enable approximate collaborative filtering recommend using vector index.
enable_collaborative_index = false

# Minimal recall for approximate collaborative filtering recommend.
collaborative_index_recall = 0.9

# Maximal number of fit epochs for approximate collaborative filtering recommend vector index.
collaborative_index_fit_epoch = 3
  • Redis: Remove incompatible stale cache.
redis-cli KEYS "item_neighbors*" | xargs redis-cli DEL
redis-cli KEYS "user_neighbors*" | xargs redis-cli DEL