Gorse Versions Save

Feature

• Update latest items in real-time on the server node (#342).
• Update categorized popular items in real-time on the server node (#343).

Performance

• Optimize copier by skipping equal maps and reusing slice space (#337).
• Add data skipping indices for ClickHouse (#349).
• Limit the number feedback used in fallback item-based similarity recommendation (#350).

Fix

• Fix concurrent map read in configuration (#338 authorized by @winwill2012, #347).
• Fix worker failure when there is no feedback (#340).
• Fix inefficient implementation of hidden items (#341).

Upgrade Guide

• Configuration: A new option has been added to limit the number feedback used in fallback item-based similarity recommendation. Larger value means more accurate recommendation but spend more time.

# The number of feedback used in fallback item-based similar recommendation. The default values is 10.
num_feedback_fallback_item_based = 20

• ClickHouse: Add indices manually if using ClickHouse as data storage.

ALTER TABLE feedback ADD INDEX user_index user_id TYPE bloom_filter(0.01) GRANULARITY 1;
ALTER TABLE feedback ADD INDEX item_index item_id TYPE bloom_filter(0.01) GRANULARITY 1;

• Redis: Remove incompatible stale cache.

redis-cli KEYS "popular_items*" | xargs redis-cli DEL
redis-cli KEYS "latest_items*" | xargs redis-cli DEL

Features

• Support hidden items. Hidden items are used in training but not in recommendation (#178).
• Support item categories. Recommendations are generated for each category (#307).

Read 2.2 Item Management in the docment for detailed usage.

Performance

• Optimize deep copy by resuing memory (#315).

Upgrade Guide

Since new attributes IsHidden and Categories are added in v0.3.0. Using the follwing SQL to update the schema if using MySQL.

ALTER TABLE items ADD is_hidden BOOL NOT NULL DEFAULT FALSE;
ALTER TABLE items ADD categories json NOT NULL;

Feature

• Support modify users and items by PATCH method (8a6de0e2abb7957f2ece691b467fe2bb3e6aa737).
• Support enable or disable click-through prediction model in configuration (2e4260cfc73d5e37bfb01c3577e699a9d798c4bd).
• Support explore latest items and popular items in personalized recommendation (2e4260cfc73d5e37bfb01c3577e699a9d798c4bd).

Fix

• Fix incompatibility of MariaDB (#294).
• Fix "Invalid Date" in dashboard (8c33b7d47c84052ec94aad31c0cbb1df47a5c85a).
• Fix auto_insert_item = false not working (#301).
• Skip empty feedback list when inserting feedback (#300).

Upgrade Guide

• In configuration file：
  • enable_click_through_prediction enable/disable click-through prediction.
  • explore_recommend defines the proportion of latest/popular items in offline recommendation.
• In RESTful API, PATCH methods are implemented for users and items.
• If there are feedback come from nonextsted items casued by #299, remember to delete them in database. Eg., delete them in MySQL:

delete from feedback where item_id not in (select item_id from items)

Features

• Support offset and limit for recommendation result fetching (#287).
• Support insert feedback with future timestamps (#283).
• Support write-back-delay for recommendation result fetching (#283).
• Support prometheus metrics for master, worker, server and database (#289).
• Support write logs to files (#289).
• Support multiple read feedback types (#288).
• Support custom location for local cache (#292).

Performance

• Replace encoding/gob with custom encoder/decoder (#284).
• Replace message based model distribution with stream based protocol (#284).
• Add index for item_id in feedback table (#286).

Fix

• Fix unclosed result sets in database interfaces (#275).

Upgrade Guide

• In configuration file, read_feedback_type has been replaced by read_feedback_types, which is an array of strings.
• In recommendation API:
  • write-back has been replaced by write-back-type.
  • Use write-back-delay to set delayed timestamps of write back feedbacks.
  • Use offset to load paged recommendation results.
• In command line:
  • Use --log-path to specify the path of logging file.
  • Use --cache-path to specify the path of local cache file.

Performance

• Cache items in workers during offline recommendation (8bbe7a65089e2bf18efcb2163925fc1a27a53b00).
• Skip cold items and users during fitting CCD recommender (dc3c66f18f3e47a1c735c75c5b574b06be45dabd).
• Speed up neighbor searching by bitset and timestamps (7698ce7fe4aaa574a05af5d7570cf1ad24e3b2f0, f6d78bc17bcd8344e841a60de1dc5ff9d19df2e8, 06f604d426955380b94f57a79d303dac3314b21d).
• Optimize data loading by channel (5236e8ce70ceb0b28fcbfe8d58ff0e5e208fdd12).
• Optimize memory usage (2f34bcd3b11c28be1e3852b17997f809616a1b3c, eaecd64084b5d8b4b6025e25bec323e582e11815).

Fix

• Start HTTP server before loading datasets (#256).
• Send large model over gRPC (#246).
• Handle UTF-8 characters when reading lines from a csv file (#251, authorized by @amaaazing).
• Replace unsupported new SQL syntax with compatible SQL syntax (#249 authorized by @hetao29).
• Disable authorization for dashboard (a863e903b0a24f513589ac75b704b96644cc580c).
• Fix responses in swagger apidocs (#262, authorized by @ccfish86).

Features

• Support user import and export.
• Save statistics to data store instead of cache store.
• Use IDF-based similarity for users and items.
• Remove confusing click feedback type (#191).
• Support user-based recommendation (#242).
• Support accuracy/precision/recall/AUC metrics for CTR prediction (#204).
• Support multi-source recommendation.

Fix

• Handle multiple occurrence users/items when loading datasets from ClickHouse (5bf340a1d5d0022cba0ec1b653a939ddd486d82b).

Upgrade Guide

In configuration file:

• click_feedback_types has been removed.
• neighbor_type has been replaced by item_neighbor_type and user_neighbor_type.
• enable_latest_recommend, enable_popular_recommend, enable_user_based_recommend, enable_item_based_recommend and enable_collaborative_recommend are added.
• fallback_recommend becomes a slice of strings.

Features

• Support PostgresSQL as data storage (#201).
• Support ClickHouse as data storage (#184).
• Support user and password in Redis DSN (#216, authorized by @lbw114007).
• Implement task monitor to track the progress of all tasks.
• Implement new task scheduler.
• Implement label based similarity (#131).

Fix

• Not distribute invalid model (#214).
• Division by zero (#221, authorized by @lbw114007).

Upgrade Guide

• In the configuration file, search_jobs and fit_jobs are replaced by n_jobs, neighbor_type is added to set the type of neighbors.

Features

• Support linux arm64 (#208).

Fix

• Remove historical items when injecting latest items during offline recommendation (0a9dfa637f551d3b48c8fec92ca6e36ec0da4e04).
• Fix wrong corresponding target values of click/read feedback (13bdf1e0072011fc2b4b4fbf47b0bf8cb9fc469f).

Features

• Rank items from collaborative filternig by click-through-rate prediction to improve recommendation accuracy (#17).
• Address cold-start problem by inserting latest items to the result from collaborative filternig (#151).
• Disable click-through-rate prediction when labels are unavaliable.
• Enable RESTful server logging.

Fix

• Error 1069: Too many keys specified; max 64 keys allowed (#176).
• Panic on unknown users and items (#185).
• lastNumUsers, lastNumItems, lastNumFeedback always zero (#197).
• Documents has been updated (#180).

Features

• Dashboard: click through rate visualization
• Dashboard: active users visualization

Fix

• Fix number of returned items not working (#153)