StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries. InfoWorld’s 2023 BOSSIE Award for best open source software.
Release date: December 30, 2023
Fixed the following issue:
Release date: December 21, 2023
object_dependencies
to the database sys
. It contains the lineage information of asynchronous materialized views. #35060
max_tablet_rowset_num
for setting the maximum allowed number of rowsets. This metric helps detect possible compaction issues and thus reduces the occurrences of the error "too many versions". #36539
enable_stream_load_verbose_log
is added. The default value is false
. With this parameter set to true
, StarRocks can record the HTTP requests and responses for Stream Load jobs, making troubleshooting easier. #36113
GROUP_CONCAT_LEGACY
is added to the session variable sql_mode to provide compatibility with the implementation logic of the group_concat function in versions earlier than v2.5. #36150
aws.s3.access_key
and aws.s3.access_secret
for AWS S3 in Broker Load jobs are hidden in audit logs. #36571
be_tablets
view in the information_schema
database provides a new field INDEX_DISK
, which records the disk usage (measured in bytes) of persistent indexes. #35615
OtherMsg
, which shows information about the last failed task. #35806
Fixed the following issues:
Release date: December 1, 2023
Asynchronous materialized view
columns_from_path
.Added the following functions:
StarRocks supports access control through Apache Ranger, providing a higher level of data security and allowing the reuse of existing services of external data sources. After integrating with Apache Ranger, StarRocks enables the following access control methods:
For more information, see Manage permissions with Apache Ranger.
Asynchronous materialized view
query_rewrite_consistency
for asynchronous materialized view creation. This property defines the query rewrite rules based on the consistency check.force_external_table_query_rewrite
for external catalog-based asynchronous materialized view creation. This property defines whether to allow force query rewrite for asynchronous materialized views created upon external catalogs.fast_schema_evolution
. After this feature is enabled, the execution efficiency of adding or dropping columns is significantly improved. This mode is disabled by default (Default value is false
). You cannot modify this property for existing tables using ALTER TABLE.SET_VAR
. #35283
large_decimal_underlying_type = "panic"|"double"|"decimal"
to set the rules to deal with DECIMAL type overflow. panic
indicates returning an error immediately, double
indicates converting the data to DOUBLE type, and decimal
indicates converting the data to DECIMAL(38,s).bucket_size
when creating tables. This allows the system to dynamically adjust the number of tablets based on cluster information and the size of loaded data. Please note that once this optimization is enabled, if you need to roll back your cluster to v3.1 or earlier, you must delete tables with this optimization enabled and manually execute a metadata checkpoint (by executing ALTER SYSTEM CREATE IMAGE
). Otherwise, the rollback will fail.enable_pipeline_engine=true
in the FE configuration file fe.conf). Failure to do so will result in errors for non-Pipeline queries.To be updated.
catalog_metadata_cache_size
enable_backup_materialized_view
enable_colocate_mv_index
enable_fast_schema_evolution
json_file_size_limit
lake_enable_ingest_slowdown
lake_ingest_slowdown_threshold
lake_ingest_slowdown_ratio
lake_compaction_score_upper_bound
mv_auto_analyze_async
primary_key_disk_schedule_time
statistic_auto_collect_small_table_rows
stream_load_task_keep_max_num
stream_load_task_keep_max_second
enable_pipeline_load
.enable_sync_publish
is changed from false
to true
.enable_persistent_index_by_default
is changed from false
to true
.Data Cache-related configuration changes.
datacache_enable
to replace block_cache_enable
.datacache_mem_size
to replace block_cache_mem_size
.datacache_disk_size
to replace block_cache_disk_size
.datacache_disk_path
to replace block_cache_disk_path
.datacache_meta_path
to replace block_cache_meta_path
.datacache_block_size
to replace block_cache_block_size
.datacache_checksum_enable
to replace block_cache_checksum_enable
.datacache_direct_io_enable
to replace block_cache_direct_io_enable
.datacache_max_concurrent_inserts
to replace block_cache_max_concurrent_inserts
.datacache_max_flying_memory_mb
.datacache_engine
to replace block_cache_engine
.block_cache_max_parcel_memory_mb
.block_cache_report_stats
.block_cache_lru_insertion_point
.After renaming Block Cache to Data Cache, StarRocks has introduced a new set of BE parameters prefixed with datacache
to replace the original parameters prefixed with block_cache
. After upgrade to v3.2, the original parameters will still be effective. Once enabled, the new parameters will override the original ones. The mixed usage of new and original parameters is not supported, as it may result in some configurations not taking effect. In the future, StarRocks plans to deprecate the original parameters with the block_cache
prefix, so we recommend you use the new parameters with the datacache
prefix.
Added the following BE configuration items:
spill_max_dir_bytes_ratio
streaming_agg_limited_memory_size
streaming_agg_chunk_buffer_size
Removed the following BE configuration items:
tc_use_memory_min
tc_free_memory_rate
tc_gc_period
tc_max_total_thread_cache_byte
Default value modifications:
disable_column_pool
is changed from false
to true
.txn_commit_rpc_timeout_ms
is changed from 20000
to 60000
.thrift_port
is changed from 9060
to 0
.enable_load_colocate_mv
is changed from false
to true
.enable_pindex_minor_compaction
is changed from false
to true
.enable_per_bucket_optmize
enable_write_hive_external_table
hive_temp_staging_dir
spill_revocable_max_bytes
thrift_plan_protocol
enable_pipeline_query_statistic
enable_deliver_batch_fragments
enable_scan_block_cache
is renamed as enable_scan_datacache
.enable_populate_block_cache
is renamed as enable_populate_datacache
.Added reserved keywords OPTIMIZE
and PREPARE
.
Fixed the following issues:
information_schema.columns
. #33431
msg:Fail to parse columnsFromPath, expected: [rec_dt]
. #32720
DATA_TYPE
and COLUMN_TYPE
for BINARY or VARBINARY data types are displayed as unknown
in the information_schema.columns
view. #32678
Release date: December 19, 2023
max_tablet_rowset_num
for setting the maximum allowed number of rowsets. This metric helps detect possible compaction issues and thus reduces the occurrences of the error "too many versions". #36539
OtherMsg
, which shows information about the last failed task. #35806
query_timeout
. This prevents query failures caused by RPC request expiration. #36778
enable_stream_load_verbose_log
is added. The default value is false
. With this parameter set to true
, StarRocks can record the HTTP requests and responses for Stream Load jobs, making troubleshooting easier. #36113
update_compaction_per_tablet_min_interval_seconds
becomes mutable. #36819
Fixed the following issues:
enable_collect_query_detail_info
is set to true
. #35945
./agentctl.sh stop be
is used to stop a BE. #35108
Release date: December 18, 2023
Fixed the following issues:
Release date: December 1, 2023
Fixed the following issues:
Release date: November 29, 2023
Release date: November 28, 2023
Fixed the following issues:
Release date: November 15, 2023
columns_from_path
.Added the following functions:
StarRocks supports access control through Apache Ranger, providing a higher level of data security and allowing the reuse of existing Ranger Service of external data sources. After integrating with Apache Ranger, StarRocks enables the following access control methods:
For more information, see Manage permissions with Apache Ranger.
Asynchronous materialized view
query_rewrite_consistency
for asynchronous materialized view creation. This property defines the query rewrite rules based on the consistency check.force_external_table_query_rewrite
for external catalog-based asynchronous materialized view creation. This property defines whether to allow force query rewrite for asynchronous materialized views created upon external catalogs.
For detailed information, see CREATE MATERIALIZED VIEW.Fixed the following issues:
information_schema.columns
. #33431
DATA_TYPE
and COLUMN_TYPE
for BINARY or VARBINARY data types are displayed as unknown
in the information_schema.columns
view. #32678
Release date: November 14, 2023
Fixed the following issues: