StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries. InfoWorld’s 2023 BOSSIE Award for best open source software.
Release date: April 28, 2024
information_schema
using DROP TABLE. #43556
Fixed the following issues:
str_to_map
may cause BEs to crash. #43930
show proc '/routine_loads'
is stuck due to deadlock. #44249
pending_task_run_count
displayed on the page of leaderFE_IP:8030
is incorrect. The displayed number is the sum of Pending and Running tasks, not Pending tasks. In addition, the information of the metric refresh_pending
cannot be displayed using followerFE_IP:8030
. #43052
Invalid plan: PhysicalTopNOperator
error. #44185
Release date: April 18, 2024
Fixed the following issue:
Release date: April 12, 2024
TIP This version has been taken offline due to privilege issues in querying external tables in external catalogs such as Hive and Iceberg.
Problem: When a user queries data from an external table in an external catalog, access to this table is denied even when the user has the SELECT privilege on this table. SHOW GRANTS also shows that the user has this privilege.
Impact scope: This problem only affects queries on external tables in external catalogs. Other queries are not affected.
Temporary workaround: The query succeeds after the SELECT privilege on this table is granted to the user again. But SHOW GRANTS will return duplicate privilege entries. After an upgrade to v3.2.6, users can run REVOKE to remove one of the privilege entries.
Fixed the following issues:
Release date: March 29, 2024
TIP This version has been taken offline due to privilege issues in querying external tables in external catalogs such as Hive and Iceberg.
Problem: When a user queries data from an external table in an external catalog, access to this table is denied even when the user has the SELECT privilege on this table. SHOW GRANTS also shows that the user has this privilege.
Impact scope: This problem only affects queries on external tables in external catalogs. Other queries are not affected.
Temporary workaround: The query succeeds after the SELECT privilege on this table is granted to the user again. But SHOW GRANTS will return duplicate privilege entries. After an upgrade to v3.2.6, users can run REVOKE to remove one of the privilege entries.
IS NULL
operator, they are considered NULL values following SQL language. For example, true
is returned for SELECT parse_json('{"a": null}') -> 'a' IS NULL
(before this behavior change, false
is returned). #42815
Fixed the following issues:
orc_use_column_names
to true
, which specifies to read ORC files from Hive based on mapping by column name. #42905
Release date: March 22, 2024
replace_if_not_null
supports BITMAP columns in an Aggregate table. Users can specify replace_if_not_null
as the aggregate function for BITMAP columns in an Aggregate table. #42104
update_compaction_size_threshold
is changed from 256 MB to 64 MB to accelerate compaction. #42776
Fixed the following issues:
Release date: March 12, 2024
milliseconds_diff
. #38171
catalog
, which specifies the catalog to which the session belongs. #41329
information_schema.partitions_meta
, which records detailed metadata of partitions. #39265
sys.fe_memory_usage
, which records the memory usage for StarRocks. #40464
cbo_decimal_cast_string_strict
is used to control how CBO converts data from the DECIMAL type to the STRING type. The default value true
indicates that the logic built in v2.5.x and later versions prevails and the system implements strict conversion (namely, the system truncates the generated string and fills 0s based on the scale length). The DECIMAL type is not strictly filled in earlier versions, causing different results when comparing the DECIMAL type and the STRING type. #40619
enable_iceberg_metadata_cache
has been changed to false
. From v3.2.1 to v3.2.3, this parameter is set to true
by default, regardless of what metastore service is used. In v3.2.4 and later, if the Iceberg cluster uses AWS Glue as metastore, this parameter still defaults to true
. However, if the Iceberg cluster uses other metastore service such as Hive metastore, this parameter defaults to false
. #41826
root
user to the user who creates the materialized views. This change does not affect existing materialized views. #40670
cbo_eq_base_type
to adjust the rule used for the comparison. For example, users can set cbo_eq_base_type
to decimal
, and StarRocks then compares the columns as numeric values. #40619
s3_compatible_fs_list
to specify which S3-compatible object storage can be accessed via AWS SDK, and supports using the parameter fallback_to_hadoop_fs_list
to specify non-S3-compatible object storages that require access via HDFS Schema (this method requires the use of vendor-provided JAR packages). #41123
agg_type
of BITMAP-type columns in an Aggregate table can be set to replace_if_not_null
in order to support updates only to a few columns of the table. #42034
Fixed the following issues:
Release date: March 8, 2024
regexp_extract_all
. #42178
information_schema.partitions_meta
, which records detailed metadata of partitions. #41101
sys.fe_memory_usage
, which records the memory usage for StarRocks. #41083
root
user to the user who creates the materialized views. This change does not affect existing materialized views. #40698
cbo_eq_base_type
to adjust the default rule used for the comparison. For example, users can set cbo_eq_base_type
to decimal
, and StarRocks then compares the columns as numeric values. #41712
s3_compatible_fs_list
to specify which S3-compatible object storage can be accessed via AWS SDK, and supports using the parameter fallback_to_hadoop_fs_list
to specify non-S3-compatible object storage that require access via HDFS Schema (this method necessitates the use of vendor-provided JAR packages). #41612
current_catalog
, current_schema
, to_char
, from_hex
, to_date
, to_timestamp
, and index
. #41505 #41270 #40838
cbo_materialized_view_rewrite_related_mvs_limit
is added to control the maximum number of candidate materialized views allowed during query planning. The default value of this session variable is 64
. This session variable helps mitigate the excessive resource consumption caused by a large number of candidate materialized views for a query during the query planning. #39829
agg_type
of BITMAP-type columns in an Aggregate table can be set to replace_if_not_null
to support updates only to a few columns of the table. #42102
cbo_eq_base_type
is optimized to support specifying the implicit conversion rule applied to the comparison of data that contains both string and numeric data types. By default, such data is compared as strings. #40619
path
parameter in the SQL statement for creating a file external table supports wildcards (*
). However, like the DATA INFILE
parameter in the SQL statement for creating a Broker Load job, the path
parameter supports using wildcards (*
) to match at most one level of directory or file. #40844
Fixed the following issues:
Release date: February 8, 2024
yyyy-MM-ddTHH:mm
and yyyy-MM-dd HH:mm
to support TIMESTAMP partition fields in Apache Iceberg tables. #39986
Fixed the following issues:
storage_page_cache_limit
in certain circumstances. #37740
array<string>
data from ORC files into StarRocks (array<json>
) may cause BEs to crash. #39233
Release date: February 10, 2024
enable_strict_order_by
. When this variable is set to the default value TRUE
, an error is reported for such a query pattern: Duplicate alias is used in different expressions of the query and this alias is also a sorting field in ORDER BY, for example, select distinct t1.* from tbl1 t1 order by t1.k1;
. The logic is the same as that in v2.3 and earlier. When this variable is set to FALSE
, a loose deduplication mechanism is used, which processes such queries as valid SQL queries. #37910
enable_materialized_view_for_insert
, which controls whether materialized views rewrite the queries in INSERT INTO SELECT statements. The default value is false
. #37505
query_mem_limit
instead of exec_mem_limit
. Setting the value of query_mem_limit
to 0
indicates no limit. #34120
http_worker_threads_num
, which specifies the number of threads for HTTP server to deal with HTTP requests. The default value is 0
. If the value for this parameter is set to a negative value or 0
, the actual thread number is twice the number of CPU cores. #37530
lake_pk_compaction_max_input_rowsets
, which controls the maximum number of input rowsets allowed in a Primary Key table compaction task in a shared-data StarRocks cluster. This helps optimize resource consumption for compaction tasks. #39611
connector_sink_compression_codec
, which specifies the compression algorithm used for writing data into Hive tables or Iceberg tables, or exporting data with Files(). Valid algorithms include GZIP, BROTLI, ZSTD, and LZ4. #37912
routine_load_unstable_threshold_second
. #36222
pindex_major_compaction_limit_per_disk
to configure the maximum concurrency of compaction on a disk. This addresses the issue of uneven I/O across disks due to compaction. This issue can cause excessively high I/O for certain disks. The default value is 1
. #36681
enable_lazy_delta_column_compaction
. The default value is true
, indicating that StarRocks does not perform frequent compaction operations on delta columns. #36654
default_mv_refresh_immediate
, which specifies whether to immediately refresh the materialized view after the materialized view is created. The default value is true
. #37093
default_mv_refresh_partition_num
to 1
. This indicates that when multiple partitions need to be updated during a materialized view refresh, the task will be split in batches, refreshing only one partition at a time. This helps reduce resource consumption during each refresh. #36560
yyyy-MM-ddTHH:mm
and yyyy-MM-dd HH:mm
to support TIMESTAMP partition fields in Apache Iceberg tables. #39986
storage_medium
to the view information_schema.be_tablets
. #37070
SET_VAR
in multiple sub-queries. #36871
LatestSourcePosition
is added to the return result of SHOW ROUTINE LOAD to record the position of the latest message in each partition of the Kafka topic, helping check the latencies of data loading. #38298
%
or _
, the LIKE operator is converted into the =
operator. #37515
Fixed the following issues:
storage_page_cache_limit
in certain circumstances. #37740
bitmap_to_string
may return incorrect results due to data type overflow. #37405
SELECT ... FROM ... INTO OUTFILE
is executed to export data into CSV files, the error "Unmatched number of columns" is reported if the FROM clause contains multiple constants. #38045
Release date: February 6, 2024
Fixed the following issues: