cloud-native distributed storage
UPGRAGDE NOTICE Please refer to the notice of release v2.5.0 for how to upgrade from previous versions.
Please refer to the notice of release v2.4.0 for how to upgrade from previous versions.
fuse client fix occasional IO errors. (#1179 #1205 #1215)fuse client statfs system call returns the correct inode count. (#1192)ci unlock some useful LTP test cases; make ci results checking more strict; upload logs after ci finishes for debug convenience.objectnode support subdir checks. (#1208)if your ChuBaoFS version is v2.3.* or before, and need to upgrade to v2.4.* , you must follow these upgrade steps:
first of all, firewall open two more port (17710 & 17810) in your machine to support tcp multiplexing；
17710 is calculated by 17210, the port listened by metanode, add 500
17810 is calculated by 17310, the port listened by datanode, add 500
then add item '"enableSmuxConnPool"=false' in your datanode's config file, upgrade all datanodes; detail steps as follow:
a. using new dataNode bin file replace the old version
b. add '"enableSmuxConnPool"=false' in your datanode's config file
c. restart datanode process
d. update all the datanodes in the custer
update '"enableSmuxConnPool"=true' in your datanode's config file, restart all datanode again;
notes: before set enableSmuxConnPool as true, you must finsh step 2.
upgrade all metanode bin file and restart meta process;
upgrade all master bin file and restart master process;
upgrade all client bin file and restart client process;
so, please attention to upgrade datanode component, other components (metanode / master / client) upgrade step as usual.
dataNode smux support #1124
metaNode smux support #1124
datanode metanode support TickInterval configurable #1117
master remove token-based volume read-write permission control #1119
datanode fix build repair task panic #1124
metanode too many open/close stream when delete limit #1124
master do split on metanode partion if inode count larger than maxinodecnt #1124
datanode dp.raftPartition nil pointer #1124
datanode fix datanode repair log print error #1124
master partition be split can be write if not full and less then on step #1124
metanode new a gauge instance every time to avoid a gauge instance modified concurrently #1124
master fix conflict for histogram data when report data concurrently #1124
datanode fix bug when packet opCode=OpExtentRepairRead and network is unstable #1124
Note that when upgrading from versions prior to v2.3.0 to version v2.3.0 or beyond, servers(i.e. resource manager, metanode and datanode) must be upgraded before client(i.e. fuse client and objectnode) due to #1098.
client: support multiple subdir permissions for an individual user. #1051
sdk: introducing libsdk so applications can embed the use of chubaofs to a single binary and no additional process is required at runtime. #1082
datanode: improve the shut down process of datanode so the raft apply id is persisted. #1030
raft: auto fix crc-mismatch raft log crc. #1039
objectnode: fix key encoding issue of S3 ListObjects API. #953
datanode: fix data partition decommission timeout. #972
metanode: change mtime of the directory when creating or deleting dentry. #1000
datanode: fix statfs deviation when using EXT4 as the underlying local filesystem for datanode. #1031
client: func sortHostsByDistance change dp's hosts. #1106
datanode: Introducing data partition selector which allow users to customize the client's selection logic for data partition when reading and writing, and can switch at any time. #853
client: Add enablePosixACL configuration to enable POSIX ACL feature. #906
datanode: Improved data recovery speed. #899
datanode: Fix the issue that the information in the memory is not released immediately after deleting the extent, which causes the status information of the data partition to be incorrect. #938
datanode: Fix the issue that repeatedly sending requests to the failed data partitions. #937
objectnode: Fix the issue that recursive making directory concurrent with sample multi-layer path prefix. #904
object: Implemented S3 api 'DeleteBucketPolicy'. #757
master: Introducing management API for volume capacity expanding and shrinking. New capacity must be set more than 20% lager than used. #764
master: Introducing management API for updating node address. #813
master: Introducing management API for checking nodes. #813
metanode: Add header and checksum verification for EXTENT_DEL files. #813
cli: Add logging for CLI tool, and added several commands to call master API. #764 #801
compile:Support direct compile or docker cross compile on Arm64 platform. #779
client: Introducing nearRead option that allow client read data priority from the nearest DataNode to improve reading performance. #810
client: Update volume follower read config from master periodicity. #837
object: Improved compatibility for ListObjects and ListObjectsV2 interfaces. #769
master: Check whether the used space between the replicas is consistent when performing the load partition operation. #813
metanode datanode: Optimize batch delete Extent; add autoRepairLimitRater on DataNode. #781
datanode: Introducing autoRepair option for limit data repair speed on DataNode. #842
datanode: The data repair task will skip process extent file which has been already deleted. #842
master: When loading metadata of cluster, using the current ID instead of the loaded ID. #821
metanode: Fix the deadlock problem while MetaNode deleting dentry. #785
metanode: Fixes #760 Add inode to freeList when NLink is 0, and delete inode 7 days later. #767
datanode metanode: When DataNode and MetaNode start, the partitions not exist in cluster view will be renamed to expiredPartition and skip loading. #824
datanode metanode: Fixes issue#698, Raft instance delete itself by applying ConfChange raft log. #866
metanode: MetaNode may not free space. #838
client: Fix the problem that client opening file with outdated extents information. #783
client: Batch inode get mechanism is out of service. #847
client: Fix the problem that makes authorized user mount volume failure. #828
datanode: Fix several issue in automatic repair process for tiny extent. #855 #857
datanode: Fix the problem that the client still applies to create a new extent to a data partition that has no space. #867
console: ChubaoFS Console is a web application, which provides management for volume, file, s3, user, node, monitoring, and alarm.
The ChubaoFS Console makes it much easier to access services provided by ChubaoFS. #728metanode: Provide compatibility verification tool for meta partition. #684
object: CORS access control. #507
fuse: Introduce fsyncOnClose mount option. Choose if closing system call shall trigger fsync. #494
Release2.1.0 did a lot of work to optimize memory usage.
master: Add set/get deleteBatchCount interfaces. #608
master: Delete partitions on DataNode/MetaNode concurrently in decommission action. #724
master: Added api for getting & setting parameters for batch deleting extents. #726
metanode: Filter inode candidates that will be sent to a partition in batch-inode-get to save memory. #481
metanode: Batch delete inode, unlink&evict dentry. #586
datanode: Prioritize healthy data node when sending message to data partition. #562
metanode datanode: Check expired partition before loading. #624
object: Implemented several conditional parameters in GetObject action. #471
object: Making S3 credential compatible with earlier version of ChubaoFS. #508
object: Totally support presigned url; Partial support user-defined metadata; Making ETag versioned. #540
object: CopyObject across volumes; Support coping directory; Support coping xattr. #563
object: Parallel downloading. #548
object: Add 'Expires' and 'Cache-Control' in putObject/copyObject/getObject/headObject. #589
object: Modify part system metadata. #636
fuse: Decrease request channel size to save memory. #512
fuse: Let the client daemon inherit all the environment variables but not just PATH. #529
fuse: Introduces a debug interface /debug/freeosmemory to trigger garbage collection manually. #539
fuse: Add configuration MaxCPUs. #546
fuse: View client log through http proto. #552
fuse: Support modifying ModifyTime of inode by setAttr. #733
deployment: Add log map in docker-compose.yaml. #478
deployment: Introduce nginx to improve compatibility. #534
test: Add testing script for S3 APIs in python. #514 #595
monitor: Add detailed vol info in grafana dashboard. #522
cli: Some new features for CLI tool: bash completions, configuration setting, decommission, diagnose etc. #555 #695
metanode: Accelerate deletion. #582 #600
datanode: Limit replica number of data partition to at least 3. #587
datanode: Set if auto-repair by http interface. #672
metanode datanode fuse: Optimizations on port checking. #543 #531
metanode datanode: Validate creating partition request from master. #611
object: Refactor copy object function. #563
fuse: Replace process of token validation from meta wrapper to client. #498
fuse: Adjust rlimit config for client. #521
metanode: When overwriting a file, if the inode in a dentry is updated successfully, ignore unlink and evict inode errors. #500
metanode: Fix incorrect metrics collection for volume usedGB in monitor system. #503
metanode: VolStat can correctly update when force update metaPartitions frequently. #537
metanode: The MaxInodeID of meta partition is not synchronized when recovered from snapshot. #571
metanode: Free Inodes by raft protocol. #582
metanode: Painc with deleteMarkedInodes. #597
object: Fix the parsing issue of the two preconditions of If-Match and If-None-Match. #516
object: Change the time format in list buckets API to UTC time. #525
object: Fix xml element of DeleteResult. #532
object: Empty result in list operation when run in parallel with delete. #509
object: Change from hard link to soft link in CopyObject action. #563
object: Solved parallel-safety issue; Clean up useless data on failure in upload part. #553
object: Fixed a problem in listing multipart uploads. #595
object: Solve the problem that back-end report “NotExistErr” error when uploading files with the same key in parallel. #685
fuse: Evict inode cache when dealing with forget. #523
Release candidate version 1 for v2.1.0.
followerRead a client specific option. #382
dentry-cache to client option. #453
iget error due to metapartition split. #446
data partition and meta partition. #467
See CHANGELOG.md for details.