Glusterd2 Versions Save

Way too many changes!

• Major code refactor to improve source code structure
• Big updates to ReST and ReST API
• Loads of new CLI
• Bash completion for CLI
• Validation for xlator options
• TLS for store and elasticetcd
• Improved volgen, ie volgen2
• Internal events framework

RPMs available at https://copr.fedorainfracloud.org/coprs/kshlm/glusterd2/

• Switch vendoring tool to dep
• make dist and make dist-vendor
• RPMs using dist-vendor archive

• CLI
• Volume set
• Volgen with templates
• Requires GlusterFS experimental branch

Lot's of changes since the last release

• GD2 now uses an auto scaling etcd cluster, which automatically selects and maintains the required number of etcd servers
• Preliminary support for volume expansion has been added. (Note that rebalancing is not available yet)
• An end to end functional testing framework is now available
• RPMs for EL7 and Fedora 25 and above.

• Clients can actually get volfiles from GD2.
  • GD2 now has a working SunRPC server that implements the required mount and portmap programs and procedures for clients to be able to fetch volfiles and mount
  • Portmap doesn't work reliably, but we'll fix it soon. For now, portmap returns ports only if your brick paths are /export/brick{1..4}/data. If not, the client enters into an infinite portmap request loop.
• Multiplexed rest and sunrpc on a single port.
  • One less port to keep open. We could also get gRPC multiplexed on this port as well.
• Refactor all the servers and use supervisor-trees
  • All the current servers in GD2 -- gRPC, rest and sunrpc servers -- have been refactored. They now implement a standard service interface that allows them to be managed using supervisor trees. More information can be found in the doc/supervisor-trees.md file

Now start and use volumes!

Another @prashanthpai mostly release.

GD2 now successfully starts bricks. So you can now actually mount volumes and make use of them. To mount, for now you'll need to use the glusterfs command directly and point to the fuse volfile.

How to create and mount a volume in GD2

• Get & start GD2 cluster by following instructions in the Testing releases page
• Using the peer management apis described on the ReST API page, create a GD2 cluster.
• Create a volume using the volume create API, then start it using the volume start API.
• Mount the volume by doing, glusterfs -f <workdir>/vols/<volname>/trusted-<volname>.tcp-fuse.vol <mount-point> and use it!

Ideally this should just work. But as we don't support portmap yet, the volfile needs to be modified to add the ports to the bricks. Add option remote-port <port> to all the client xlators in the volfile.

New release! Now with embedded etcd [PR #148].

This release is all @prashanthpai. In addition to embedding etcd, he's been on a spree doing lots of minor fixes, and fixing a lot of linter issues.

This release also has a corresponding docker image gluster/glusterd2-test:v4.0dev-3, which can be used to test this release easily.

Information on how to test GD2 can be found in the Testing releases wiki.

New changes

This release has 2 big changes to GD2.

• PR #128 switched GD2 to use gRPC for internal peer-to-peer RPCs.
• PR #136 introduced the daemon framework, which allows GD2 to actually start bricks. But bricks don't start due to 2 pending issues.
  • #150 - because rundir and logdir are missing
  • #146 - because generated volfiles are incomplete and incorrect

In addition to the above changes to GD2, a new Docker image, kshlm/glusterd2-dev:centos-latest with glusterfs-server 3.8.4 installed. This is used by the included Vagrantfile to spin up 4 nodes which can be used to test during GD2 development.

How to test

A docker image and Vagrantfile with the latest release will be provided soon to help easily test GD2.

Running GD2

• Download the released binary
• Extract and add glusterd2 binary to your $PATH
  • GD2 can run as a non-root user if CAP_SYS_ADMIN is set on the binary
    • Run sudo setcap cap_sys_admin+ep <path-to>/glusterd2
• Ensure etcd is available in your $PATH
• Create an empty working directory
• Run GD2 glusterd2 --workdir <workdir path>

Start GD2 on 2 or more machines/VMs/Containers etc.

Testing commands

GD2 does not have a CLI yet. The ReST API is the only way to communicate with GD2 right now.

The ReST API can be used using curl or a ReST API client tool of your choice. My recommendation is postman.

Refer to the API documentation for more information on the available commands.

Known Issues

• For volume start to work, WORKDIR/run/gluster and WORKDIR/log/glusterfs/bricks. This atleast allows the brick process to attempt to parse and load the volfile, but it fails anyway.

This is just a preview of what has been happening in GlusterD-2.0 (GD2), to give users a taste of how GD2 is evolving.

GD2 can now form a cluster, list peers, create/delete,(psuedo) start/stop and list volumes. Most of these will undergo changes and refined as we progress.

We'll be providing periodic dev builds for people to test.

Currently builds are only available for Linux on x86_64. Vagrant and docker releases are planned to make it easier to test GD2.

How to test

Running GD2

• Download the released binary
• Extract and add glusterd2 binary to your $PATH
  • GD2 can run as a non-root user if CAP_SYS_ADMIN is set on the binary
    • Run sudo setcap cap_sys_admin+ep <path-to>/glusterd2
• Ensure etcd is available in your $PATH
• Create an empty working directory
• Run GD2 glusterd2 --workdir <workdir path>

Start GD2 on 2 or more machines/VMs/Containers etc.

Testing commands

GD2 does not have a CLI yet. The ReST API is the only way to communicate with GD2 right now.

The ReST API can be used using curl or a ReST API client tool of your choice. My recommendation is postman.

Refer to the API documentation for more information on the available commands.