Irmin Versions Save

Irmin is a distributed database that follows the same design principles as Git

3.1.0

2 years ago

CHANGES:

Fixed

  • irmin-pack

    • Drop unnecessary runtime dependency on ppx_irmin. (#1782, @hhugo)
    • Split the unix part of irmin-pack into irmin-pack.unix (#1783, @hhugo)
  • irmin-unix

    • Fix conflicting command line arguments for push, pull, fetch and clone (#1776, @zshipko)
    • Fix issues with Sync functions by provided a better default Mimic.ctx. A side-effect of this update is that the remote function now returns an Lwt promise. (#1778, @zshipko)

Added

  • libirmin
    • Create libirmin package providing a C interface to the irmin API (#1713, @zshipko)

Changed

  • irmin-bench
    • Make trace replay API public and simpler (#1781, @Ngoguey42)

3.0.0

2 years ago

CHANGES:

Fixed

  • irmin

    • Fix the implementation of comparison on Irmin.Tree objects to use the comparison defined on hashes. The previous implementation was unstable. (#1519, @CraigFe)
    • Default implementation for contents, nodes and commits can no longer trigger pre_hash collisions. This changes their hash. (#1715, @Ngoguey42, @icristescu)
  • irmin-pack

    • Improve the performance of Index encode and decode operations by eliminating intermediate allocations. (#1577, @CraigFe)
  • irmin-unix

    • Fix terms that can be manipulated at runtime by delaying computation (#1645, @zshipko)

Added

  • irmin

    • Add Read_only.S and Read_only.Maker module types (#1343, @samoht)
    • Append-only and content-addressable backend implementations have to provide close and batch functions (#1345, @samoht)
    • Atomic-write backend implementations have to provide a close function (#1345, @samoht)
    • Add a function Store.Tree.singleton for building trees with a single contents binding. (#1567, @CraigFe)
    • Add with_handler and head to Store.Backend.Node and Store.Backend.Node_portableto work with recursive node structures from irmin core. (#1712, #1746 @Ngoguey42). Forward port of #1692 and #1670.
    • Add proof, to_proof and of_proof to Store.Backend.Node_portable (#1716, @Ngoguey42). Forward port from #1583.
    • Add hash_exn to Store.Backend.Node.Val and Store.Backend.Node_portable (#TODO, @Ngoguey42) Forward ported from #1741.
    • Add a Store.Tree.kinded_hash function. (#1767, @Ngoguey) Forward ported from #1625.
    • Add Contents.String_v2, Node.Generic_key.Make_v2 and Commit.Generic_key.Make_v2 for backward compatibility with older stores. (#1715, @icristescu)
  • irmin-bench

    • Many improvements to the actions trace replay:
      • Support for the layered store (#1293, @Ngoguey42)
      • Fix replay for the first ~650k commits (was ~13k) (#1314, @Ngoguey42)
      • Can change inode configuration prior to replay (#1326, @Ngoguey42)
      • Check hash of commits (#1328, @icristescu)
      • Fix the path flattening technique (#1357, @Ngoguey42)
      • Introduce a new actions trace that can support replaying up to ~1300k commits. (#1358, @Ngoguey42)
      • Improve the stats collection and stats report (#1367, #1384, #1403, #1404, #1416, #1429, #1438, #1501, #1616, @Ngoguey42, @maiste)
      • Enable replay in CI (#1430, @Ngoguey42)
      • Enable replay in CB (#1441, @Ngoguey42)
  • irmin-mem

    • Add Irmin_mem.Content_addressable (#1369, @samoht)
  • irmin-pack

    • Add a stat-store command to irmin-fsck to output stats on the tree under a specified commit (#1391, @icristescu, @Ngoguey42, @CraigFe).
    • Add new counters in Stats (#1570, @Ngoguey42).
    • Add an option to configure the index function and pick the relevant bits in a cryptographic hash by default (#1677 #1699, @samoht)
    • Verify inode depth invariants (#1711, @Ngoguey42). Forward port of #1665.
  • irmin-unix

    • Update irmin CLI to raise an exception when an invalid/non-existent config file is specified (#1413, @zshipko)
    • Add --commit flag to CLI to load a store from a specific commit hash (#1721, @zshipko)
  • irmin-tezos

    • Add a new package to mirror Tezos tezos-context.encoding library. That'll simplify building benchmarks and custom tools (#1579, @samoht)

Changed

  • irmin

    • Irmin.Sync is now a namespace: use Irmin.Sync.Make(S) instead of Irmin.Sync(S) (#1338, @samoht)

    • Irmin.Private is now Irmin.Backend (#1530, @CraigFe)

    • Store.master is now Store.main. The existing Store.master function is deprecated and will be removed in a future release. (#1564, @CraigFe)

    • Store.Private is now Store.Backend (#1530, @CraigFe)

    • Store.Private.Sync is now Store.Backend.Remote (#1338, @samoht)

    • Irmin.Branch.S.master is now Irmin.Branch.S.main (#1564, @CraigFe)

    • Irmin.Private.{Commit,Node} are now Irmin.{Node,Commit}. (#1471, @CraigFe)

    • All module types are now using snake-case and are not capitalized anymore. (#1341, @samoht)

    • Move signatures for backend stores into their own modules. All the X_STORE sigs have moved to X.S:

      • APPEND_ONLY_STORE is now Append_only.S
      • CONTENT_ADDRESSABLE_STORE is now Content_addressable.S
      • ATOMIC_WRITE_STORE is now Irmin.Atomic_write.S And all the X_STORE_MAKER have moved to X.Maker:
      • APPEND_ONLY_STORE_MAKER is now Append_only.Maker
      • CONTENT_ADDRESSABLE_STORE_MAKER is now Content_addressable.Maker
      • ATOMIC_WRITE_STORE_MAKER is now Atomic_write.Maker This gives some space to move convenient functors closer to where they belong:
      • Content_addressable is now Content_addressable.Make
      • New Content_adddressable.Check_closed and Atomic_write.Check_closed (#1342, @samoht)
    • Rename Irmin.Make into Irmin.Maker ; stage its result to return Make functor once provided with a content-addressable and an atomic-writes stores (#1369, @samoht)

    • Rename Irmin.Make_ext into Irmin.Maker_ext ; stage its result to return Make functor once provided with a content-addressable and an atomic-writes stores, as well as node and commit makers (#1369, @samoht)

    • Require at least lwt.5.3.0 to use Lwt.Syntax in the codebase (#1401, @samoht)

    • Info implementations are not part of store: use S.Info.v instead of Irmin.Info.v (#1400, @samoht)

    • Rename Commit.V1 to Commit.V1.Make. This functor now takes separate hash and key implementations as arguments. (#1431 #1634, @CraigFe @icristescu)

    • Introduce a Schema module to hold all the types that users can define in an Irmin store. Use this as a parameter to every Maker functor. This is a large change which touches all the backends. (#1470, @samoht, @CraigFe)

    • Add Irmin.Backend.Conf.Schema for grouping configuration keys. Now Irmin.Backend.Conf.key takes an additional ~spec parameter. (#1492, @zshipko)

    • Tree.empty and Node.empty now both take a unit argument. (#1566 #1629, @CraigFe)

    • Rename key type to path and Key module to Path when it is in a path context in Tree and Store. (#1569, @maiste)

    • Move Node.default metadata default values into a Node.Metadata.default to give room for other metadata values (#1611, @samoht)

    • Add support for non-content-addressed ("generic key") backend stores. This allows Irmin to work with backends in which not all values are addressed by their hash. In particular, this includes:

      • New functions: Store.{Commit,Contents,Tree}.of_key.
      • Adds Irmin.{Node,Commit}.Generic_key modules.
      • Adds new types that must be provided by backends: Node.Portable and Commit.Portable.
      • Adds a new type of backend store: Irmin.Indexable.S. (#1510 #1647, @CraigFe)
    • Cache hits in several Tree functions are more frequent than before. (#1724, @Ngoguey42, @CraigFe)

    • Add a new Pruned_hash tag to the error case of several Store.Tree functions (#1744 @Ngoguey42). Forward ported from #1583.

  • irmin-containers

    • Removed Irmin_containers.Store_maker; this is now equivalent to Irmin.Content_addressable.S (#1369, @samoht)
    • Renamed Irmin_containers.CAS_maker to Irmin_containers.Content_addressable (#1369, @samoht)
  • irmin-fs

    • Renamed Irmin_fs.Make into Irmin_fs.Maker (#1369, @samoht)
    • Renamed Irmin_fs.Make_ext into Irmin_fs.Maker_ext (#1369, @samoht)
  • irmin-git

    • All of the configuration keys have moved into their own namespace:
      • Irmin_git.root is now Irmin_git.Conf.root
      • Irmin_git.head is now Irmin_git.Conf.head
      • Irmin_git.bare is now Irmin_git.Conf.bare
      • Irmin_git.level is now Irmin_git.Conf.level
      • Irmin_git.buffers is now Irmin_git.Conf.buffers
      • Irmin_git.dot_git is now Irmin_git.Conf.dot_git (#1347, @samoht)
    • Renamed Irmin_git.Make into Irmin_git.Maker (#1369, @samoht)
    • Require at least git.3.7.0 in the codebase (#1632, @dinosaure)
  • irmin-graphql:

    • Changed the name of the default branch node from master to main in the GraphQL API. (#1564, @CraigFe)
    • Updated to be compatible with generic keys.
      • The Key type is now called Path to match the new name in irmin
      • All key fields and parameters have been renamed to path (#1618, @zshipko)
  • irmin-mirage

    • Renamed Irmin_mirage_git.Make into Irmin_mirage_git.Maker (#1369, @samoht)
  • irmin-pack

    • Changed the implementation of backend store keys to use direct pointers to store contents (by offset in the pack file) when possible, rather than querying the index on each lookup. (#1659, @CraigFe @ngoguey42 @icristescu)
    • The Irmin_pack.Maker module type now no longer takes a Conf argument. (#1641, @CraigFe)
    • The backend configuration type Conf.S requires a new parameter contents_length_header that (optionally) further specifies the encoding format used for commits in order to improve performance. (#1644, @CraigFe)
    • Upgraded on-disk format of pack files to support more efficient lookups and reduce indexing overhead. This change is fully backwards-compatible with existing stores using irmin-pack.2.x versions, but not forwards compatible. (#1649 #1655, @CraigFe @Ngoguey42)
    • Added support for user-specified indexing strategies. The default strategy is to index all objects appended to the pack file (as before), but users may now choose to index fewer objects in order to improve the write performance of the store, at the cost of introducing potential duplicate values to the pack file. (#1664, #1761, @CraigFe, @maiste)
  • irmin-unix

    • Clean up command line interface. Allow config file to be specified when using Irmin_unix.Resolver.load_config and make command line options take precedence over config options. (#1464, #1543, #1607 @zshipko)
    • Irmin_unix.Resolver.destruct has been removed (and partially replaced by Resolver.spec). (#1603, @CraigFe)
    • Update irmin CLI to support empty path in list subcommand. (#1575, @maiste)
    • Add new commands to CLI: branches for listing available branches and log which is similar to git log (#1609, #1727, @zshipko)
    • Update irmin watch to take parameters to specify a command that should be executed when there are new changes (#1608, @zshipko)

Removed

  • irmin-pack

    • Removed the irmin-pack.layered library. Support for the layered store will be restored on a future release of irmin-pack. (#1651, @CraigFe)
    • Removed support for the clear operation in irmin-pack. This operation is incompatible with performance optimisations made in this release. (#1655, @CraigFe)
  • irmin-layers

    • This experimental package has been removed.

2.10.2

2 years ago

CHANGES:

Fixed

  • irmin
    • Fixed a bug causing stream proof extender nodes to have their segments be returned in reverse order (i.e. bottom to top, rather then top-down). (#1742, @CraigFe)

    • Fixed a bug that allowed the creation of overly-large stable inodes via stream proofs. (#1741, @Ngoguey42)

Added

  • irmin
    • Add Store.Private.Node.Val.hash_exn (#1741, @Ngoguey42)

2.10.1

2 years ago

CHANGES:

Fixed

  • irmin
    • Fix bug introduced in #1683 which causes Tree.seq and Tree.list to produce pruned children (#1720, @Ngoguey42)

2.9.1

2 years ago

CHANGES:

Fixed

2.10.0

2 years ago

CHANGES:

Fixed

  • irmin

  • irmin-pack

    • Fix proofs for large inodes by tracking side-effects reads inside the inode implementation (#1670, @samoht, @Ngoguey42)
    • Flush branch store without calling Repo.close (#1707, @zshipko)

Added

  • irmin

    • Add Tree.produce_proof and Tree.verify_proof to produce and verify proofs from complex computations. produce_proof and verify_proof takes a callback over tree and instead of a static list of operations -- this now means that the full Tree API can now be used in proofs, including sub-tree operations, folds and paginated lists (#1625, #1663, #1683, @samoht, @Ngoguey42)
    • Add Tree.produce_stream and Tree.verify_stream to produce and verify stream proofs (#1684, #1692, #1691, @samoht, @Ngoguey42, @icristescu)
  • irmin-pack

    • Verify inode depth invariants (#1665, @samoht)
  • irmin-unix

    • Add tezos store type for irmin command-line (#1678, @zshipko)

Changed

  • irmin

    • Remove Tree.Proof.of_keys. Use Tree.produce_proof instead (#1625, @samoht)
    • Tree.empty now takes a unit argument. (#1566, @CraigFe)
    • Tree.length now takes a tree as argument (#1676, @samoht)
    • Tree.Proof.t now uses a more precise datatype to encode value invariants (#1688, @samoht)
  • irmin-pack

    • irmin-pack: add an option to configure the index function and pick the relevant bits in cryptographic a hash by default (#1677, @samoht)
  • irmin-git

    • Require at least git.3.7.0 in the codebase (#1637, @dinosaure)

2.9.0

2 years ago

CHANGES:

Fixed

  • irmin-pack
    • Improved the performance of Index encode and decode operations by eliminating intermediate allocations (up to 5% fewer minor words allocated) (#1577, @CraigFe)
    • Reduce the number of backend nodes built during export (up to 20% fewer minor words allocated) (#1553, @Ngoguey42)

Added

  • irmin
    • Add Merkle Proofs and expose function to convert a proof to and from a tree. Once converted, normal tree operations can be performed on the proof, as long at it access values contained in the proof. (#1583, @samoht, @Ngoguey42, @icristescu)

Changed

  • irmin-pack
    • Limit inode depth (#1596, #samoht)
    • Adapt to index 1.5.0 (#1593, @icristescu)

2.8.0

2 years ago

CHANGES:

Fixed

  • irmin
    • Tree operations now raise a Dangling_hash exception when called with a path that contains dangling hashes in the underlying store, rather than interpreting such paths as ending with empty nodes (#1477, @CraigFe)
    • Fix the pre-hashing function for big-endian architectures. (#1505, @Ngoguey42, @dinosaure)
    • Fix a bug in Tree.export where nodes could be exported before some of their contents, resulting in indirect hashes in irmin-pack (#1508, @Ngoguey42)

Added

  • irmin
    • Node.seq and Node.of_seq are added to avoid allocating intermediate lists when it is not necessary (#1508, @samoht)
    • New optional cache parameter to Tree.hash, Tree.Contents.hash, Tree.list, Node.list, Node.seq and Node.find to control the storing of lazily loaded data (#1526, @Ngoguey42)
    • Add Node.clear to clear internal caches (#1526, @Ngoguey42)
    • Added a tree argument to Tree.fold to manipulate the subtrees (#1527, @icristescu, @Ngoguey42)
    • Add a function Store.Tree.pruned for building purely in-memory tree objects with known hashes. (#1537, @CraigFe)
    • Added a order argument to specify the order of traversal in Tree.fold (#1548, @icristescu, @CraigFe)

Changed

  • irmin

    • Node.v is renamed to Node.of_list (#1508, @samoht)
    • Rewrite Tree.export in order to minimise the memory footprint. (#1508, @Ngoguey42)
    • Remove the ~force:`And_clear case parameter from Tree.fold, ~force:`True ~cache:false is the new equivalent. (#1526, @Ngoguey42)
    • `Tree.fold ~force:`True and `Tree.fold ~force:`False don't cache the lazily loaded data any more. Pass ~cache:true to enable it again. (#1526, @Ngoguey42)
    • Do not allocate large lists in Irmin.Tree.clear (#1515, @samoht)
  • irmin-git

    • Upgrade irmin-git to git.3.5.0. (#1495, @dinosaure)

2.7.2

2 years ago

CHANGES:

Added

  • irmin-pack
    • Added integrity-check-index command in irmin-fsck. (#1480, #1487 @icristescu, @samoht)

Changed

  • irmin-pack
    • reconstruct_index is now traverse_pack_file, it allows for both index reconstruction and index checking (#1478, @Ngoguey42)

2.7.1

2 years ago

CHANGES:

Fixed

  • irmin-pack
    • Fix termination condition of reconstruct index (#1468, @Ngoguey42)