Xxhashdir Save

⚡Fast filysystem fingerprinting using xxHash

Project README

xxhashdir

Build Status

Usage

this package does fast filesystem fingerprinting using xxHash

# instead of "find . -type f -exec xxhsum {} \+"
$ ./xxhashdir .
...
880788507839261490    README.md
11541949788444589007  .travis.yml
6467850080536788703   bin/xxhashdir.go
...

typical CLI use:

./xxhashdir dir > before
# modify fs
./xxhashdir dir > after
diff <(sort before) <(sort after) | sort -nk3

Speed

Times faster than find + exec. Digesting xcode-10.2 with >250K files:

Time Cmd
656 sec time find /Applications/Xcode.app -type f -exec xxhsum {} ; > xxhsum.txt
88 sec time find /Applications/Xcode.app -type f -exec xxhsum {} + > xxhsum.txt
45 sec time ./xxhashdir /Applications/Xcode.app > xxhsumdir.txt

Golang api

func Hashdir(root string, out chan Entry)

where

type Entry struct {
    Path   string
    Xxhash uint64
}
Open Source Agenda is not affiliated with "Xxhashdir" Project. README Source: lunatic-cat/xxhashdir
Stars
50
Open Issues
0
Last Commit
4 years ago
License
MIT

Open Source Agenda Badge

Open Source Agenda Rating