Rtlamr Versions Save

Fix R900 Decoder and New Release Tool

This minor release is a fix for the R900 decoder.

I've also moved to a new release tool that greatly simplifies making binary releases for new versions. As a result, I'm now only providing a signature for the checksums file generated by the tool, which should be sufficient to verify that I built the binaries uploaded for each release.

Changes:

• 4805445c Fix version flags for use with goreleaser.
• cec57c9d Store decoder packet config in r900 parser. Fixes #102

This release contains primarily code and documentation cleanup. There are a few optimizations and some functionality has been both added and removed.

There has also been an unusual amount of time/work between releases, so it's time to provide new pre-compiled binaries. All of the binary releases were compiled using go version go1.11 linux/amd64.

Big Changes:

• Removed -logfile, see 3fef2118 for console redirection equivalent functionality.
• Removed -decimation due to performance improvements, no longer necessary.
• Added new -msgtype value r900bcd for those funky R900 meters that transmit binary-coded digits for consumption values.
• Added new -msgtype value netidm for type 8 net meters that track both consumption and production simultaneously.
• For ease of use with docker and systemd, flags can now be set with environment variables, see 9e258fdd.
• Add netidm for type 8 power meters. Same preamble and CRC as standard IDM, different packet structure includes aggregate consumption and generation.
• Add preliminary multi-protocol decoding support.

All Changes:

• 6dcad791 Fix travis-ci configuration.
• fb5c2462 Update travis-ci for new go versions.
• 44e1520d Remove generator test.
• 0d41d2f9 Simplify input buffering.
• 7f303236 Move decode and parse to protocol.
• 6bfcafd3 Add multi-protocol decoding support.
• d4b98558 Correct filter benchmark. Remove symbol length test.
• 4be127ca Rename wiki page Command Line Flags to Configuration.
• 7d7707c8 Move usage from README.md to wiki. Expand message type section.
• 1d3a52bd Merge branch 'netidm'.
• 301eb821 Switch intervals back to unsigned integers.
• 8f43170d Move content of help.md to wiki.
• c6540fef Add multiple of 8 requirement to symbollength.
• a0452b41 Move quantization into filter.
• 74bd955a Change feedback.
• b2f22547 Delete docs folder for conversion from github pages to wiki.
• 596595bc Add scm_polynomial.svg
• 4540f7d6 Add images and equations for migration to wiki.
• cbd4df4e Round packed signal length up to nearest byte.
• 19cf8234 Replace transposition and byteFinder preamble search.
• dfc901fd Add LastGeneration field.
• bc732cba Assume intervals are signed 14-bit integers.
• 42bfbac0 Invert quantize output to match previous behavior.
• f56d86ef Apply a few new optimizations.
• 26da5a35 Remove decimation from tests and DecCfg from Decoder.
• bac80317 Remove decimation functionality.
• 50292b19 Add link to rtlamr-collect.
• 26f7dba5 Add rtltcp as git subtree.
• 48b7cada Remove rtltcp submodule.
• ce8f749a Change FROM to allow arm compatibility.
• 83208e44 Fix table in help.md
• 19e32367 Change format description, update help.md Fixes #81
• 31f4d924 Update Dockerfile and vendor bemasher/rtltcp.
• 6dbef3d6 Add description for Transmit Time Offset.
• b84344e2 Don't use channel as buffer, rtl_tcp handles that sufficiently.
• f8b5ccb4 Update readme and travis-ci with new version requirements.
• f25fa425 Fix race condition introduced in 8a0e86e
• 152b4169 Update Dockerfile (#79)
• 8a0e86eb Handle blocks asynchronously.
• 5910c8f4 Send "Time Limit Reached" to stderr. Fixes #76
• bf6b099c Don't fail hard on invalid environment variable values.
• 914fe8dc Fix front-matter.
• 3fef2118 Remove -logfile flag.
• b2dba490 Revert change to Quantize invocation.
• 57926fc5 Refactor symbolsPerBlock, should have no effect though. Add decoder benchmarks.
• 1da5fcdc Minor optimizations in Transpose and Quantize.
• 9e258fdd Allow overriding flag defaults with environment variables.
• a0ec0e82 Revert change to invocation of Quantize.
• 9f1f0def Remove reference to alpha-max beta-min magnitude calculation.
• fcc41fd4 Remove remnants of MathJax and duplicate katex loads.
• 89d2a9ac Wrapping with divs works, but need unescaped $$'s for KaTeX.
• 4e31f28d Yet another attempt at getting kramdown to behave with latex.
• 0788ed93 Output html entities as is.
• a23d0b5d Kramdown, again. Using nomarkdown tags.
• 8a9045af Convert <p>'s to <span>'s...
• 830cd370 Wrap KaTeX blocks in <p>, kramdown is still mangling these.
• 3e0b0423 Escape $$'s, kramdown mangles these with their own latex handling.
• 2ace5627 Add KaTeX fonts.
• ce569ed6 Add auto-render source file.
• a318039c Fix typo in KaTeX's auto-render invocation.
• 80402429 Replace pygments with rouge for code highlighting.
• 823ed3a6 Switch to KaTeX for equation typesetting.
• 911713fc Move github pages content into /docs
• afde396e Add Decoder benchmark.
• e7e0e2dc Rename SymbolLength to ChipLength and SymbolLength2 to SymbolLength.
• 2ae54854 Minor fix and optimizations.
• 49906adc Update comment about break point in SCM+ decoder.
• 8e985098 Add ProtocolID check to SCM+ decoder.
• 7de771cb Correct random frequency selection for SCM generation test.

Cut away a lot of the fat in various buffers and implemented a protocol wrapper for R900 meters that transmit consumption as a hexadecimal coded digits. This also marks the first version which performs well enough to receive at full sample rate on an RPi2 without dropping any samples.

Changes:

• 2b386b0 Add R900 BCD Protocol (#58)
• 17b0016 Remove IQ buffer from decoder, implement buffer for sample dumping in main program.
• cedc2f8 Rename SCM test and update meters.csv
• 082d452 Maintain internal signal buffer since we trimmed the decoder-specific one.
• b567710 Resize Filter and csum buffers. Rename Pack to Transpose.
• 1c5bbb4 Simulation (#51)
• 2bee3a8 Rename recv.go to main.go
• 1742cc5 Fix length of bits to pack.
• 508c42c Remove old calculation from MagLUT.
• 1378c68 Remove Sqrt in MagLUT, sensitivity is unaffected. Use appropriate scaling in LUT calculation.
• faba109 Trim Signal and Filtered buffers to only necessary lengths.

Decimation was broken in the previous release when the new preamble search was implemented. This release fixes that bug.

Changes:

• 31bbf3f Use decimated packet configuration in preamble search. Fixes #47.

This release includes a lot of changes most important of which is SCM+ support!

Changes:

• e8ea9d9 Remove cpuprofile flag, add version flag.
• dad7ae0 Check lowercase version of msgType against registered parsers.
• 99be3e7 Parsers register themselves like in database/sql.
• 206c0dc Refactor packet config to allow modification. Closes #43
• 64a0ded Merge pull request #42 from bemasher/SCM+
• a6286ba Add prototype SCM+ parser.
• 99f5b70 Merge pull request #41 from bemasher/search cb511be Use the go standard library implementation of Boyer-Moore string search from the strings * package for preamble searching.
• 40470d4 Wait for a message from each meter given in -filterid when combined with -single. Closes #38
• 1e03a51 Clarify Pack description.
• c45fdef Remove old travis-ci badge.
• 1364a85 Remove flag debug print.
• 5555d22 Fix travis-ci badge.
• d453c86 Remove filter chain debug print.
• 42b1b3d Merge branch 'msghandler'
• 6b376a3 Write plain text encoder.
• f8797da Update travis-ci config to use new infrastructure.
• 938607e Oops, Filter is the method, MessageFilter is the interface.
• 7314e7a Simplify filter flag handling. Rename Filter interface.
• 548dac1 Refactor message filtering.
• 767fea7 Remove -fastmag from usage.
• 42ceb2c Remove FastMag
• 30ee13e Remove unnecessary benchmark.
• de00b1d Update usage and unique flag description.
• c4340c3 Fix symbol index for checksum copy.
• 34b9eb6 Refactor to check against checksum for uniqueness.
• dbdbcae minor fixes
• b9240f9 add -unique

This release is sort of half-way between major and minor:

• Decimation is now available through the -decimation flag, reduces cpu resource cost at the expense of sensitivity, factors between 2 and 8 still seem to provide reasonably good sensitivity at full bandwidth. Will require simulation to determine effective sensitivity (coming soon).
• Some minor refactoring in SCM and IDM packet handlers.
• Whole blocks of samples are now read and buffered from rtl_tcp.

Changes:

• ae0364d Merge branch 'decimation'
• 07607af Refactor message unpacking in SCM and IDM.
• 7f05214 Add decimation factor checking. Clarify warning.
• 8f3cb52 Apply gofmt to everything.
• 04d73e4 Update logging to include decimated configuration.
• de4bffe Fix csum length.
• 953f371 Move R900 message statement.
• 22906a5 Update link in README to point to gh-pages documentation.
• 3bd0a4e Update flags in README.
• efea189 Implement decimation in AlphaMaxBetaMinLUT.
• 3a29e9b Implement decimation in R900 decoder.
• 586359c Move decimation to demodulator, only MagLUT for now.
• 1373ceb Fix block read, reads might span writes to the pipe.
• 588ef52 Read whole blocks from server concurrently with io.Pipe().
• f782879 Remove config debugging statements.
• c2855e7 Implement decimation, except in R900 decoder.
• f3d51f2 Update compatible meter table location.
• 3249c73 Move compatible meter table to csv format.

This is a minor release:

• The r900 decoder now handles a few more of the unknown fields.
• Documentation added for a few new supported meters submitted by users.
• Optimizations in the magnitude execute loop and the preamble search.

Changes:

• e1d9b04 Merge pull request #27 from sandeen/master
• 2336082 r900: decode leak, fraud, and backflow status flags
• 9be534c Add use cases.
• d4afba8 Add 100GDLAN, update commodity type list.
• 15e1d1d Optimize execute loop for magnitude demodulators.
• 169c530 Switch to bytes.Equal in preamble search. Simplify calculations for slice allocation.
• 0f08d69 Add user-submitted meters.

Now with experimental Neptune R900 support!

Changes:

• eb2b055 Fix galois field package path.
• a88bbf1 Simplify symbol decoding process.
• 57e1be9 Add gf package license.
• 6f3d973 Integrate R900 decoder.
• 8a57030 Add R900 decoder.
• 19dadbb Move CenterFreq to PacketConfig for per protocol defaults.
• 8865d2e Update travis-ci configuration.
• 979aa93 Fix rounding problem in parser.
• 0c47867 Move s to correct scope.
• c9c1d99 Simplify duplicate packet check.
• 3c66161 Separate demodulated signal from filtered signal.
• dbb4227 Set actual field lengths.
• 86a2abf Refactor for signal access from parser.
• 11d5802 Rename for later protocols.
• 03369d1 Simplify packed byte array length calculation.
• 3bcb52b Fix packed byte array length calculation.
• b326ea9 Limit checksum to valid length.
• 2d12445 Update usage.
• 76ee4cb Update badges.
• ea81369 Add new error case for SCM parsing.
• 2b055fc Convert filterid and filtertype flags to comma-separated lists.
• ed47c5c Fix NewDataFromBits
• 9994aaa Add calculator for valid symbol lengths.

This is a minor release mostly dealing with documentation and some refactoring. The refactoring will make future work on FHSS tools less painful.

This is also the first release to include detached gpg signatures with each archive:

# Extract a binary archive.
tar -xzvf rtlamr_linux_amd64.tar.gz

# Retrieve public key used to sign the binary.
gpg --keyserver pgp.mit.edu --recv-keys DD1EC7EE

# Verify the signature.
gpg --verify rtlamr.asc rtlamr

See my keybase profile for proof of ownership: https://keybase.io/bemasher

Changes:

• f955a45 Fix magnitude benchmarks.
• 5f508e5 Refactor parsing into packages for reuse.
• 0092dd1 Fix FastMag.
• 659a8b3 Fix for package refactor.
• b650170 Refactor decoding to package for reuse.
• d0f46b3 Add comments on decoding methods.
• df217be Add format message structures.
• bdd008f Update sensitivity.
• 16e4b4a Added description for each flag.
• 69d955c Create help.md
• 1f4e9dd Update purpose for clarity.

First major release without any external dependencies! FFTW is no longer required for builds which allows cross-compilation.

Changes:

• README.md updated to reflect new build process along with other information new to the project.
• Flags which override values determined by rtlamr are now handled properly (gain, center frequency, sample rate).
• Matched filter optimized, ~2x performance increase.

Future:

• Changes to log format in plain-text may change for IDM packets, they're still huge.
• FHSS tools may need to wait. Testing shows rtl-sdr dongles don't have good enough rejection at the moment to reasonably determine that a decoded message occured in the band we expected them to due to aliasing and the like. These will take a lot more effort in signal processing to accurately break the FHSS hopping pattern.
• Writing raw samples to file still need to be non-blocking.