Chronicle Etl Versions Save

• Add internal list of known plugins in https://github.com/chronicle-app/chronicle-etl/pull/60 and chronicle-etl plugins:list shows all known plugins
• Coerce dates (from yml files) into Time objects #57
• Always write yml config with string keys
• Add list of connectors from available plugins to README

• Adds authorizer feature (#51) for doing oauth flows for third-party providers. Plugins can now implement their own authorizers

• Chronicle Schema: add analogous edge; add provider_namespace attribute (in which provider_id is unique within) (285b1db30758aab3d3875d00052ecdb096ed6a94)
• Improve output when CLI run without flags (#44)
• Improve error handling for bad runs (#26)

• add homebrew installation
• better UX for saving/using saved jobs
• fix json/csv output when using STDOUT (#31)
• add non-line-separated JSON loading (#43)

• Better handling of non-installed plugins (#34)
• Add fuzzy durations for --since and --until (e.g. $ chronicle-etl --since 10d). #32
• Add secret management system #36
• Set config file permission to 0600 #40
• Add specs for config related stuff (using FakeFS)

• improve handling of plugins failing to load

• Revamp connector installation system to be oriented around plugins (f5bfcfc412186a0cf5a70c11b8b35fc5a6b58d41)
• Add numeric type to settings (setting :foo, type: :numeric)
• Add ability to disable color output (#19)
• Improve CSV, JSON loaders

• Add --silent flag for setting log level to SILENT (#17)
• Add better support for jobs which preserve raw extraction data
• Add basic type coercion for configuration settings
• Make TableLoader default and remove StdoutLoader
• Improve CSV, file, and JSON extractors

• Add Configurable mixin for extractor/transformer/loader classes that gives a common pattern for registering settings for classes

Test < Chronicle::ETL::Extractor
  include Chronicle::ETL::Configurable
  setting :when, type: :date, required: true
end

t = Test.new(when: '2022-02-24')
t.config.when

• Add connectors:show PHASE IDENTIFIER command to CLI
• Add specs for CLI and many internal classes; add Guardfile; integrate CI with Github Actions
• Set minimum Ruby version to 2.7
• Add universal flags for jobs:run for common extractor/transformer/loader options
  • --input
  • --until
  • --since
  • --limit
  • --output
  • --fields

• Use JSONAPI serialization by default for rest and stdout loaders
• Entities can have Attachments
• Add image transformer for processing jpeg/png images files
• Add internal Registry for keeping track of built-in and external connectors
  • connectors:install handles the gem installation for external connectors
  • connector classes must call register_connector (example) to register itself with the Registry
• Add better job logging to stderr and control via --log-level and --verbose flags
• Add ability to do a dry run of a jog via the --dry-run flag