:lipstick: Durable and asynchronous data imports for consuming data at scale and publishing testable SDKs.
Unifies the synchronous and asynchronous APIs, so they are one and the same.
Replaces coroutines with fibers as the new async technology.
This release reframes Providers as services, encouraging dependency injection (including configuration) into the constructor and exposing the Provider at import time via ImportConnector, received through ProviderResource::fetch().
getProvider() to ImportConnector.ThrottledConnector to rate-limit all resources owned by a provider with a shared throttle.Throttle support for async imports only.Porter v5 introduces asynchronous imports and complete strict type safety (excluding union types and generics).
Connector::fetch string source parameter with new DataSource interface.ConnectionContext from Connector interface.SingleRecordResource interface that resources must implement to be used with Porter::importOne().RecoverableConnectorException with RecoverableException interface.ConnectorOptions and EncapsulatedOptions.CacheKeyGenerator.ForeignResourceException to Porter's namespace.Porter v4 fixes all known design flaws (#31, #43) and critically re-evaluates every part of Porter's design. All base classes have been discarded (AbstractProvider, AbstractResource), moving their code within Porter, relying solely on interfaces instead. This frees up the inheritance chain for applications to use as they wish, making it much easier to integrate Porter into existing projects.
The new design is much simpler, removing the redundant fetch() method from Provider and removing the redundant and confusing EncapsulatedOptions parameters from all fetch() methods. There is no longer any need to figure out how to merge different sets of options coming from different parts of the application because there is only one source of truth for connector options now, and they live within the connector itself, because it has a 1:1 relationship with its options.
Porter v4 is super slim; we no longer bundle any unnecessary dependencies such as connectors you don't need. connectors/http has also dropped URL building support and all the associated dependencies because it is not the job of the connector to build URLs; do this in providers if needed, by whatever mechanism best suits its needs.
In development, on and off, for a little over a year, I sincerely hope you find this new version of Porter useful and easier to use than ever before.
AbstractResource. (#35)ContainerInterface which must contain the providers. (#38)connectors/http and connectors/soap must be required manually if needed. (#39)Connector to receive ConnectionContext as its first parameter. Context includes a retry() method to provide preconfigured, immutable durability features to the connector. (#42)Connector implementations no longer have to extend CachingConnector to provide caching facilities: all connectors can be decorated with CachingConnector with no prior knowledge of the existence of such facility. This completely removes the burden on implementations to be aware of caching concerns. (#44)AbstractProvider. (#41)EncapsulatedOptions parameter from Connector::fetch() method. (#48)callable to FetchExceptionHandler to fix #43. (#50)RecordCollections to return arrays. Previously, the documentation claimed collections were iterators of arrays but the software did not enforce this; now it does. (#52)Net namespace and moved Connector implementations to separate repositories.CacheKeyGenerator interface. (@markchalloner)