Index and query dependencies across your company's private repositories.
This started as an idea, and a quick implementation came together rather quick. After a quick refactor of the database support structures, adding support for this was much easier. This indexes a handful of fields (namely, url on sources and language and name for modules). This allows clients to quickly search for nodes in their graph.
New Endpoints
By adding support for field indexes, we were able to implement the following endpoints in REST and gRPC.
| Description | Endpoints |
|---|---|
| List languages deps.cloud has modules for. | grpc - depscloud.api.v1beta.LanguageService#List |
rest - /v1beta/languages |
|
cli - deps get languages |
|
| Search modules containing the provided name part. | grpc - depscloud.api.v1beta.ModuleService#Search |
rest - /v1beta/modules/search?like.name=xxx |
|
cli - deps search modules |
|
| Search sources containing the provided url part. | grpc - depscloud.api.v1beta.SourceService#Search |
rest - /v1beta/sources/search?like.url=xxx |
|
cli - deps search sources |
For better compatibility with the standard tooling provided by the community, we moved from the gogo to upstream protobuf. From a usage perspective, you shouldn't notice a difference. A big benefit to the project is that we should be able to leverage the reflection API instead of managing a static routing table in gateway.
We've upgraded the extractor process to use NodeJS 16. This is a pretty major upgrade considering the last version we were running on was NodeJS 12. For the most part, not much changed that impacted the extractor process. There were some minor modifications, but nothing that appeared to impact the overall functionality.
Alright. After a lot of work, we've finally made it to v0.3.0. A lot has changed. And I mean, a lot.
First, we've completely overhauled the API to better support our data structure. /v1beta simplifies queries for project dependencies, dependents, and associated trees. It also enables us to support many newer features like following library versions as we traverse the edges of the graph. v1alpha will still be available, but will only receive critical updates. In v0.4.0, it will be removed.
Second, the indexer process writes to the new v1beta endpoints. If you want to keep v1alpha up to date for some time after the upgrade, you can deploy two copies of the indexer. One should use v0.3.0 to populate the v1beta store. The other should use v0.2.33 to populate the v1alpha store. Once callers have upgraded to v1beta, you can clean up the v1alpha store (i.e the dts_graphdata table). Edit: You will also need a similar setup for the extractor process.
Third, the deps CLI has been updated to call the new v1beta endpoints. This means that if a user installs a v0.3.x version of the CLI before the server has been upgraded, all requests to the server will likely result in an unimplemented response. Using deps debug, you can quickly determine if there are inconsistencies between the client and server versions. To resolve this, you’ll need to uninstall the latest version of the depscloud CLI and install the v0.2.x specific versions.
# OSX
brew uninstall depscloud/tap/depscloud-cli
brew install depscloud/tap/[email protected]
# Linux
sudo apt-get install depscloud-cli=0.2.34
Then, once the server-processes are upgraded, you can re-install the latest version of the CLI.
# OSX
brew uninstall depscloud/tap/[email protected]
brew install depscloud/tap/depscloud-cli
# Linux
sudo apt-get install depscloud-cli
Finally, we’ve dropped support for 32-bit architectures. 32-bit architecture support has held back many much-needed upgrades such as our NodeJS major version.
Edit: Additionally, this release also features support for tracking helm chart dependencies and preliminary support for python dependencies via Pipfile (preferred) or requirements.txt (early).
v0.3.0