a social algorithm for computing cred
Critical bug patch to solve discord server ids passing the quintillion mark
3c76945ef1c7b4d892602764d6a1ec70aebfecbc Discord message IDs have passed the quintillion mark and this has led to an infinite loop in the codebase due to a string comparison on numeric IDs. This PR proposes a simple fix of casting the IDs to numbers before doing the comparison. (#3361)
Critical bug patch to solve [Error: ENOENT: no such file or directory, open '/home/runner/work/org/repo/output/graphs/sourcecred/discourse/graph'] during the graph step of the Generate Cred Action.
24f8be32f4fbbfb15bee8d299d322b988af4e935 Bugfix: explicit directory creation for plugin outputs (#3348)
Note: critical bug involving directory creation patched in 0.11.1
Wowee! As our core developers move on to other opportunities, this will be the last 1st-party update for the foreseeable future. But it is an EXCITING one. And it should be a safe, zero-effort upgrade (except for a small breaking change in our developer library).
CredEquate is a new scoring algorithm based on simple algebra. It can be used instead of, or in parallel with, our existing pagerank-based algorithm CredRank. As of this release we consider CredEquate to be sufficiently production-ready. So far, it supports implementations in the Discord and Github plugins, and it comes with a broad set of advantages compared to CredRank. To learn more about the comparison, and to learn how to try it out, go to https://sourcecred.io/docs/guides/core-algorithm
Setting up a SourceCred instance has long been a very manual and technical process that requires local git managment, terminal commands, and hand-editing JSON. In this version, a new Configuration Editor GUI makes a major step towards reducing the barriers to using SourceCred.
Now, you can edit the sourcecred.json file (which is especially important when using CredEquate) in a local-hosted admin portal by navigating to the new menu option "Edit Plugins." There, your sourcecred.json file will be loaded into a webform, where you can edit it and then click "Save" to save the changes back to disk. For more information on how to configure the CredEquate plugins section of this page, go to https://sourcecred.io/docs/guides/core-algorithm
Additionally, we have created an experimental no-clone, no-terminal way of using this GUI with a static remote-hosted site. This experience involves inputting a GitHub API key so that the page can read and write to GitHub instead of to a local disk. The API key is only stored in your browser's memory, and only used to make client-to-GitHub API calls. There are potential security risks with this method, but the benefits are worth exploring. To try it out, go to https://cred.sourcecred.io/#/instance-config in Google Chrome (make sure you trust your Chrome extensions).
Our dream is that one day, this config editing experience will be available for all of the JSON configuration files used by a SourceCred instance. We hope that others will be inspired to build this dream out using the foundation that we have laid here.
FINALLY, a feature that makes it easy to distribute tokens using SourceCred! While this is not a direct on-chain integration, it is a HUGE improvement to treasury workflows. The basic outline is:
For detailed instructions, go to https://sourcecred.io/docs/guides/csv-grain
Discord has always been a major scalability limit. Some Discord servers are just too big to reasonable load and process all their historical data. Now, you can add an attribute to the config/plugins/sourcecred/discord/config.json file to set a beginning date, and the plugin will only load data after that date. For example, to only load Discord data after December 1st, 2021, add "beginningDate": "2021/12/1"
In 0.10.1/0.10.2 we renamed the github writing class from GithubStorage to WritableGithubStorage, which was a breaking change for 3rd-party code that used GithubStorage. In this version, we have renamed the export back to GithubStorage. We apologize to developers who have already updated to 0.10.2 for the churn, but we are glad to say that developers who hadn't updated yet will not experience a breaking change as they move to 0.11.0.
Our library now exports the new WriteInstance module, which allows for easy reading and writing of SourceCred inputs and outputs. It's constructor accepts an object that implements our WritableDataStorage interface (get and set methods). The module also includes a factory for creating a GitHub-compatible WriteInstance like so: sc.instance.writeInstance.getGithubWriteInstance(myApiKey, myRepo, myBranch)
cb45917cd2aa06404754367273e90d010928f867 Create a GUI config editor for RawInstanceConfig (#3343) 0506e483a68e742d55ef1a5a02ac0ed3628cc243 Addng WriteInstance into LocalInstance inheritance hierarchy (#3325) 6f66bcec7591a0145a3857766cdcbbe2d98d4267 Github action for generating docs (#3346) 6a4c808d72f5866220ce01c9d2ebbe61893f5a8a CredRank: add timestamp to MarkovProcessGraph nodes (#3345) 37144a0cb7a5f31a6d593eea6900413ff8d78fef take and pass args to storage in persist (#3339) 3939ea0a5d32ae9fea631f02f095cabd71c63bc1 fix backward compatibility for gh storage (#3340) 67af2540b38961913e458f0497872b1e1da20e70 Discord: fetch backwards and add a beginningDate config (#3329) 404b11cc92e648fbe2844f85a77650c1596cf5a4 Change the Go CLI to run contributions, and use hybrid instead of credrank (#3337) ac5d05df39bb13a9620b4f5ceb7dff29e3c2ae63 CredEquate: create MVP GitHub plugin (#3338) 68b9fbd835c95e016cb148def5099aa92f613255 Helper method for finding CredEquate contributions by key/subkey (#3330) e8950b3624b6fee976f1aed2f5cc1faf67e9baa6 Dependency cred support for CredEquate (#3328) b5298c04723fe49a163049757cb35b968a4a419a Get CSV grain integration production-ready (#3320) 86189db34b3635b975ce37595b5c2478b99f9f24 Add hybrid CLI that calls both credrank and credequate (#3322) 77dde287650ee0aa80a11bca54f978f330b57436 Add static method for merging CredGrainViews (#3323) 7e4518db2446418a04e1f7a05f2af22230a8085b Explorer Home: Make Grain column name match configuration (#3318) 8bafdf9c89bb4bd3957597eaf6ff9f583808eebc Support for Gnosis in Csv Grain Integration (#3321) 2f5e6ef3a776a83814117ef80a50a8a5e521fa3e CredEquate support in Load CLI (#3319) 5e8e86ed9c85fdcbd4d1f849c63c52e7dc0c4545 Nav & Sidebar Adjustments (#3266)
Full Changelog: https://github.com/sourcecred/sourcecred/compare/sourcecred-v0.10.3...sourcecred-v0.11.0
Windows users (and some other environments) have not been able to import or set up sourcecred properly due to some autogenerated assets that had file names exceeding the max for those systems. This has been fixed.
We will document this more after some initial internal testing. If you want to try it out for yourself in the meantime, see the test plan of https://github.com/sourcecred/sourcecred/pull/3299, and if you want to read more about what this is and why we're building it, see the original design spec.
966c48f1a1f2252b83f8e495be37228b5734f695 CredEquate: Fix circular dependencies (#3312)
0fe70ab0f3e9d50cc02b30c676e11d2e8f7df1f5 Instructional sharness snapshot error message (#3309)
505d5da1a76bdaa285ede27255ab19e3e45177f4 CredEquate: expose core code in library (#3310)
d77bf0eda1cc6e0863b4b3f113b8f4e7e77c2a9f Mergeable CredEquate Design Doc for recordkeeping (#3304)
f735638cc5f9cc809862cc8c5ea331b70cd779b7 Change the grain API to take a credGrainview instead of a credGraph (#3303)
8b9871100d6dd3d2ff2c3a8ed2228bfe8c5a4a21 CredEquate: create credequate CLI/API (#3299)
e6c6147126e535d6149f49cd734715464f9784d8 CredEquate: add Discord plugin and Contributions CLI/API (#3295)
94b9e2a4d6442276b0db55d59cddc2ef5e9b7c9f CredEquate core algorithm, experimental alternative to CredRank (#3273)
809f6fc6365e489768c2365d350c9c92b5ddaf3e Ignore contributions from huxbot (#3289)
b8110a1c7126b03c6f24c964bcc1e6e286b04993 Author Pre-Audit Merkle Distribution Contract (#3148)
b5709e6429db7cb7814e1ffe19c96d77d93ddc1e Feature: Disable balance accounting in the ledger (#3192)
86517338e94f5d1a3ab20b48e6938c6ff8248658 combo bugfix: spacer option renamed to space (#3280)
47af83f878316b023459fbcd6961bb6969b60fcc Convert snapshot file name serialization to hashes (#3279)
This patch only concern 3rd-party developers or users of 3rd-party tools. This changes our API's sc.ledger.storage.GithubStorage export to a newer class of the same name. Because the interface is unchanged for users of the LedgerManager, scripts using that class will work after changing GithubStorage to WritableGithubStorage. If you use the GithubStorage class directly without a LedgerManager, then you should note that read and write have been changed to get and set and take different parameters.
fff0d2a7c38829c881936f84ed34e0cd80f8d961 Change GithubStorage in API to the newer version (#3277) 9217fc1db9510b5616e8e2ed1a2929b009f2994a v0.10.1 docs generation (#3276)
If you are not a 3rd-party developer or user of 3rd-party tools for managing your ledger, there are no breaking changes that concern you and this is a stable build.
This version changes the GithubStorage input required by the LedgerManager class in our JS library. Unfortunately, we also forgot to switch the GithubStorage provided in the API to the new one. So if you use the LedgerManager class, this is not a viable build. The correct WritableGithubStorage class is provided in 0.10.2, however.
Neo4j is a graph database with its own query language and visual explorer. We now support exporting the SourceCred graph to Neo4j after generating it. Running yarn sourcecred analysis -n will generate CSV files in a output/neo4j folder and log pre-built neo4j-admin commands that can be used with Neo4j's admin CLI to import the CSV files into a Neo4j databse. The fastest way to get started is by downloading Neo4j Desktop and Java 11 and setting up local Neo4j db.
Note, you can also add yarn sourcecred analysis -n to your instance's GitHub Action configuration to automatically generate these files in your gh-pages branch.
We have primitive API docs! We'll be working on making sure they're all accurate and providing entry points to know where to start. In the mean time, enjoy our progress so far: https://api.sourcecred.io/sourcecred/
If you've been using our getNetworkReadInstance library functionality to pull SourceCred data from an instance, there is now an improved getRawGithubReadInstance interface that does the same thing without requiring you to find the raw.githubusercontent base URL. Just pass in the github org name, repo name, and branch name like this:
instance = sc.instance.readInstance.getRawGithubReadInstance("sourcecred","cred","gh-pages")
Don't know what this means but interested? You can find info about our ReadOnlyInstance interface here: https://api.sourcecred.io/sourcecred/#readonlyinstance https://observablehq.com/@sourcecred/api-tutorial
2380c7ba8f849766463ab2f8762c911acd91a7d3 Add Needs Triage label to feature request template (#3274) 3c80039de006e6c0ede700a706c4a4b8c61e1e9d Support Pagination and Filter-by-accont for Grains Accounts (#3267) 0e03eb150e81aa4557c6daeccd92ef26d2be9455 Analysis API: Neo4j csv files (#3258) 71387ef9bc1a48bcc527624ab976936891c6cad9 Util: improve performance of batch (#3261) a61df47cfabb830882b6d8c232a529df81e202f9 Add support for sorting by Active accounts. (#3263) 9f356fac509248014ecc19148c144dd50f42c14a Fix sorting in accounts preview (#3262) 24cf73a134f4f7f15a50001b7e5401f7370ea8dd Util: add batchArray and batchIterator methods (#3259) 5459e0cfe884d9db9abced58a1cca3cacdb3f5d0 Better error message for dependency id missing in ledger (#3255) 362b72ecb80b6bea589ea270cf8033523d090d0d GitHub storage conforms data storage (#3209) 5f88abefa482edc38e1dca325910723a37ed1d92 Set up GH Pages config for api docs (#3252) a40646af181cbf4f40e545d3b74e3e78ef48b64f Add computer-generated API documentation (#3251) 2c913044ebf1a7eabfbbfa10af6f37163894a575 Add RawGithubReadInstance wrapper for improved UX (#3250) 584d45f6656d15ca2d8c31657ea06acdbc3d3c41 Using Node as default jest's test environment & Utilizing jest's docblock (#3242) dfbade6a6a47334331599376b773da63d607b8f6 ADD Postable Origin Storage (#3235) 7f665757c73c3a35bcf1bdef691947b9be1c6df1 Add Payout Address Text Box for Identities (#3233) b95f7fbafce8fb7757db96debb1e8bd2ab3e9eaa Utilize an Opaque type to key on currencies in the Ledger (#3231) eb2600c14494d98604e816b8839acc6e329dbc05 Fix colSpan so row takes up full width of table (#3238) 261e48daca7b05af88efd7cd7ff8b8b57a97df1a ADD Cred per contributor for each period. (#3236)
The below Grain Config format change is breaking for many instances. Fortunately, we included an updater. To use:
yarn && yarn sourcecred update
data/updater.txt
Before, the only way to fully exclude channels from the load and graph steps was to wrestle the bot's permissions. Now, channels with a weight of 0 (explicitly or using defaultWeight) will not be loaded. This should improve performance significantly for instances with zero-weighted channels. This may affect cred scores slightly.
Now the Discord plugin excludes messages that don't have any positively weighted reactions attached. This will improve performance, and may also affect cred scores slightly.
Months ago, we introduced the option to configure your grain distributions using an array format in the attribute allocationPolicies. In this release, we are deprecating the old format in favor of the new. See the Breaking Changes note above to easily reformat your existing grain.json
Now, you can set a weight for a discord channel category, which will be used if a channel's weight is not explicitly set. To do so, simply add the category's ID into the same channelWeightConfig.weights array that you use to configure channel weights.
Many communities have salaried employees, which can complicate their use of SourceCred since they don't want people double-dipping. Now, you can pay salaried employees directly through SourceCred by adding a SPECIAL distribution for that employee and adding them to the new exclusions attribute on the RECENT distribution. This allows SourceCred's BALANCED policy to do smarter accounting and even pay salaried employees if their contributions exceed their salary. This also improves transparency in the community.
Example grain.json
{
"maxSimultaneousDistributions": 100,
"allocationPolicies": [
{
"policyType": "SPECIAL",
"budget": 100,
"memo": "Ongoing salary for Garnet's full-time community organizing contributions",
"recipient": "6wyPOVWLmZ2b35eCmOrz1w"
},
{
"policyType": "RECENT",
"budget": 100,
"discount": 0.16,
"exclusions": ["6wyPOVWLmZ2b35eCmOrz1w"]
},
{
"policyType": "BALANCED",
"budget": 100,
"numIntervalsLookback": 0
}
]
}
For our 3rd-party developers out there, our library now has some extra goodies included when importing into a node environment.
sc.api.instance.LocalInstance("path/to/instance/root") streamlines reading and writing instance configurations and outputs on local disk.graph api with ease.1c31ba102fd8efe1fcf8478669e1efd549f53e6f Bugfix: use discord user id if the name is all special characters (#3223) b2bfd167b418688eb383afd8064a09000887b86e Support Sorting in Current Balance and Earned Column. (#3227) b472cc9fdc7d4d7482871d1c49b33bbac0794b60 Deprecate old GrainConfig attributes + add updater (#3210) 852294fadb8bab542850e7dbf8d978edf00496b1 API: create extended library for node (#3214) e905f331acfd758e005790e2ce84870aa0b6689c Discord: Simplify default graph + simplifyGraph config (#3203) 00a2f5929f315622eeb4caa1a09f421048675b06 Add Spacing to stringify output. (#3202) 29b4c69e4ed3c4a565456173bfef19e406405085 Fix color property (#3211) 4f8aba2001a84c2fb99de3c74a87034da497db46 Add raw GrainConfig parsers (#3206) bcdb96822b5393898fc7763bb3e0940052b201cd Combo: Add fmap as a parser class method (#3207) 55adb2945959a87de3a492ce950473a30449e54e Recent Policy: add exclusions list of identities (#3204) 4cc0484cc24a1e6c092e50e565c9c2ba9527b959 Initial Cleanup of UI color palette (#3197) 90c7efcf6abf1a8091efbbbf4b25cb194901ea7b Discord: Channel Category weight to use inheritance instead of multiply (#3200) 04abbb8e936a4d2302ed0ce31c22a462df4f697a - now accounts update when update name (#3198) 46706bebdfac55f974b2c2974809d16c6f8e53a3 Discord: Skip channels with a weight of zero (#3199) 0ea3d8a242b0af0c8219f5e16b93a13143f8fdd5 Discord: add support for Channel Category weights (#3195) 8ecc7798953c55115e52f31fb608456cc3ecd15e support delimiter for Discord's guildid (#3194) 435f6c32d5b993e986c6eee8d072ceac91593544 Implement Grain Integration functionality in the CLI (#3085)
This is primarily a patch for the build bugs that broke 0.9.2, though it does also contain some minor UX improvements on our Ledger History and Transfer Grain UI pages.
There were a bunch of awesome new features added in 0.9.2, read about those here.
This release has bugs in the build. To use the awesome features from this release, please upgrade to 0.9.3.
Now, you can add text descriptors to certain attributes by adding // and any description you want afterwards at the end of the attribute key. Example:
"477551557723029514//general channel": 2,
The attributes that support this are:
The ExternalPlugin is used for loading a graph and optionally a declaration and identityProposals from either:
config/plugins/external/X folder.config.json file in the
instance's config/plugins/external/X folder with form:
{ "graphUrl": "https://www.myhost.com/path/to/graph", "declarationUrl": "https://www.myhost.com/path/to/declaration", "identityProposalsUrl": "https://www.myhost.com/path/to/proposals" }
Supported files for either method are:
graph.json/graph (required) - works whether or not it is
compressed using our librarydeclaration.json (optional) - if omitted, a default declaration with
minimal node/edge types is used, but also graphs don't have to adhere to the
declaration if they don't desire to be configured using our
Weight Configuration UI.identityProposals.json (optional) - if omitted, no identities are proposedThe External plugin can be used multiple times, because it simply uses the PluginId pattern "external/X" where X can be any name an instance admin chooses.
We are working on this tutorial for developers to work with the ExternalPlugin. It should be done by the end of August 2021. https://observablehq.com/@sourcecred/external-plugin-tutorial
If you want to get the non-compressed JSON of your credGraph, credGrainView, or other compressed output files, you can simply run:
yarn sourcecred graph --no-zip
yarn sourcecred credrank --no-zip
77e68983dd236aa35c6ac89ffa9d7a5ade412fd7 Add --no-zip flag to CLI, deprecate gzip suffix (#3162) 41d60442a4349c164995c0eef928921c90cbd1bf Add ConstructorPlugin for ExternalPlugin testing (#3154) 5270b582b86f8df9315070539a72abdad589ccac CredGrainView: Add advice to common error message (#3164) 14d7d07f7d34d1f36c94df22356dc1977848adff Add Cache version suffix in circleCI (#3159) 82378e5bef4484f31832cb5183c8094991d9bb37 Use combo parser instead of compat for PluginDeclaration (#3153) 4d468acde2c04e6675978f90659539692b7e4eac Bsodenkamp allocation identities refactor (#3115) c124d7b7e4e804da0fd009d0878e87e28c03c4a1 Revert "UI: Add Filtering to to Ledger View (#3122)" (#3147) d4df10d0b90da803176c5b660476d2eb757dd33f Create External Plugin that reads from disk or url (#3114) ea05c0033ea2767e229873378b05c724b9eb89e8 Add support for a text descriptor in number strings in config files (#3145) c347f5ad89434f7f0cd590507c08d0f74002da0f Set up Grain Integration support in Config API (#3079) 56aa0798173b5173ef284d4ac7392965a6f49310 UI: Add Filtering to to Ledger View (#3122) 3d3eb215f3aeaaf40cdf21f6562c0e37339f9396 Implement CSV Grain Integration Plugin (#3078) 9f11c25732c8e47f45e60f20b5ed53b7afd1c1e0 Update the root clean cmd to remove all node_modules (#3124) 57ecb9cf1004fc292dc8e8456a897bddbbf2aed4 Update Grain Integration to Accept file output (#3077) 1b723b10d9e76f05413e23f211f8ae6f9d426039 UI: Fixed react leak on identities page (#3104) 216e1c73114ba7d43deec700256ce3ac7f72ae0c converted immediate policy to CredGrainView (#3102)
If using the allocationPolicies attribute in the GrainConfig, numIntervalsLookback must now be added to BALANCED policies to filter the cred-per-grain that the policy takes into account to the last X weeks. To look at all-time, set numIntervalsLookback to 0. Example:
{
allocationPolicies: [
{
policyType: "BALANCED",
budget: 5000,
numIntervalsLookback: 0
}
],
maxSimultaneousDistributions: 100
}
Now the credrank CLI outputs a smaller aggregated data structure called CredGrainView, which the UI now loads instead of the much larger CredGraph. This results in a much faster UI load time.
The behavior of the Discord plugin has changed to include messages in the graph even after their author leaves or is kicked from the server. The author will no longer be included in the cred graph, but their messages will be, so anyone that their messages mentioned will keep that cred. This will likely affect Cred scores upon upgrading.
A new optional confidenceDampener number attribute can be used in the Discord reactionWeightConfig when applyAveraging is enabled to add "0 votes" to the averaging logic. This means that as more people react to a message, the total cred logarithmically approaches the role-weighted average. This calculates out along the lines of:
Confidence dampener = 1
1 people adding 1 emoji => 1/2 => 0.5
2 people adding 1 emoji => 2/3 => 0.67
3 people adding 1 emoji => 3/4 => 0.75
4 people adding 1 emoji => 4/5 => 0.80
5 people adding 1 emoji => 5/6 => 0.83
Confidence dampener = 2
1 people adding 1 emoji => 1/3 => 0.33
2 people adding 1 emoji => 2/4 => 0.5
3 people adding 1 emoji => 3/5 => 0.6
4 people adding 1 emoji => 4/6 => 0.67
5 people adding 1 emoji => 5/7 => 0.71
7cfd3b8277acc2b21f3df3b0515e51ac9e766bbd UI: Add identity id tooltip to Grain Accounts page (#3099) f47248066247d7b334f2dcfb9bb842735e036ceb Bsodenkamp balanced policy lookback 2678 (#3093) 60ffcfb5c372cebcae78f0ef07e8ac9c0b659208 Discord: add confidence dampener config to emoji weighting (#3098) a28fc773cb58ceeac789bddc96cd6e0f21907ed5 Add default labels to issue templates (#3095) 650985230832aa354a03da556e4dbd391defda43 Monorepo fix: put github templates in root (#3094) fc06d08f9b4263fabe9906b959a5f4087e2ef92b changed block under conditional check for author existencenow message node always added (#3092) 461d60b19553b37e79e5c9e9da9352de6ff07524 Add dev flag to LoginButton 3f7967fc9948b45db9cdb06bc03a2a311f75c870 Add web3 login button in AppBar 0722e53dea208fa0383bef52151d59a578ad5ee8 Setup Web3Context with web3modal and ethers.js f44ae63acd3b0139825f1c6c231482d5c0ade55d localinstance extends readinstance (code dedup) (#3088) ec13926d719e912d13ed580e87ec1ea6f3dd81fb Cosmetic updates to ledger events (#3084) d3fdf74fe6c1b3b8e70a6a94fb2c11e0f098bdad Cache CredGrainView for improved UI load times (#3086) 6fc431c2211f07b11c7497b8ad9d5655080c0e1a Better error for wrong repository name (#3083)