Self-describing content-addressed identifiers for distributed systems
Self-describing content-addressed identifiers for distributed systems
CID is a format for referencing content in distributed information systems, like IPFS. It leverages content addressing, cryptographic hashing, and self-describing formats. It is the core identifier used by IPFS and IPLD. It uses a multicodec to indicate its version, making it fully self describing.
You can read an in-depth discussion on why this format was needed in IPFS here: https://github.com/ipfs/specs/issues/130 (first post reproduced here)
A CID is a self-describing content-addressed identifier. It uses cryptographic hashes to achieve content addressing. It uses several multiformats to achieve flexible self-description, namely:
Concretely, it's a typed content address: a tuple of (content-type, content-address)
.
Current version: CIDv1
CIDv1 is a binary format composed of unsigned varints prefixing a hash digest to form a self-describing "content address":
<cidv1> ::= <CIDv1-multicodec><content-type-multicodec><content-multihash>
# or, expanded:
<cidv1> ::= <`0x01`, the code for `CIDv1`><another code from `ipld` entries in multicodec table that signals content type of data being addressed><multihash of addressed data>
Where
<multicodec-cidv1>
is a multicodec representing the version of CID, here for upgradability purposes.<multicodec-content-type>
is a multicodec code representing the content type or format of the data being addressed.<multihash-content-address>
is a multihash value, which uses a registry of hash function abbreviations to prefix a cryptographic hash of the content being addressed, thus making it self-describing.Since CIDs have many applications outside of binary-only contexts, a given CID may need to be base-encoded multiple ways for different consumers or for different transports. In such applications, CIDs are often expressed as a Unicode string rather than a bytestring, which adds a single code-point prefix. In these contexts, then, the full string form is:
<cidv1> ::= <multibase-codec><multibase-encoding(<CIDv1-multicodec><multicodec><multihash>)>
Where
<multibase-codec>
is a multibase prefix (1 Unicode code point in length) that renders the base-encoded unicode string following it self-describing for simpler conversion back to binary.It is often advantageous to translate a CID, which is already modular and self-describing, into a human-readable expansion of its self-describing parts, for purposes such as debugging, unit testing, and documentation. We can easily transform a Stringified CID to a "Human-Readable CID" by translating and segmenting its constituent parts as follows:
<hr-cid> ::= <hr-mbc> "-" <hr-cid-mc> "-" <hr-mc> "-" <hr-mh>
Where each sub-component is replaced with its own human-readable form from the relevant registry:
<hr-mbc>
is the name of the multibase code (eg z
--> base58btc
)<hr-cid-mc>
is the name of the multicodec for the version of CID used (eg 0x01
--> cidv1
)<hr-mc>
is the name of the multicodec code (eg 0x51
--> cbor
)<hr-mh>
is the name of the multihash code (eg sha2-256-256
) followed by a final dash and the hash itself -abcdef0123456789...
)For example:
# example CID
zb2rhe5P4gXftAwvA4eXQ5HJwsER2owDyS9sKaQRRVQPn93bA
# corresponding human readable CID
base58btc - cidv1 - raw - sha2-256-256-6e6ff7950a36187a801613426e858dce686cd7d7e3c0fc42ee0330072d245c95
See: https://cid.ipfs.io/#zb2rhe5P4gXftAwvA4eXQ5HJwsER2owDyS9sKaQRRVQPn93bA
CIDs design takes into account many difficult tradeoffs encountered while building IPFS. These are mostly coming from the multiformats project.
CIDv0 is a backwards-compatible version, where:
multibase
of the string representation is always base58btc
and implicit (not written)multicodec
is always dag-pb
and implicit (not written)cid-version
is always cidv0
and implicit (not written)multihash
is written as is but is always a full (length 32) sha256 hash.cidv0 ::= <multihash-content-address>
See the section: How does it work?
<cidv1> ::= <multibase-prefix><multicodec-cidv1><multicodec-content-type><multihash-content-address>
To decode a CID, follow the following algorithm:
Qm...
, it's a CIDv0. Decode it as base58btc and continue to step 2.cid
):
[0x12, 0x20, ...]
, it's a CIDv0.
cid
.N
be the first varint in cid
. This is the CID's version.
N == 0x01
(CIDv1):
cid
cid
(after the second varint).N == 0x02
(CIDv2), or N == 0x03
(CIDv3), the CID version is reserved.N
is equal to some other multicodec, the CID is malformed.Q. I have questions on multicodec, multibase, or multihash.
Please check their repositories: multicodec, multibase, multihash.
Q. Why does CID exist?
We were using base58btc encoded multihashes in IPFS, and then we needed to switch formats to IPLD. We struggled with lots of problems of addressing data with different formats until we created CIDs. You can read the history of this format here: https://github.com/ipfs/specs/issues/130
Q. Is the use of multicodec similar to file extensions?
Yes, kind of! like a file extension, the multicodec identifier establishes the format of the data. Unlike file extensions, these are in the middle of the identifier and not meant to be changed by users. There is also a short table of supported formats.
Q. What formats (multicodec codes) does CID support?
We are figuring this out at this time.
It will likely be a subset of multicodecs for secure distributed systems.
So far, we want to address IPFS's UnixFS and raw blocks (dag-pb
, raw
), IPNS's libp2p-key
, and IPLD's dag-json
/dag-cbor
formats.
Q. What is the process for updating CID specification (e.g., adding a new version)?
CIDs are a well established standard. IPFS uses CIDs for content-addressing and IPNS. Making changes to such key protocol requires a careful review which should include feedback from implementers and stakeholders across ecosystem.
Due to this, changes to CID specification MUST be submitted as an improvement proposal to ipfs/specs repository (PR with IPIP document), and follow the IPIP process described there.
Captain: @jbenet.
Contributions welcome. Please check out the issues.
Check out our contributing document for more information on how we work, and about contributing in general. Please be aware that all interactions related to IPLD are subject to the IPFS Code of Conduct.
Small note: If editing the README, please conform to the standard-readme specification.
This repository is only for documents. These are licensed under a CC-BY 3.0 Unported License © 2016 Protocol Labs Inc.