Merge branch 'main' into web-redirects-clarifications

Author: lidel

KEYSTORE.md

@@ -1,258 +1,3 @@
-# ![](https://img.shields.io/badge/status-wip-orange.svg?style=flat-square) Keystore
+# Keystore
 
-**Authors(s):**
-- [whyrusleeping](github.com/whyrusleeping)
-- [Hector Sanjuan](github.com/hsanjuan)
-
-* * *
-
-**Abstract**
-
-This spec provides definitions and operations for the keystore feature in IPFS.
-
-# Table of Contents
-
-TODO
-
-## Goals:
-
-To have a secure, simple and user-friendly way of storing and managing keys
-for use by ipfs. As well as the ability to share these keys, encrypt, decrypt,
-sign and verify data.
-
-## Planned Implementation
-
-### Key storage
-
-Storage layout and format is defined in the [`REPO_FS`](REPO_FS.md) part of the spec.
-
-### Interface
-
-#### ipfs key
-
-```
-
-    ipfs key - Interact with ipfs keypairs
-
-SUBCOMMANDS:
-
-    ipfs key gen                  - Generates a new named ipfs keypair
-    ipfs key list                 - Lists out all local keypairs
-    ipfs key info <key>           - Get information about a given key
-    ipfs key rm     <key>             - Delete a given key from your keystore
-    ipfs key rename <key> <name>  - Renames a given key
-    ipfs key show <key>               - Print out a given key
-
-    ipfs key send <key> <peer>    - Shares a specified private key with the given peer
-
-    Use 'ipfs key <subcmd> --help' for more information about each command.
-
-DESCRIPTION:
-
-    'ipfs key' is a command used to manage ipfs keypairs.
-
-```
-
-#### ipfs crypt
-
-```
-    ipfs crypt - Perform cryptographic operations using ipfs keypairs
-
-SUBCOMMANDS:
-
-    ipfs crypt sign <data>          - Generates a signature for the given data with a specified key
-    ipfs crypt verify <data> <sig>  - Verify that the given data and signature match
-    ipfs crypt encrypt <data>       - Encrypt the given data
-    ipfs crypt decrypt <data>       - Decrypt the given data
-
-DESCRIPTION:
-
-    `ipfs crypt` is a command used to perform various cryptographic operations
-    using ipfs keypairs, including: signing, verifying, encrypting and decrypting.
-```
-
-#### Some subcommands:
-
-##### ipfs key Gen
-
-
-```
-    ipfs key gen <name> - Generate a new ipfs keypair
-
-OPTIONS:
-    -t, -type       string         - Specify the type and size of key to generate (i.e. rsa)
-    -s. -size       int            - Size of the key to generate
-    -p, -passphrase string         - Passphrase for encrypting the private key on disk
-
-DESCRIPTION:
-
-    'ipfs key gen' is a command used to generate new keypairs.
-```
-
-* * *
-
-##### Key Send
-```
-
-    ipfs key send <key> <peer> - Send a keypair to a given peer
-
-OPTIONS:
-
-    -y, -yes        bool        - Yes to the prompt
-
-DESCRIPTION:
-
-    'ipfs key send' is a command used to share keypairs with other trusted users.
-
-    It will first look up the peer specified and print out their information and
-    prompt the user "are you sure? [y/n]" before sending the keypair. The target
-    peer must be online and dialable in order for the key to be sent.
-
-    Note: while it is still managed through the keystore, ipfs will prevent you from
-            sharing your nodes private key with anyone else.
-
-```
-
-##### Comments:
-
-Ensure that the user knows the implications of sending a key.
-
-* * *
-
-##### Crypt Encrypt
-```
-
-    ipfs crypt encrypt <data> - Encrypt the given data with a specified key
-
-ARGUMENTS:
-
-    data                        - The filename of the data to be encrypted ("-" for stdin)
-
-OPTIONS:
-
-    -k, -key        string        - The name of the key to use for encryption (default: localkey)
-    -o, -output     string        - The name of the output file (default: stdout)
-    -c, -cipher     string        - The cipher to use for the operation
-    -m, -mode       string        - The block cipher mode to use for the operation
-
-DESCRIPTION:
-
-    'ipfs crypt encrypt' is a command used to encypt data so that only holders of a certain
-    key can read it.
-
-```
-
-##### Comments:
-
-This should probably just operate on raw data and not on DAGs.
-
-* * *
-
-##### Other Interface Changes
-
-We will also need to make additions to support keys in other commands, these changes are as follows:
-
-- `ipfs add`
-    - Support for a `-encrypt-key` option, for block encrypting the file being added with the key
-        - also adds an 'encrypted' node above the root unixfs node
-    - Support for a `-sign-key` option to attach a signature node above the root unixfs node
-
-- `ipfs block put`
-    - Support for a `-encrypt-key` option, for encrypting the block before hashing and storing
-
-- `ipfs object put`
-    - Support for a `-encrypt-key` option, for encrypting the object before hashing and storing
-
-- `ipfs name publish`
-    - Support for a `-key` option to select which keyspace to publish to
-
-### Code changes and additions
-
-This sections outlines code organization around this feature.
-
-#### Keystore package
-
-The fsrepo carries a `keystore` that can be used to load/store keys. The keystore is implemented following this interface:
-
-```go
-// Keystore provides a key management interface
-type Keystore interface {
-	// Has returns whether or not a key exist in the Keystore
-	Has(string) (bool, error)
-	// Put stores a key in the Keystore, if a key with the same name already exists, returns ErrKeyExists
-	Put(string, ci.PrivKey) error
-	// Get retrieves a key from the Keystore if it exists, and returns ErrNoSuchKey
-	// otherwise.
-	Get(string) (ci.PrivKey, error)
-	// Delete removes a key from the Keystore
-	Delete(string) error
-	// List returns a list of key identifier
-	List() ([]string, error)
-}
-```
-
-Note: Never store passwords as strings, strings cannot be zeroed out after they are used.
-using a byte array allows you to write zeroes over the memory so that the users password
-does not linger in memory.
-
-#### Unixfs
-
-- new node types, 'encrypted' and 'signed', probably shouldn't be in unixfs, just understood by it
-- if new node types are not unixfs nodes, special consideration must be given to the interop
-
-- DagReader needs to be able to access keystore to seamlessly stream encrypted data we have keys for
-    - also needs to be able to verify signatures
-
-#### Importer
-
-- DagBuilderHelper needs to be able to encrypt blocks
-    - Dag Nodes should be generated like normal, then encrypted, and their parents should
-        link to the hash of the encrypted node
-- DagBuilderParams should have extra parameters to accommodate creating a DBH that encrypts the blocks
-
-#### New 'Encrypt' package
-
-Should contain code for crypto operations on dags.
-
-Encryption of dags should work by first generating a symmetric key, and using
-that key to encrypt all the data. That key should then be encrypted with the
-public key chosen and stored in the Encrypted DAG structure.
-
-Note: One option is to simply add it to the key interface.
-
-### Structures
-Some tentative mockups (in json) of the new DAG structures for signing and encrypting
-
-Signed DAG:
-```
-{
-    "Links" : [
-        {
-            "Name":"@content",
-            "Hash":"QmTheContent",
-        }
-    ],
-    "Data": protobuf{
-        "Type":"Signed DAG",
-        "Signature": "thesignature",
-        "PubKeyID": "QmPubKeyHash",
-    }
-}
-```
-
-Encrypted DAG:
-```
-{
-    "Links" : [
-        {
-            "Name":"@content",
-            "Hash":"QmRawEncryptedDag",
-        }
-    ],
-    "Data": protobuf{
-        "Type":"Encrypted DAG",
-        "PubKeyID": "QmPubKeyHash",
-        "Key": "ephemeral symmetric key, encrypted with public key",
-    }
-}
-```
+Moved to https://github.com/ipfs/kubo/blob/master/docs/specifications/keystore.md

Makefile

@@ -1,4 +1,4 @@
-SPEC_GENERATOR_VER=1.3.2
+SPEC_GENERATOR_VER=1.4.0
 
 .PHONY: install
 

REPO.md

@@ -1,128 +1,3 @@
-# ![](https://img.shields.io/badge/status-wip-orange.svg?style=flat-square) IPFS Repo Spec
+# Repository
 
-**Author(s)**:
-- [Juan Benet](github.com/jbenet)
-
-* * *
-
-**Abstract**
-
-This spec defines an IPFS Repo, its contents, and its interface. It does not specify how the repo data is actually stored, as that is done via swappable implementations.
-
-# Table of Contents
-
-TODO
-
-## Definition
-
-A `repo` is the storage repository of an IPFS node. It is the subsystem that
-actually stores the data IPFS nodes use. All IPFS objects are stored
-in a repo (similar to git).
-
-There are many possible repo implementations, depending on the storage media
-used. Most commonly, IPFS nodes use an [fs-repo](REPO_FS.md).
-
-Repo Implementations:
-- [fs-repo](REPO_FS.md) - stored in the os filesystem
-- mem-repo - stored in process memory
-- s3-repo - stored in amazon s3
-
-<center>
-  <img src="img/ipfs-repo.png" width="256" />
-</center>
-
-## Repo Contents
-
-The Repo stores a collection of [IPLD](https://github.com/ipld/specs#readme) objects that represent:
-
-- **config** - node configuration and settings
-- **datastore** - content stored locally, and indexing data
-- **keystore** - cryptographic keys, including node's identity
-- **hooks** - scripts to run at predefined times (not yet implemented)
-
-Note that the IPLD objects a repo stores are divided into:
-- **state** (system, control plane) used for the node's internal state
-- **content** (userland, data plane) which represent the user's cached and pinned data.
-
-Additionally, the repo state must determine the following. These need not be IPLD objects, though it is of course encouraged:
-
-- **version** - the repo version, required for safe migrations
-- **locks** - process semaphores for correct concurrent access
-- **datastore_spec** - array of mounting points and their properties
-
-Finally, the repo also stores the blocks with blobs containing binary data.
-
-![](/img/ipfs-repo-contents.png)
-
-### version
-
-Repo implementations may change over time, thus they MUST include a `version` recognizable across versions. Meaning that a tool MUST be able to read the `version` of a given repo type.
-
-For example, the `fs-repo` simply includes a `version` file with the version number. This way, the repo contents can evolve over time but the version remains readable the same way across versions.
-
-### datastore
-
-IPFS nodes store some IPLD objects locally. These are either (a) **state objects** required for local operation -- such as the `config` and `keys` -- or (b) **content objects** used to represent data locally available. **Content objects** are either _pinned_ (stored until they are unpinned) or _cached_ (stored until the next repo garbage collection).
-
-The name "datastore" comes from [go-datastore](https://github.com/jbenet/go-datastore), a library for swappable key-value stores. Like its name-sake, some repo implementations feature swappable datastores, for example:
-- an fs-repo with a leveldb datastore
-- an fs-repo with a boltdb datastore
-- an fs-repo with a union fs and leveldb datastore
-- an fs-repo with an s3 datastore
-- an s3-repo with a cached fs and s3 datastore
-
-This makes it easy to change properties or performance characteristics of a repo without an entirely new implementation.
-
-### keystore
-
-A Repo typically holds the keys a node has access to, for signing and for encryption.
-
-Details on operation and storage of the keystore can be found in [`REPO_FS.md`](REPO_FS.md) and [`KEYSTORE.md`](KEYSTORE.md).
-
-### config (state)
-
-The node's `config` (configuration) is a tree of variables, used to configure various aspects of operation. For example:
-- the set of bootstrap peers IPFS uses to connect to the network
-- the Swarm, API, and Gateway network listen addresses
-- the Datastore configuration regarding the construction and operation of the on-disk storage system.
-
-There is a set of properties, which are mandatory for the repo usage. Those are `Addresses`, `Discovery`, `Bootstrap`, `Identity`, `Datastore` and `Keychain`.
-
-It is recommended that `config` files avoid identifying information, so that they may be re-shared across multiple nodes.
-
-**CHANGES**: today, implementations like js-ipfs and go-ipfs store the peer-id and private key directly in the config. These will be removed and moved out.
-
-### locks
-
-IPFS implementations may use multiple processes, or may disallow multiple processes from using the same repo simultaneously. Others may disallow using the same repo but may allow sharing _datastores_ simultaneously. This synchronization is accomplished via _locks_.
-
-All repos contain the following standard locks:
-- `repo.lock` - prevents concurrent access to the repo. Must be held to _read_ or _write_.
-
-### datastore_spec
-
-This file is created according to the Datastore configuration specified in the `config` file. It contains an array with all the mounting points that the repo is using, as well as its properties. This way, the `datastore_spec` file must have the same mounting points as defined in the Datastore configuration.
-
-It is important pointing out that the `Datastore` in config must have a `Spec` property, which defines the structure of the ipfs datastore. It is a composable structure, where each datastore is represented by a json object.
-
-### hooks (TODO)
-
-Like git, IPFS nodes will allow `hooks`, a set of user configurable scripts to run at predefined moments in IPFS operations. This makes it easy to customize the behavior of IPFS nodes without changing the implementations themselves.
-
-## Notes
-
-#### A Repo uniquely identifies an IPFS Node
-
-A repository uniquely identifies a node. Running two different IPFS programs with identical repositories -- and thus identical identities -- WILL cause problems.
-
-Datastores MAY be shared -- with proper synchronization -- though note that sharing datastore access MAY erode privacy.
-
-#### Repo implementation changes MUST include migrations
-
-**DO NOT BREAK USERS' DATA.** This is critical. Thus, any changes to a repo's implementation **MUST** be accompanied by a **SAFE** migration tool.
-
-See https://github.com/jbenet/go-ipfs/issues/537 and https://github.com/jbenet/random-ideas/issues/33
-
-#### Repo Versioning
-
-A repo version is a single incrementing integer. All versions are considered non-compatible. Repos of different versions MUST be run through the appropriate migration tools before use.
+Moved to https://github.com/ipfs/kubo/blob/master/docs/specifications/repository.md