docs: consolidate and update docs (#3364)

* Consolidates docs under `/docs`
* Adds missing licenses, CoC, READMEs, etc
* Updates docs where information was incorrect

Author: achingbrain

CONTRIBUTING.md

@@ -0,0 +1,9 @@
+# Contributing guidelines
+
+IPFS as a project, including js-ipfs and all of its modules, follows the [standard IPFS Community contributing guidelines](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md).
+
+We also adhere to the [IPFS JavaScript Community contributing guidelines](https://github.com/ipfs/community/blob/master/CONTRIBUTING_JS.md) which provide additional information of how to collaborate and contribute in the JavaScript implementation of IPFS.
+
+We appreciate your time and attention for going over these. Please open an issue on [ipfs/community](https://github.com/ipfs/community) if you have any question.
+
+Thank you.

COPYRIGHT

@@ -0,0 +1,5 @@
+This project is transitioning from an MIT-only license to a dual MIT/Apache-2.0 license.
+Unless otherwise noted, all code contributed prior to 2019-11-21 and not contributed by
+a user listed in [this signoff issue](https://github.com/ipfs/js-ipfs/issues/2624) is
+licensed under MIT-only. All new contributions (and past contributions since 2019-11-21)
+are licensed under a dual MIT/Apache-2.0 license.

README.md

@@ -1,6 +1,11 @@
-# IPFS Architecture
+# IPFS Architecture <!-- omit in toc -->
 
-![](../img/architecture.png)
+## Table of Contents <!-- omit in toc -->
+
+- [Code Architecture and folder Structure](#code-architecture-and-folder-structure)
+  - [Source code](#source-code)
+
+![](./img/architecture.png)
 
 [Annotated version](https://user-images.githubusercontent.com/1211152/47606420-b6265780-da13-11e8-923b-b365a8534e0e.png)
 
@@ -14,7 +19,7 @@ What does this image explain?
 
 ## Code Architecture and folder Structure
 
-![](img/overview.png)
+![](./img/overview.png)
 
 ### Source code
 

packages/ipfs/docs/ARCHITECTURE.md renamed to docs/ARCHITECTURE.md

@@ -1,4 +1,11 @@
-# Using JS IPFS in the Browser
+# Using JS IPFS in the Browser <!-- omit in toc -->
+
+## Table of Contents <!-- omit in toc -->
+
+- [Limitations of the Browser Context](#limitations-of-the-browser-context)
+- [Addressing Limitations](#addressing-limitations)
+- [Best Practices](#best-practices)
+- [Code Examples](#code-examples)
 
 JS IPFS is the implementation of IPFS protocol in JavaScript. It can run on any
 evergreen browser, inside a service or web worker, browser extensions, Electron and in Node.js.
@@ -44,8 +51,8 @@ You can find detailed information about running js-ipfs [here](https://github.co
 
 ## Best Practices
 
-- Configure nodes for using self-hosted `*-star` signaling and transport service.  When in doubt, use WebSockets ones.
-- Run your own instance of `*-star` signaling service.
+- Configure nodes for using self-hosted `*-star` signalling and transport service.  When in doubt, use WebSockets ones.
+- Run your own instance of `*-star` signalling service.
   The default ones are under high load and should be used only for tests and development.
 - Make sure content added to js-ipfs running in the browser is persisted/cached somewhere on regular IPFS daemon
   - Manually `pin` or preload CIDs of interest with `refs -r` beforehand.

docs/BROWSERS.md

@@ -202,7 +202,7 @@ The base64 encoded protobuf describing (and containing) the nodes private key.
 
 ## `Keychain`
 
-We can customize the key management and criptographically protected messages by changing the Keychain options. Those options are used for generating the derived encryption key (`DEK`). The `DEK` object, along with the passPhrase, is the input to a PBKDF2 function.
+We can customize the key management and cryptographically protected messages by changing the Keychain options. Those options are used for generating the derived encryption key (`DEK`). The `DEK` object, along with the passPhrase, is the input to a PBKDF2 function.
 
 Default:
 ```json

packages/ipfs/docs/CLI.md renamed to docs/CLI.md

@@ -0,0 +1,23 @@
+# CORS <!-- omit in toc -->
+
+## Table of Contents <!-- omit in toc -->
+
+- [Overview](#overview)
+- [Configure CORS headers](#configure-cors-headers)
+
+## Overview
+
+Cross-origin Resource Sharing is a browser security mechanism that prevents unauthorized scripts from accessing resources from different domains.
+
+By default the HTTP RPC API of js-IPFS will cause any request sent from a CORS-respecting browser to fail.
+
+## Configure CORS headers
+
+You can configure your node to allow requests from other domains to proceed by setting the appropriate headers in the node config:
+
+```console
+$ jsipfs config --json API.HTTPHeaders.Access-Control-Allow-Origin  '["http://example.com"]'
+$ jsipfs config --json API.HTTPHeaders.Access-Control-Allow-Methods '["PUT", "POST", "GET"]'
+```
+
+Restart the daemon for the settings to take effect.

docs/CONFIG.md

@@ -1,8 +1,11 @@
-# Configuring Delegate Routers
+# Configuring Delegate Routers <!-- omit in toc -->
+
+- [What is it?](#what-is-it)
+- [How do I do it?](#how-do-i-do-it)
 
 ## What is it?
 
-Delegate routers peform tasks on behalf of nodes that may be missing functionality, so for example they may search the DHT for peers or content providers on behalf of IPFS implementations that do not have a DHT.
+Delegate routers perform tasks on behalf of nodes that may be missing functionality, so for example they may search the DHT for peers or content providers on behalf of IPFS implementations that do not have a DHT.
 
 The delegate node is started and the client of the delegate calls API methods using the IPFS HTTP API client.
 

docs/CORS.md

@@ -1,9 +1,11 @@
 # Development <!-- omit in toc -->
 
+> Getting started with development on IPFS
+
 - [Clone and install dependencies](#clone-and-install-dependencies)
 - [Run tests](#run-tests)
 - [Lint](#lint)
-- [Build a dist version](#build-a-dist-version)
+- [Build types and minified browser bundles](#build-types-and-minified-browser-bundles)
 - [Publishing new versions](#publishing-new-versions)
 - [Using prerelease versions](#using-prerelease-versions)
 - [Testing strategy](#testing-strategy)
@@ -32,14 +34,26 @@ See the scripts section of the root [`package.json`](./package.json) for more co
 # run all the unit tests
 > npm test
 
+# run individual tests (findprovs)
+> npm run test -- --grep findprovs
+
 # run just IPFS tests in Node.js
-> npm run test:node
+> npm run test -- -- -- -t node
+
+# run just IPFS tests in a headless browser
+> npm run test -- -- -- -t browser
+
+# run the interface tests against ipfs-core
+> npm run test:interface:core
+
+# run the interface tests over HTTP against js-ipfs
+> npm run test:interface:http-js
 
-# run just IPFS tests in a browser
-> npm run test:browser
+# run the interface tests over HTTP against go-ipfs from a browser
+> npm run test:interface:http-go -- -- -- -t browser
 
-# run just IPFS tests in a webworker
-> npm run test:webworker
+# run the interop tests against js-ipfs and go-ipfs on the Electron main process
+> npm run test:interop -- -- -- -t electron-main
 ```
 
 More granular test suites can be run from each submodule.
@@ -54,7 +68,7 @@ Please run the linter before submitting a PR, the build will not pass if it fail
 > npm run lint
 ```
 
-## Build a dist version
+## Build types and minified browser bundles
 
 ```sh
 > npm run build

packages/ipfs/docs/DAEMON.md renamed to docs/DAEMON.md

@@ -1,4 +1,9 @@
-# Early Testers Programme
+# Early Testers Programme <!-- omit in toc -->
+
+- [What is it?](#what-is-it)
+- [What are the expectations?](#what-are-the-expectations)
+- [Who has signed up?](#who-has-signed-up)
+- [How to sign up?](#how-to-sign-up)
 
 ## What is it?
 

packages/ipfs/docs/DELEGATE_ROUTERS.md renamed to docs/DELEGATE_ROUTERS.md

@@ -1,15 +1,16 @@
-# FAQ
-
-- [FAQ](#faq)
-  - [Why isn't there DHT support in js-IPFS?](#why-isnt-there-dht-support-in-js-ipfs)
-    - [Node.js](#nodejs)
-    - [Browser](#browser)
-  - [How to enable WebRTC support for js-ipfs in the Browser](#how-to-enable-webrtc-support-for-js-ipfs-in-the-browser)
-  - [Is there WebRTC support for js-ipfs with Node.js?](#is-there-webrtc-support-for-js-ipfs-with-nodejs)
-  - [How can I configure an IPFS node to use a custom `signaling endpoint` for my WebRTC transport?](#how-can-i-configure-an-ipfs-node-to-use-a-custom-signaling-endpoint-for-my-webrtc-transport)
-  - [I see some slowness when hopping between tabs Chrome with IPFS nodes, is there a reason why?](#i-see-some-slowness-when-hopping-between-tabs-chrome-with-ipfs-nodes-is-there-a-reason-why)
-  - [Can I use IPFS in my Electron App?](#can-i-use-ipfs-in-my-electron-app)
-  - [Have more questions?](#have-more-questions)
+# FAQ <!-- omit in toc -->
+
+## Table of Contents <!-- omit in toc -->
+
+- [Why isn't there DHT support in js-IPFS?](#why-isnt-there-dht-support-in-js-ipfs)
+  - [Node.js](#nodejs)
+  - [Browser](#browser)
+- [How to enable WebRTC support for js-ipfs in the Browser](#how-to-enable-webrtc-support-for-js-ipfs-in-the-browser)
+- [Is there WebRTC support for js-ipfs with Node.js?](#is-there-webrtc-support-for-js-ipfs-with-nodejs)
+- [How can I configure an IPFS node to use a custom `signaling endpoint` for my WebRTC transport?](#how-can-i-configure-an-ipfs-node-to-use-a-custom-signaling-endpoint-for-my-webrtc-transport)
+- [I see some slowness when hopping between tabs Chrome with IPFS nodes, is there a reason why?](#i-see-some-slowness-when-hopping-between-tabs-chrome-with-ipfs-nodes-is-there-a-reason-why)
+- [Can I use IPFS in my Electron App?](#can-i-use-ipfs-in-my-electron-app)
+- [Have more questions?](#have-more-questions)
 
 ## Why isn't there DHT support in js-IPFS?
 

docs/DEVELOPMENT.md

@@ -0,0 +1,75 @@
+# IPLD Codecs <!-- omit in toc -->
+
+## Table of Contents <!-- omit in toc -->
+
+- [Overview](#overview)
+- [Bundled Codecs](#bundled-codecs)
+- [Adding additional codecs](#adding-additional-codecs)
+- [Next steps](#next-steps)
+
+## Overview
+
+The IPFS repo contains a blockstore that holds [Blocks](https://github.com/ipld/js-ipld-block). These blocks can be thought of as a [CID][] and associated byte array.
+
+The [CID][] contains a `codec` property that lets us know how to interpret the byte array associated with it.
+
+In order to perform that interpretation, an [IPLD Format][] must be loaded that corresponds to the `codec` property of the [CID][].
+
+## Bundled Codecs
+
+js-IPFS ships with three bundled codecs, the ones that are required to create and interpret [UnixFS][] structures.
+
+These are:
+
+1. [ipld-dag-pb](https://github.com/ipld/js-ipld-dag-pb) - used for file and directory structures
+2. [ipld-raw](https://github.com/ipld/js-ipld-raw) - used for file data where imported with `raw-leaves=true`
+3. [ipld-dag-cbor](https://github.com/ipld/js-ipld-dag-cbor) - used for general storage of JavaScript Objects
+
+## Adding additional codecs
+
+If your application requires support for extra codecs, you can configure them as follows:
+
+1. Configure the [IPLD layer](https://github.com/ipfs/js-ipfs/blob/master/packages/ipfs/docs/MODULE.md#optionsipld) of your IPFS daemon to support the codec. This step is necessary so the node knows how to prepare data received over HTTP to be passed to IPLD for serialization:
+    ```javascript
+    const ipfs = require('ipfs')
+
+    const node = await ipfs({
+      ipld: {
+        // either specify them as part of the `formats` list
+        formats: [
+          require('my-format')
+        ],
+
+        // or supply a function to load them dynamically
+        loadFormat: async (format) => {
+          return require(format)
+        }
+      }
+    })
+2. Configure your IPFS HTTP API Client to support the codec. This is necessary so that the client can send the data to the IPFS node over HTTP:
+    ```javascript
+    const ipfsHttpClient = require('ipfs-http-client')
+
+    const client = ipfsHttpClient({
+      url: 'http://127.0.0.1:5002',
+      ipld: {
+        // either specify them as part of the `formats` list
+        formats: [
+          require('my-format')
+        ],
+
+        // or supply a function to load them dynamically
+        loadFormat: async (format) => {
+          return require(format)
+        }
+      }
+    })
+    ```
+
+## Next steps
+
+* See [examples/custom-ipld-formats](https://github.com/ipfs/js-ipfs/tree/master/examples/custom-ipld-formats) for runnable code that demonstrates the above with in-process IPFS nodes, IPFS run as a daemon and also the http client
+
+[cid]: https://www.npmjs.com/package/cids
+[ipld format]: https://github.com/ipld/interface-ipld-format
+[unixfs]: https://github.com/ipfs/specs/blob/master/UNIXFS.md

packages/ipfs/docs/DOCKER.md renamed to docs/DOCKER.md

@@ -8,7 +8,7 @@ Impact key:
 * 🍋 medium - involved refactoring in application code
 * 🍊 hard - complicated refactoring in application code
 
-## ToC <!-- omit in toc -->
+## Table of Contents <!-- omit in toc -->
 
 - [Migrating from callbacks](#migrating-from-callbacks)
 - [Migrating from `PeerId`](#migrating-from-peerid)

docs/EARLY_TESTERS.md

@@ -16,11 +16,22 @@ Whether accessed directly or over HTTP, both methods support the full [Core API]
 
 ## How tos and other documentation
 
+* [Architecture overview](./ARCHITECTURE.md)
 * [How to run js-IPFS in the browser](./BROWSERS.md)
+* [Running js-IPFS on the CLI](./CLI.md)
 * [js-IPFS configuration options](./CONFIG.md)
+* [How to configure CORS for use with the http client](./CORS.md)
+* [Running js-IPFS as a daemon](./DAEMON.md)
+* [Configuring Delegate Routers](./DELEGATE_ROUTERS.md)
+* [Running js-IPFS under Docker](./DOCKER.md)
+* [FAQ](./FAQ.md)
+* [How to configure additional IPLD codecs](./IPLD.md)
+* [Running js-IPFS in your application](./MODULE.md)
+* [How to get metrics out of js-IPFS](./MONITORING.md)
 
 ## Development documentation
 
+* [Getting started](./DEVELOPMENT.md)
 * [Release issue template](./RELEASE_ISSUE_TEMPLATE.md)
 * [Early testers](./EARLY_TESTERS.md)
 * [Releases](./RELEASES.md)

packages/ipfs/docs/FAQ.md renamed to docs/FAQ.md

@@ -1,6 +1,6 @@
-# Releases
+# Releases <!-- omit in toc -->
 
-## Table of Contents
+## Table of Contents <!-- omit in toc -->
 
 - [Release Philosophy](#release-philosophy)
 - [Release Flow](#release-flow)
@@ -13,7 +13,7 @@
   - [Patch Releases](#patch-releases)
 - [Performing a Release](#performing-a-release)
 - [Release Version Numbers](#release-version-numbers)
-  - [Pre-Releases and Release Candidates](#pre-releases-and-release-candidates)
+  - [Release Candidates](#release-candidates)
 
 ## Release Philosophy
 
@@ -100,19 +100,11 @@ Post `1.x.x` (future), MAJOR version number increments will contain BREAKING CHA
 
 We do not yet retroactively apply fixes to older releases (no Long Term Support releases for now), which means that we always recommend users to update to the latest, whenever possible.
 
-### Pre-Releases and Release Candidates
+### Release Candidates
 
-Prior to or during the early stages of a release cycle, js-ipfs may release a pre-release version for users who want to try out the "bleeding edge". This typically happens when a new major feature or breaking change lands in master. When this happens is entirely at the discretion of the Lead Maintainer.
+Every commit to master results in the publishing of a Release Candidate. These are made available for users who want to try out the "bleeding edge" and can be installed using version numbers with the form `x.y.z-rc.n` where `x`, `y`, and `z` are the usual MAJOR, MINOR and PATCH version numbers and `n` (starting at 0) which is the number of commits to master since the last full release.
 
-Pre-release version numbers take the form `x.y.z-pre.n`.
-
-Where `x`, `y`, and `z` are the usual MAJOR, MINOR and PATCH version numbers and `n` (starting at 0) increments by 1 for every pre-release version that is released.
-
-Following any pre-releases, at least one release candidate will be published during stage 1/2.
-
-Release candidate version numbers take the form `x.y.z-rc.n`.
-
-Where `x`, `y`, and `z` are the usual MAJOR, MINOR and PATCH version numbers and `n` (starting at 0) increments by 1 for every release candidate that is released.
+Alternatively the latest RC is tagged `next` on npm and can be installed using `npm install ipfs@next`.
 
 ---
 

docs/IPLD.md

@@ -0,0 +1,3 @@
+# Contributor Code of Conduct
+
+The `js-ipfs` project follows the [`IPFS Community Code of Conduct`](https://github.com/ipfs/community/blob/master/code-of-conduct.md)

docs/MIGRATION-TO-ASYNC-AWAIT.md

@@ -0,0 +1,9 @@
+# Contributing guidelines
+
+IPFS as a project, including js-ipfs and all of its modules, follows the [standard IPFS Community contributing guidelines](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md).
+
+We also adhere to the [IPFS JavaScript Community contributing guidelines](https://github.com/ipfs/community/blob/master/CONTRIBUTING_JS.md) which provide additional information of how to collaborate and contribute in the JavaScript implementation of IPFS.
+
+We appreciate your time and attention for going over these. Please open an issue on [ipfs/community](https://github.com/ipfs/community) if you have any question.
+
+Thank you.

packages/ipfs/docs/MODULE.md renamed to docs/MODULE.md

@@ -0,0 +1,5 @@
+This project is transitioning from an MIT-only license to a dual MIT/Apache-2.0 license.
+Unless otherwise noted, all code contributed prior to 2019-11-21 and not contributed by
+a user listed in [this signoff issue](https://github.com/ipfs/js-ipfs/issues/2624) is
+licensed under MIT-only. All new contributions (and past contributions since 2019-11-21)
+are licensed under a dual MIT/Apache-2.0 license.

packages/ipfs/docs/MONITORING.md renamed to docs/MONITORING.md

@@ -0,0 +1,5 @@
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

docs/README.md

@@ -0,0 +1,19 @@
+The MIT License (MIT)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.