Releases: ipfs/kubo
v0.9.0-rc1
Tracking Issue: #8058
v0.8.0
go-ipfs 0.8.0 Release
We're happy to announce go-ipfs 0.8.0! This release is focused on making it easier to work with pins. We have some snazzy new features around being able to ask remote services to pin data for you, and modifying large pin sets is much faster than ever before.
🔦 Highlights
🧷 Remote pinning services
There is now support for asking remote services to pin data for you.
This comes with a redesign of how we're thinking about pinning and includes some commonly requested features such as:
- Pins can have names (and coming soon metadata)
- Data can be pinned in either the foreground or background
- Pins can be searched by name, CID, or status
Command-line users benefit from ipfs pin remote
commands, which simplify remote pinning operations. The built-in pinning service API client executes all the necessary remote calls under the hood. HTTP API users have access to new commands under /api/v0/pin/remote
.
As long a service supports the vendor-agnostic IPFS Pinning Service API, it can be used directly in go-ipfs.
Examples include:
ipfs pin remote service add mysrv https://my-service.example.com/api-endpoint myAccessToken
ipfs pin remote service ls --stat # confirm service mysrv is available
ipfs pin remote add /ipfs/bafymydata --service=mysrv --name=myfile # will block until status is pinned
ipfs pin remote ls --service=mysrv --name=myfile
ipfs pin remote rm --serivce=mysrv --name=myfile
ipfs pin remote add /ipfs/bafymydata2 --service=mysrv --name=myfile2 --background # queue pin request and finish instantly
ipfs pin remote ls --service=mysrv --cid=bafymydata2 --status=queued,pinning,pinned,failed
ipfs pin remote rm --serivce=mysrv --cid=bafymydata2 --status=queued,pinning,pinned,failed
More examples can be found under ipfs pin remote --help
A few notes:
- Remote pinning services work with recursive pins. This means commands like
ipfs pin remote ls
will not list indirectly pinned CIDs. - By default, only finished, successful pins are listed. To list or remove pending/failed pins, pass explicit status:
--status=queued,pinning,pinned,failed
- While pinning service data is stored in the configuration file it cannot be edited directly via the
ipfs config
commands due to the sensitive nature of pinning service API keys. Theipfs pin remote service
commands can be used for interacting with remote service settings. - An OpenAPI ipfs-pinning-service.yaml makes it easy to create or generate a compatible client/server. Anyone can implement it and allow for pin management.
🏠 Remote MFS pinning policy
Every service added via ipfs pin remote service add
can be tasked to update a pin every time MFS root changes:
$ ipfs config --json Pinning.RemoteServices.mysrv.Policies.MFS.Enable
To avoid flooding remote service with many updates, go-ipfs will send them at most once every five minutes.
Details about customizing behavior of this feature can be found in configuration docs.
📌 Faster local pinning and unpinning
The pinning subsystem has been redesigned to be much faster and more flexible in how it tracks pins. For users who are working with many pins this will lead to a big speed increase in listing and modifying the set of pinned items as well as decreased memory usage.
Part of the redesign was setup to account for being able to interact with local pins the same way we can now interact with remote pins (e.g. names, being allowed to pin the same CID multiple times, etc.). Keep posted for more improvements to pinning.
🔒 DNSLink names on https:// subdomains
Previously DNSLink names would have trouble loading over subdomain gateways with HTTPS support since there is no way to get multilevel wildcard certificates (e.g. en.wikipedia-on-ipfs.org.ipns.dweb.link
cannot be covered by TLS wildcard cert for *.ipns.dweb.link
). Therefore, when trying to load DNSLink names over https:// subdomains go-ipfs we now forward to an inlined DNS name. Since DNS labels cannot contain .
in them they are escaped using -
.
/ipns/en.wikipedia-on-ipfs.org
→
ipns://en.wikipedia-on-ipfs.org
→
https://dweb.link/ipns/en.wikipedia-on-ipfs.org
→
https://en-wikipedia--on--ipfs-org.ipns.dweb.link
👈 a single DNS label, no TLS error 🔒
Note: The last redirect is specific to HTTPS, and is triggered only when X-Forwarded-Proto: https
header is present.
Recipes for setting up your own public gateway can be found in configuration docs.
💨 QUIC update
QUIC support has received a number of upgrades, including the ability to take advantage of larger UDP receive buffers for increased performance.
Linux users may notice a logged error on daemon startup if your system needs extra configuration to allow IPFS increase the buffer size. A helpful link for resolving this is in the log message as well as here.
👋 No more Darwin 386 builds
Go 1.15 (the latest version of Go) no longer supports Darwin 386 and so we are dropping support as well.
Changelog
Full Changelog
- github.com/ipfs/go-ipfs:
- Release v0.8.0
- docs: RepinInterval
- style: docs/config.md
- style: improved MFS PinName example
- docs: Pinning.RemoteServices.Policies
- fix: decrease log level of opencensus initialization (ipfs/go-ipfs#7815)
- Register oc metrics (ipfs/go-ipfs#7593)
- add remote pinning to ipfs command (#7661) (ipfs/go-ipfs#7661)
- More p2p proxy checks (ipfs/go-ipfs#7797)
- Use datastore based pinning (ipfs/go-ipfs#7750)
- fix: return an error when an unknown object type is passed (ipfs/go-ipfs#7795)
- clarify why ipfs file ls is being deprecated (ipfs/go-ipfs#7755)
- fix: ipfs dag export uses the CoreAPI and respects the offline flag (ipfs/go-ipfs#7753)
- return an error when trying to download fs-repo-migrations for linux + musl (ipfs/go-ipfs#7735)
- fix: do not create a new (unused) peerID when initializing from config (ipfs/go-ipfs#7730)
- docs: Add a link in config.md (ipfs/go-ipfs#7780)
- update libp2p for stream closure refactor (ipfs/go-ipfs#7747)
- Fix typo in ipfs dag stat command (ipfs/go-ipfs#7761)
- docs(readme): key rotation in docker (#7721) (ipfs/go-ipfs#7721)
- fix(dnslink-gw): breadcrumbs and CID column when dir listing (ipfs/go-ipfs#7699)
- fix(gw): preserve query on website redirect (ipfs/go-ipfs#7727)
- feat: ipfs-webui v2.11.4 (ipfs/go-ipfs#7716)
- docs: how the ipfs snap is built and published (ipfs/go-ipfs#7725)
- fix: webui on ipv6 localhost (ipfs/go-ipfs#7731)
- Add missing plugin support on FreeBSD (ipfs/go-ipfs#7722)
- fix error when computing coverage (ipfs/go-ipfs#7726)
- docs(config): X-Forwarded-Host (ipfs/go-ipfs#7651)
- chore: webui v2.11.2 (ipfs/go-ipfs#7703)
- Add task for updating CLI docs right after updating the HTTP-api docs (ipfs/go-ipfs#7711)
- feat(gateway): Content-Disposition improvements (ipfs/go-ipfs#7677)
- fix build on Plan 9 (ipfs/go-ipfs#7690)
- docs: update changelog for v0.7.0
- chore: bump webui version
- fix: remove the (empty) alias for --peerid-base
- fix: use override GOFLAGS changes from 480defa but keep trimpath usage as is
- Revert "fix: override GOFLAGS"
- Fix --ipns-base alias (ipfs/go-ipfs#7659)
- docs: update config to indicate SECIO deprecation (ipfs/go-ipfs#7630)
- fix: ipfs dht put/get commands with peerIDs encoded as CIDs (ipfs/go-ipfs#7633)
- update version to 0.8.0-dev (ipfs/go-ipfs#7629)
- github.com/ipfs/go-bitswap (v0.2.20 -> v0.3.3):
- feat: configurable engine blockstore worker count (#449) (ipfs/go-bitswap#449)
- fix: set the score ledger on start (ipfs/go-bitswap#447)
- feat: update for go-libp2p-core 0.7.0 interface changes (ipfs/go-bitswap#445)
- fix: guard access to the mock wiretap with a lock ([ipfs/go-bitswap#446](https://github.com/...
v0.8.0-rc2
Tracking Issue: #7707
v0.8.0-rc1
Tracking Issue: #7707
Release v0.7.0
We're happy to announce go-ipfs v0.7.0, it has a number of user facing changes so if you're wondering what happened to SECIO or why you're starting to see more peers with peerIDs that don't start with Qm checkout the highlights.
🔦 Highlights
🔒 Secio is now disabled by default
As part of deprecating and removing support for the Secio security transport, we have disabled it by default. TLS1.3 will remain the default security transport with fallback to Noise. You can read more about the deprecation in the blog post, https://blog.ipfs.io/2020-08-07-deprecating-secio/. If you're running IPFS older than 0.5, this may start to impact your performance on the public network.
🗝️ Ed25519 keys are now used by default
Previously go-ipfs generated 2048 bit RSA keys for new nodes, but it will now use ed25519 keys by default. This will not affect any existing keys, but newly created keys will be ed25519 by default. The main benefit of using ed25519 keys over RSA is that ed25519 keys have an inline public key. This means that someone only needs your PeerId to verify things you've signed, which means we don't have to worry about storing those bulky RSA public keys.
Rotating keys
Along with switching the default, we've added support for rotating keys. If you would like to change the key type of your IPFS node, you can now do so with the rotate command. NOTE: This will affect your Peer Id, so be sure you want to do this! Your existing identity key will be backed up in the Keystore.
ipfs key rotate -o my-old-key -t ed25519
📦 Key export/import
We've added commands to allow you to export and import keys from the IPFS Keystore to a local .key file. This does not apply to the IPFS identity key, self
.
ipfs key gen mykey
ipfs key export -o mykey.key mykey # ./<name>.key is the default path
ipfs key import mykey mykey.key # on another node
#️⃣ IPNS paths now encode the key name as a base36 CIDv1 by default
Previously go-ipfs encoded the key names for IPNS paths as base58btc multihashes (e.g. Qmabc...). We now encode them as base36 encoded CIDv1s as defined in the peerID spec (e.g. k51xyz...) which also deals with encoding of public keys. This is nice because it means that IPNS keys will by default be case-insensitive and that they will fit into DNS labels (e.g. k51xyz...ipns.localhost) and therefore that subdomain gateway redirections (e.g. from localhost:8080/ipns/{key} to {key}.ipns.localhost) will look better to users in the default case.
Many commands will accept a --ipns-base
option that allows changing command outputs to use a particular encoding (i.e. base58btc multihash, or CIDv1 encoded in any supported base)
📮 Multiaddresses now accept PeerIDs encoded as CIDv1
In preparation for eventually changing the default PeerID representation multiaddresses can now contain strings like /p2p/k51xyz...
in addition to the default /p2p/Qmabc...
. There is a corresponding --peerid-base
option to many functions that output peerIDs.
🧮 dag stat
Initial support has been added for the ipfs dag stat
command. Running this command will traverse the DAG for the given root CID and report statistics. By default, progress will be shown as the DAG is traversed. Supported statistics currently include DAG size and number of blocks.
ipfs dag stat bafybeihpetclqvwb4qnmumvcn7nh4pxrtugrlpw4jgjpqicdxsv7opdm6e # the IPFS webui
Size: 30362191, NumBlocks: 346
🔌 Plugin build changes 🚨
We have changed the build flags used by the official binary distributions on dist.ipfs.io (or /ipns/dist.ipfs.io
) to use the simpler and more reliable -trimpath
flag instead of the more complicated and brittle -asmflags=all=-trimpath="$(GOPATH)" -gcflags=all=-trimpath="$(GOPATH)"
flags, however the build flags used by default in go-ipfs remain the same.
The scripts in https://github.com/ipfs/go-ipfs-example-plugin have been updated to reflect this change. This is a breaking change to how people have been building plugins against the dist.ipfs.io binary of go-ipfs and plugins should update their build processes accordingly see ipfs/go-ipfs-example-plugin#9 for details.
Changelog
Full Changelog
- github.com/ipfs/go-ipfs:
- chore: bump webui version
- fix: remove the (empty) alias for --peerid-base
- Release v0.7.0-rc2
- fix: use override GOFLAGS changes from 480defa but keep trimpath usage as is
- Revert "fix: override GOFLAGS"
- fix: remove the (empty) alias for --ipns-base
- refactor: put all --ipns-base options in one place
- docs: update config to indicate SECIO deprecation
- fix: ipfs dht put/get commands now work on keys encoded as peerIDs and fail early for namespaces other than /pk or /ipns
- Release v0.7.0-rc1
- chore: cleanup (ipfs/go-ipfs#7628)
- namesys: fixed IPNS republisher to not overwrite IPNS record lifetimes (ipfs/go-ipfs#7627)
- Fix #7624: Do not fetch dag nodes when checking if a pin exists (ipfs/go-ipfs#7625)
- chore: update dependencies (ipfs/go-ipfs#7610)
- use t.Cleanup() to reduce the need to clean up servers in tests (ipfs/go-ipfs#7550)
- fix: ipfs pin ls - ignore pins that have errors (ipfs/go-ipfs#7612)
- docs(config): fix Peering header (ipfs/go-ipfs#7623)
- sharness: use dnsaddr example in ipfs p2p command tests (ipfs/go-ipfs#7620)
- fix(key): dont allow backup key to be named 'self' (ipfs/go-ipfs#7615)
- [BOUNTY] Directory page UI improvements (ipfs/go-ipfs#7536)
- fix: make assets deterministic (ipfs/go-ipfs#7609)
- use ed25519 keys by default (ipfs/go-ipfs#7579)
- feat: wildcard support for public gateways (ipfs/go-ipfs#7319)
- fix: fix go-bindata import path (ipfs/go-ipfs#7605)
- Upgrade graphsync deps (ipfs/go-ipfs#7598)
- Add --peerid-base to ipfs id command (ipfs/go-ipfs#7591)
- use b36 keys by default for keys and IPNS (ipfs/go-ipfs#7582)
- add ipfs dag stat command (#7553) (ipfs/go-ipfs#7553)
- Move key rotation command to ipfs key rotate (ipfs/go-ipfs#7599)
- Disable secio by default (ipfs/go-ipfs#7600)
- Stop searching for public keys before doing an IPNS Get (#7549) (ipfs/go-ipfs#7549)
- feat: return supported protocols in id output (ipfs/go-ipfs#7409)
- docs: fix typo in default swarm addrs config docs (ipfs/go-ipfs#7585)
- feat: nice errors when failing to load plugins (ipfs/go-ipfs#7429)
- doc: document reverse proxy bug (ipfs/go-ipfs#7478)
- fix: ipfs name resolve --dht-record-count flag uses correct type and now works
- refactor: get rid of cmdDetails awkwardness
- IPNS format keys in b36cid (ipfs/go-ipfs#7554)
- Key import and export cli commands (ipfs/go-ipfs#7546)
- feat: add snap package configuration (ipfs/go-ipfs#7529)
- chore: bump webui version
- repeat gateway subdomain test for all key types (#7542) (ipfs/go-ipfs#7542)
- fix: override GOFLAGS
- update QUIC, enable the RetireBugBackwardsCompatibilityMode
- Document add behavior when the daemon is not running (ipfs/go-ipfs#7514)
- (ipfs/go-ipfs#7515)
- Choose Key type at initialization (ipfs/go-ipfs#7251)
- feat: add flag to ipfs key and list to output keys in b36/CIDv1 (#7531) (ipfs/go-ipfs#7531)
- feat: support ED25519 libp2p-key in subdomains
- chore: fix a typo
- docs: document X-Forwarded-Host
- feat: support X-Forwarded-Host when doing gateway redirect
- chore: update test deps for graphsync
- chore: bump test dependencies (ipfs/go-ipfs#7524)
- fix: use static binaries in docker container (ipfs/go-ipfs#7505)
- chore:bump webui version to 2.10.1 (ipfs/go-ipfs#7504)
- chore: bump webui version (ipfs/go-ipfs#7501)
- update version to 0.7.0-dev
- Merge branch 'release' into master
- systemd: specify repo path, to avoid unnecessary subdirectory (ipfs/go-ipfs#7472)
- doc(prod): start documenting production stuff (ipfs/go-ipfs#7469)
- Readme: Update link about init systems (and import old readme) ([#7473](https://github.com/ipfs/go-ip...
v0.7.0-rc2
Tracking Issue: #7560
Release v0.7.0-rc1
Tracking Issue: #7560
Release v0.6.0
This is a relatively small release in terms of code changes, but it contains some significant changes to the IPFS protocol.
Highlights
The highlights in this release include:
- The QUIC transport is enabled by default. Furthermore, go-ipfs will automatically run a migration to listen on the QUIC transport (on the same address/port as the TCP transport) to make this upgrade process seamless.
- The new NOISE security transport is now supported but won't be selected by default. This transport will replace SECIO as the default cross-language interoperability security transport. TLS 1.3 will still remain the default security transport between go-ipfs nodes for now.
MIGRATION: This release contains a small config migration to enable listening on the QUIC transport in addition the TCP transport. This migration will:
- Normalize multiaddrs in the bootstrap list to use the
/p2p/Qm...
syntax for multiaddrs instead of the/ipfs/Qm...
syntax. - Add QUIC addresses for the default bootstrapers, as necessary. If you've removed the default bootstrappers from your bootstrap config, the migration won't add them back.
- Add a QUIC listener address to mirror any TCP addresses present in your config. For example, if you're listening on
/ip4/0.0.0.0/tcp/1234
, this migration will add a listen address for/ip4/0.0.0.0/udp/1234/quic
.
QUIC by default
This release enables the QUIC transport (draft 28) by default for both inbound and outbound connections. When connecting to new peers, libp2p will continue to dial all advertised addresses (tcp + quic) in parallel so if the QUIC connection fails for some reason, the connection should still succeed.
The QUIC transport has several key benefits over the current TCP based transports:
- It takes fewer round-trips to establish a connection. With the QUIC transport, the IPFS handshake takes two round trips (one to establish the QUIC connection, one for the libp2p handshake). In the future, we should be able to reduce this to one round trip for the initial connection, and zero round trips for subsequent connections to a previously seen peer. This is especially important for DHT requests that contact many new peers.
- Because it's UDP based instead of TCP based, it uses fewer file descriptors. The QUIC transport will open one UDP socket per listen address instead of one socket per connection. This should, in the future, allow us to keep more connections open.
- Because QUIC connections don't consume file descriptors, we're able to remove the rate limit on outbound QUIC connections, further speeding up DHT queries.
Unfortunately, this change isn't without drawbacks: the QUIC transport may not be able to max out some links (usually due to poorly tuned kernel parameters). On the other hand, it may also be faster in some cases
If you hit this performance issue on Linux, you should tune the net.core.rmem_default
and net.core.rmem_max
sysctl parameters to increase your UDP receive buffer sizes.
If necessary, you can disable the QUIC transport by running:
> ipfs config --json Swarm.Transports.Network.QUIC false
NOTE: The QUIC transport included in this release is backwards incompatible with the experimental QUIC transport included in previous releases. Unfortunately, the QUIC protocol underwent some significant breaking changes and supporting multiple versions wasn't an option. In practice this degrades gracefully as go-ipfs will simply fall back on the TCP transport when dialing nodes with incompatible QUIC versions.
Noise Transport
This go-ipfs release introduces a new security transport: libp2p Noise (built from the Noise Protocol Framework). While TLS1.3 remains the default go-ipfs security transport, Noise is simpler to implement from scratch and will be the standard cross-platform libp2p security transport going forward.
This brings us one step closer to deprecating and removing support for SECIO.
While enabled by default, Noise won't actually be used by default it's negotiated. Given that TLS1.3 is still the default security transport for go-ipfs, this usually won't happen. If you'd like to prefer Noise over other security transports, you can change its priority in the config (Swarm.Transports.Security.Noise
).
Gateway
This release brings two gateway-relevant features: custom 404 pages and base36 support.
Custom 404
You can now customize 404 Not Found
error pages by including an ipfs-404.html
file somewhere in the request path. When a requested file isn't found, go-ipfs will look for an ipfs-404.html
in the same directory as the requested file, and in each ancestor directory. If found, this file will be returned (with a 404 status code) instead of the usual error message.
Support for Base36
This release adds support for a new multibase encoding: base36. Base36 is an optimally efficient case-insensitive alphanumeric encoding. Case-insensitive alphanumeric encodings are important for the subdomain gateway as domain names are case insensitive.
While base32 (the current default encoding used in subdomains) is simpler than base36, it's not optimally efficient and base36 Ed25519 IPNS keys are 2 characters too big to fit into the 63 character subdomain length limit. The extra efficiency from base36 brings us under this limit and allows Ed25519 IPNS keys to work with the subdomain gateway.
This release adds support for base36 but won't use it by default. If you'd like to re-encode an Ed25519 IPNS key into base36, you can use the ipfs cid format
command:
$ ipfs cid format -v 1 --codec libp2p-key -b base36 bafzaajaiaejca4syrpdu6gdx4wsdnokxkprgzxf4wrstuc34gxw5k5jrag2so5gk k51qzi5uqu5dj16qyiq0tajolkojyl9qdkr254920wxv7ghtuwcz593tp69z9m
Gossipsub Upgrade
This release brings a new gossipsub protocol version: 1.1. You can read about it in the blog post.
Connectivity
This release introduces a new "peering" feature. The peering subsystem configures go-ipfs to connect to, remain connected to, and reconnect to a set of nodes. Nodes should use this subsystem to create "sticky" links between frequently useful peers to improve reliability.
Use-cases:
- An IPFS gateway connected to an IPFS cluster should peer to ensure that the gateway can always fetch content from the cluster.
- A dapp may peer embedded go-ipfs nodes with a set of pinning services or textile cafes/hubs.
- A set of friends may peer to ensure that they can always fetch each other's content.
Changelog
- github.com/ipfs/go-ipfs:
- fix 3 bugs responsible for a goroutine leak (plus one other bug) (ipfs/go-ipfs#7491)
- docs(config): update toc (ipfs/go-ipfs#7483)
- feat: transport config (ipfs/go-ipfs#7479)
- fix the minimal go version under 'Build from Source' (ipfs/go-ipfs#7459)
- fix(migration): migrate /ipfs/ bootstrappers to /p2p/
- fix(migration): correctly migrate quic addresses
- chore: add migration to listen on QUIC by default
- backport fixes (ipfs/go-ipfs#7405)
- Use bitswap sessions for
ipfs refs
. - Update to webui 2.9.0
- Use bitswap sessions for
- feat: add noise support (ipfs/go-ipfs#7365)
- feat: implement peering service (ipfs/go-ipfs#7362)
- Include the git blob id of the dir-index bundle in the ETag (ipfs/go-ipfs#7360)
- feat: bootstrap in dht when the routing table is empty (ipfs/go-ipfs#7340)
- quic: remove experimental status and add it to the default config (ipfs/go-ipfs#7349)
- fix: support directory listings even if a 404 page is present (ipfs/go-ipfs#7339)
- doc(plugin): document plugin config (ipfs/go-ipfs#7309)
- test(sharness): fix fuse tests (ipfs/go-ipfs#7320)
- docs: update experimental-features doc with IPNS over pubsub changes. (ipfs/go-ipfs#7334)
- docs: cleanup config formatting (ipfs/go-ipfs#7336)
- fix(gateway): ensure directory listings have Content-Type text/html (ipfs/go-ipfs#7330)
- test(sharness): test the local symlink (ipfs/go-ipfs#7332)
- misc config/experimental-features doc fixes (ipfs/go-ipfs#7333)
- fix: correctly trim resolved IPNS addresses (ipfs/go-ipfs#7331)
- Gateway renders pretty 404 pages if available (ipfs/go-ipfs#4233)
- feat: add a dht stat command (ipfs/go-ipfs#7221)
- fix: update dists url for OpenBSD support (ipfs/go-ipfs#7311)
- docs: X-Forwarded-Proto: https (ipfs/go-ipfs#7306)
- fix(mkreleaselog): make robust against running in different working directories (ipfs/go-ipfs#7310)
- fix(mkreleasenotes): include commits directly to master (ipfs/go-ipfs#7296)
- write a...
Release v0.6.0-rc7
Tracking issue: #7347
Release v0.6.0-rc6
Tracking issue: #7347