docs: add developer guide for local development workflow

CONTRIBUTING.md

@@ -1,6 +1,10 @@
-IPFS as a project, including go-ipfs and all of its modules, follows the [standard IPFS Community contributing guidelines](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md).
+# Contributing to Kubo
 
-We also adhere to the [GO IPFS Community contributing guidelines](https://github.com/ipfs/community/blob/master/CONTRIBUTING_GO.md) which provide additional information of how to collaborate and contribute in the Go implementation of IPFS.
+**For development setup, building, and testing, see the [Developer Guide](docs/developer-guide.md).**
+
+IPFS as a project, including Kubo and all of its modules, follows the [standard IPFS Community contributing guidelines](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md).
+
+We also adhere to the [Go IPFS Community contributing guidelines](https://github.com/ipfs/community/blob/master/CONTRIBUTING_GO.md) which provide additional information on how to collaborate and contribute to the Go implementation of IPFS.
 
 We appreciate your time and attention for going over these. Please open an issue on ipfs/community if you have any questions.
 

README.md

@@ -98,11 +98,6 @@ Before opening an issue, consider using one of the following locations to ensure
   - [Troubleshooting](#troubleshooting-1)
 - [Packages](#packages)
 - [Development](#development)
-  - [Map of Implemented Subsystems](#map-of-implemented-subsystems)
-  - [CLI, HTTP-API, Architecture Diagram](#cli-http-api-architecture-diagram)
-  - [Testing](#testing)
-  - [Development Dependencies](#development-dependencies)
-  - [Developer Notes](#developer-notes)
 - [Maintainer Info](#maintainer-info)
 - [Contributing](#contributing)
 - [License](#license)
@@ -450,41 +445,7 @@ See [IPFS in GO](https://docs.ipfs.tech/reference/go/api/) documentation.
 
 ## Development
 
-Some places to get you started on the codebase:
-
-- Main file: [./cmd/ipfs/main.go](https://github.com/ipfs/kubo/blob/master/cmd/ipfs/main.go)
-- CLI Commands: [./core/commands/](https://github.com/ipfs/kubo/tree/master/core/commands)
-- Bitswap (the data trading engine): [go-bitswap](https://github.com/ipfs/go-bitswap)
-- libp2p
-  - libp2p: https://github.com/libp2p/go-libp2p
-  - DHT: https://github.com/libp2p/go-libp2p-kad-dht
-- [IPFS : The `Add` command demystified](https://github.com/ipfs/kubo/tree/master/docs/add-code-flow.md)
-
-### Map of Implemented Subsystems
-**WIP**: This is a high-level architecture diagram of the various sub-systems of this specific implementation. To be updated with how they interact. Anyone who has suggestions is welcome to comment [here](https://docs.google.com/drawings/d/1OVpBT2q-NtSJqlPX3buvjYhOnWfdzb85YEsM_njesME/edit) on how we can improve this!
-<img src="https://docs.google.com/drawings/d/e/2PACX-1vS_n1FvSu6mdmSirkBrIIEib2gqhgtatD9awaP2_WdrGN4zTNeg620XQd9P95WT-IvognSxIIdCM5uE/pub?w=1446&amp;h=1036">
-
-### CLI, HTTP-API, Architecture Diagram
-
-![](./docs/cli-http-api-core-diagram.png)
-
-> [Origin](https://github.com/ipfs/pm/pull/678#discussion_r210410924)
-
-Description: Dotted means "likely going away". The "Legacy" parts are thin wrappers around some commands to translate between the new system and the old system. The grayed-out parts on the "daemon" diagram are there to show that the code is all the same, it's just that we turn some pieces on and some pieces off depending on whether we're running on the client or the server.
-
-### Testing
-
-```
-make test
-```
-
-### Development Dependencies
-
-If you make changes to the protocol buffers, you will need to install the [protoc compiler](https://github.com/google/protobuf).
-
-### Developer Notes
-
-Find more documentation for developers on [docs](./docs)
+See the [Developer Guide](docs/developer-guide.md) for build instructions, testing, architecture, and contribution workflow.
 
 ## Maintainer Info
 

docs/README.md

@@ -14,9 +14,9 @@ Otherwise, check out the following guides to using and developing IPFS:
 
 ## Developing `kubo`
 
-- First, please read the Contributing Guidelines [for IPFS projects](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md) and then the Contributing Guidelines for [Go code specifically](https://github.com/ipfs/community/blob/master/CONTRIBUTING_GO.md)
-- Building on…
-    - [Windows](windows.md)
+- **[Developer Guide](developer-guide.md)** - prerequisites, build, test, and contribute
+- Contributing Guidelines [for IPFS projects](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md) and for [Go code specifically](https://github.com/ipfs/community/blob/master/CONTRIBUTING_GO.md)
+- Building on [Windows](windows.md)
 - [Performance Debugging Guidelines](debug-guide.md)
 - [Release Checklist](releases.md)
 

docs/developer-guide.md

@@ -0,0 +1,316 @@
+# Developer Guide
+
+By the end of this guide, you will be able to:
+
+- Build Kubo from source
+- Run the test suites
+- Make and verify code changes
+
+This guide covers the local development workflow. For user documentation, see [docs.ipfs.tech](https://docs.ipfs.tech/).
+
+## Table of Contents
+
+- [Prerequisites](#prerequisites)
+- [Quick Start](#quick-start)
+- [Building](#building)
+- [Running Tests](#running-tests)
+- [Running the Linter](#running-the-linter)
+- [Common Development Tasks](#common-development-tasks)
+- [Code Organization](#code-organization)
+- [Architecture](#architecture)
+- [Troubleshooting](#troubleshooting)
+- [Development Dependencies](#development-dependencies)
+- [Further Reading](#further-reading)
+
+## Prerequisites
+
+Before you begin, ensure you have:
+
+- **Go** - see `go.mod` for the minimum required version
+- **Git**
+- **GNU Make**
+- **GCC** (optional) - required for CGO (Go's C interop); without it, build with `CGO_ENABLED=0`
+
+## Quick Start
+
+```bash
+git clone https://github.com/ipfs/kubo.git
+cd kubo
+make build
+./cmd/ipfs/ipfs version
+```
+
+You should see output like:
+
+```
+ipfs version 0.34.0-dev
+```
+
+The binary is built to `cmd/ipfs/ipfs`. To install it system-wide:
+
+```bash
+make install
+```
+
+This installs the binary to `$GOPATH/bin`.
+
+## Building
+
+| Command | Description |
+|---------|-------------|
+| `make build` | build the `ipfs` binary to `cmd/ipfs/ipfs` |
+| `make install` | install to `$GOPATH/bin` |
+| `make nofuse` | build without FUSE support |
+| `make build CGO_ENABLED=0` | build without CGO (no C compiler needed) |
+
+For Windows-specific instructions, see [windows.md](windows.md).
+
+## Running Tests
+
+Kubo has two types of tests:
+
+- **Unit tests** - test individual packages in isolation. Fast and don't require a running daemon.
+- **End-to-end tests** - spawn real `ipfs` nodes, run actual CLI commands, and test the full system. Slower but catch integration issues.
+
+Note that `go test ./...` runs both unit and end-to-end tests. Use `make test` to run all tests. CI runs unit and end-to-end tests in separate jobs for faster feedback.
+
+<!-- TODO: uncomment when https://github.com/ipfs/kubo/pull/11113 is merged
+| Command | What it runs |
+|---------|--------------|
+| `make test_unit` | unit tests only (excludes `test/cli`) |
+| `make test_cli` | CLI end-to-end tests only (requires `make build` first) |
+| `make test_sharness` | sharness end-to-end tests only |
+| `make test` | all tests (unit + CLI + sharness) |
+-->
+
+For end-to-end tests, Kubo has two suites:
+
+- **`test/cli`** - modern Go-based test harness that spawns real `ipfs` nodes and runs actual CLI commands. All new tests should be added here.
+- **`test/sharness`** - legacy bash-based tests. We are slowly migrating these to `test/cli`.
+
+When modifying tests: cosmetic changes to `test/sharness` are fine, but if significant rewrites are needed, remove the outdated sharness test and add a modern one to `test/cli` instead.
+
+### Before Running Tests
+
+**Environment requirements**: some legacy tests expect default ports (8080, 5001, 4001) to be free and no mDNS (local network discovery) Kubo service on the LAN. Tests may fail if you have a local Kubo instance running. Before running the full test suite, stop any running `ipfs daemon`.
+
+Two critical setup steps:
+
+1. **Rebuild after code changes**: if you modify any `.go` files outside of `test/`, you must run `make build` before running integration tests.
+
+2. **Set environment variables**: integration tests use the `ipfs` binary from `PATH` and need an isolated `IPFS_PATH`. Run these commands from the repository root:
+
+```bash
+export PATH="$PWD/cmd/ipfs:$PATH"
+export IPFS_PATH="$(mktemp -d)"
+```
+
+### Unit Tests
+
+```bash
+go test ./...
+```
+
+### CLI Integration Tests (`test/cli`)
+
+These are Go-based integration tests that invoke the `ipfs` CLI.
+
+Instead of running the entire test suite, you can run a specific test to get faster feedback during development.
+
+Run a specific test (recommended during development):
+
+```bash
+go test ./test/cli/... -run TestAdd -v
+```
+
+Run all CLI tests:
+
+```bash
+go test ./test/cli/...
+```
+
+Run a specific test:
+
+```bash
+go test ./test/cli/... -run TestAdd
+```
+
+Run with verbose output:
+
+```bash
+go test ./test/cli/... -v
+```
+
+**Common error**: "version (16) is lower than repos (17)" means your `PATH` points to an old binary. Check `which ipfs` and rebuild with `make build`.
+
+### Sharness Tests (`test/sharness`)
+
+Shell-based integration tests using [sharness](https://github.com/chriscool/sharness) (a portable shell testing framework).
+
+```bash
+cd test/sharness
+```
+
+Run a specific test:
+
+```bash
+timeout 60s ./t0080-repo.sh
+```
+
+Run with verbose output (this disables automatic cleanup):
+
+```bash
+./t0080-repo.sh -v
+```
+
+**Cleanup**: the `-v` flag disables automatic cleanup. Before re-running tests, kill any dangling `ipfs daemon` processes:
+
+```bash
+pkill -f "ipfs daemon"
+```
+
+### Full Test Suite
+
+```bash
+make test        # run all tests
+make test_short  # run shorter test suite
+```
+
+## Running the Linter
+
+Run the linter using the Makefile target (not `golangci-lint` directly):
+
+```bash
+make -O test_go_lint
+```
+
+## Common Development Tasks
+
+### Modifying CLI Commands
+
+After editing help text in `core/commands/`, verify the output width:
+
+```bash
+go test ./test/cli/... -run TestCommandDocsWidth
+```
+
+### Updating Dependencies
+
+Use the Makefile target (not `go mod tidy` directly):
+
+```bash
+make mod_tidy
+```
+
+### Editing the Changelog
+
+When modifying `docs/changelogs/`:
+
+- update the Table of Contents when adding sections
+- add user-facing changes to the Highlights section (the Changelog section is auto-generated)
+
+### Running the Daemon
+
+Always run the daemon with a timeout or shut it down promptly.
+
+With timeout:
+
+```bash
+timeout 60s ipfs daemon
+```
+
+Or shut down via API:
+
+```bash
+ipfs shutdown
+```
+
+For multi-step experiments, store `IPFS_PATH` in a file to ensure consistency.
+
+## Code Organization
+
+| Directory | Description |
+|-----------|-------------|
+| `cmd/ipfs/` | CLI entry point and binary |
+| `core/` | core IPFS node implementation |
+| `core/commands/` | CLI command definitions |
+| `core/coreapi/` | Go API implementation |
+| `client/rpc/` | HTTP RPC client |
+| `plugin/` | plugin system |
+| `repo/` | repository management |
+| `test/cli/` | Go-based CLI integration tests |
+| `test/sharness/` | legacy shell-based integration tests |
+| `docs/` | documentation |
+
+Key external dependencies:
+
+- [go-libp2p](https://github.com/libp2p/go-libp2p) - networking stack
+- [go-libp2p-kad-dht](https://github.com/libp2p/go-libp2p-kad-dht) - distributed hash table
+- [boxo](https://github.com/ipfs/boxo) - IPFS SDK (including Bitswap, the data exchange engine)
+
+For a deep dive into how code flows through Kubo, see [The `Add` command demystified](add-code-flow.md).
+
+## Architecture
+
+**Map of Implemented Subsystems** ([editable source](https://docs.google.com/drawings/d/1OVpBT2q-NtSJqlPX3buvjYhOnWfdzb85YEsM_njesME/edit)):
+
+<img src="https://docs.google.com/drawings/d/e/2PACX-1vS_n1FvSu6mdmSirkBrIIEib2gqhgtatD9awaP2_WdrGN4zTNeg620XQd9P95WT-IvognSxIIdCM5uE/pub?w=1446&amp;h=1036">
+
+**CLI, HTTP-API, Core Diagram**:
+
+![](./cli-http-api-core-diagram.png)
+
+## Troubleshooting
+
+### "version (N) is lower than repos (M)" Error
+
+This means the `ipfs` binary in your `PATH` is older than expected.
+
+Check which binary is being used:
+
+```bash
+which ipfs
+```
+
+Rebuild and verify PATH:
+
+```bash
+make build
+export PATH="$PWD/cmd/ipfs:$PATH"
+./cmd/ipfs/ipfs version
+```
+
+### FUSE Issues
+
+If you don't need FUSE support, build without it:
+
+```bash
+make nofuse
+```
+
+Or set the `TEST_FUSE=0` environment variable when running tests.
+
+### Build Fails with "No such file: stdlib.h"
+
+You're missing a C compiler. Either install GCC or build without CGO:
+
+```bash
+make build CGO_ENABLED=0
+```
+
+## Development Dependencies
+
+If you make changes to the protocol buffers, you will need to install the [protoc compiler](https://github.com/google/protobuf).
+
+## Further Reading
+
+- [The `Add` command demystified](add-code-flow.md) - deep dive into code flow
+- [Configuration reference](config.md)
+- [Performance debugging](debug-guide.md)
+- [Experimental features](experimental-features.md)
+- [Release process](releases.md)
+- [Contributing guidelines](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md)
+
+## Source Code
+
+The complete source code is at [github.com/ipfs/kubo](https://github.com/ipfs/kubo).