Skip to content

Commit 649e4b6

Browse files
pojntfxpojntfx
committed
docs: Add usage and download instructions
1 parent c5dc0d4 commit 649e4b6

File tree

5 files changed

+141
-6
lines changed

5 files changed

+141
-6
lines changed

‎.github/workflows/hydrun.yaml

+2-2
Original file line numberDiff line numberDiff line change
@@ -33,13 +33,13 @@ jobs:
3333
flags: -e '--privileged'
3434
cmd: ./Hydrunfile gccgo ram-dl
3535
dst: out/*
36-
- id: go-ram-dl-server
36+
- id: go-ram-ul
3737
src: .
3838
os: golang:bullseye
3939
flags: ""
4040
cmd: ./Hydrunfile go ram-dl
4141
dst: out/*
42-
- id: gccgo-ram-dl-server
42+
- id: gccgo-ram-ul
4343
src: .
4444
os: ghcr.io/pojntfx/bagccgop-base-sid
4545
flags: -e '--privileged'

‎Makefile

+1-1
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ OUTPUT_DIR ?= out
55
DST ?=
66

77
# Private variables
8-
obj = ram-dl ram-dl-server
8+
obj = ram-dl ram-ul
99
all: $(addprefix build/,$(obj))
1010

1111
# Build

‎README.md

+135-2
Original file line numberDiff line numberDiff line change
@@ -5,13 +5,146 @@
55
A tool to download more RAM (yes, seriously!)
66

77
[![hydrun CI](https://github.com/pojntfx/ram-dl/actions/workflows/hydrun.yaml/badge.svg)](https://github.com/pojntfx/ram-dl/actions/workflows/hydrun.yaml)
8+
[![Matrix](https://img.shields.io/matrix/go-nbd:matrix.org)](https://matrix.to/#/#ram-dl:matrix.org?via=matrix.org)
89

910
## Overview
1011

11-
🚧 This project is a work-in-progress! Instructions will be added as soon as it is usable. 🚧
12+
`ram-dl` is a tool that allows you to **upload** and **download** RAM to/from your system. Well, sort of :P
13+
14+
It is mostly intended as a tech demo for [r3map](https://github.com/pojntfx/r3map), a library for efficient distributed `mmap` with mounting and migration support, and [go-nbd](https://github.com/pojntfx/go-nbd), a pure Go NBD server and client library. Despite this it can however also be of use for a few experimental use cases, such as:
15+
16+
- **Extending a system's memory capacity over the network**: By leveraging the fast [fRPC](https://frpc.io/) RPC framework, you can use `ram-dl` and `ram-ul` to expand your local system's RAM (effectively **"downloading RAM"**) without having to use local disk space or memory.
17+
- **Mounting a remote system's memory**: By allocating a memory region with `ram-ul` (effectively **"uploading RAM"**), it is possible to mount a remote system's RAM to your local system with minimal latency.
18+
- **Getting insights into memory usage and contents**: By inspecting the "downloaded"/mounted memory with `ram-ul`, you can get insights into the contents of a remote system's memory.
19+
20+
`ram-dl` achieves this by essentially doing the following:
21+
22+
1. Starting a `go-nbd` server _and_ client locally with r3map's [Device API](https://pkg.go.dev/github.com/pojntfx/r3map@main/pkg/device)
23+
2. Connecting the NBD _server_ to a remote fRPC backend provided by `ram-ul`
24+
3. Calling `mkswap`, `swapon` and `swapoff`, which enables paging out to the block device provided by the NBD client and thus to the connected remote fRPC backend.
25+
26+
For most real-world use cases, using [r3map](https://github.com/pojntfx/r3map)'s [Mount API](https://pkg.go.dev/github.com/pojntfx/r3map@main/pkg/mount) or [Migration API](https://pkg.go.dev/github.com/pojntfx/r3map@main/pkg/migration) directly is the better (and much faster) option, but if you just want to see the [Download More RAM](https://knowyourmeme.com/memes/download-more-ram) meme implemented in real life, you've come to the right place!
27+
28+
## Installation
29+
30+
Static binaries are available on [GitHub releases](https://github.com/pojntfx/ram-dl/releases).
31+
32+
On Linux, you can install them like so:
33+
34+
```shell
35+
$ curl -L -o /tmp/ram-dl "https://github.com/pojntfx/ram-dl/releases/latest/download/ram-dl.linux-$(uname -m)"
36+
$ curl -L -o /tmp/ram-ul "https://github.com/pojntfx/ram-dl/releases/latest/download/ram-ul.linux-$(uname -m)"
37+
$ sudo install /tmp/ram-dl /usr/local/bin
38+
$ sudo install /tmp/ram-ul /usr/local/bin
39+
```
40+
41+
You can find binaries for more architectures on [GitHub releases](https://github.com/pojntfx/ram-dl/releases).
42+
43+
## Usage
44+
45+
> TL;DR: "Upload" RAM with `ram-ul`, "download" the RAM with `ram-dl`, done!
46+
47+
### 1. Upload RAM
48+
49+
On a remote (or local) system, first start `ram-ul`. This component exposes a memory region, file or directory as a fRPC server:
50+
51+
```shell
52+
$ ram-ul --size 4294967296
53+
2023/06/30 14:52:12 Listening on :1337
54+
```
55+
56+
### 2. Download RAM
57+
58+
On your local system, start `ram-dl`. This will mount the remote system's exposed memory region, file or directory using fRPC and r3map as swap space, and umount it as soon as you interrupt the app:
59+
60+
```shell
61+
$ sudo ram-dl --raddr localhost:1337
62+
2023/06/30 14:54:22 Connected to localhost:1337
63+
2023/06/30 14:54:22 Ready on /dev/nbd0
64+
```
65+
66+
This should give you an extra 4GB of local memory/swap space, without using up significant local memory (or disk space):
67+
68+
```shell
69+
# Before
70+
$ free -h
71+
total used free shared buff/cache available
72+
Mem: 30Gi 7.9Gi 6.5Gi 721Mi 16Gi 21Gi
73+
Swap: 8.0Gi 0B 8.0Gi
74+
75+
# After
76+
$ free -h
77+
total used free shared buff/cache available
78+
Mem: 30Gi 7.9Gi 6.5Gi 717Mi 16Gi 21Gi
79+
Swap: 11Gi 0B 11Gi
80+
```
81+
82+
🚀 **That's it!** We hope you have fun using `ram-dl`, and if you're interested in more like this, be sure to check out [r3map](https://github.com/pojntfx/r3map)!
83+
84+
## Reference
85+
86+
### ram-dl
87+
88+
```shell
89+
$ ram-dl --help
90+
Usage of ram-dl:
91+
-chunk-size int
92+
Chunk size to use (default 4096)
93+
-chunking
94+
Whether the backend requires to be interfaced with in fixed chunks (default true)
95+
-raddr string
96+
Remote address for the fRPC r3map backend server (default "localhost:1337")
97+
-verbose
98+
Whether to enable verbose logging
99+
```
100+
101+
### ram-ul
102+
103+
```shell
104+
$ ram-ul --help
105+
Usage of ram-ul:
106+
-addr string
107+
Listen address (default ":1337")
108+
-backend string
109+
Backend to use (one of [file memory directory]) (default "file")
110+
-chunk-size int
111+
Chunk size to use (default 4096)
112+
-chunking
113+
Whether the backend requires to be interfaced with in fixed chunks in tests (default true)
114+
-location string
115+
Backend's directory (for directory backend) or file (for file backend) (default "/tmp/ram-ul")
116+
-size int
117+
Size of the memory region or file to allocate (default 4294967296)
118+
-verbose
119+
Whether to enable verbose logging
120+
```
121+
122+
## Acknowledgements
123+
124+
- [pojntfx/go-bd](https://github.com/pojntfx/go-nbd) provides the Go NBD client and server.
125+
- [pojntfx/r3map](https://github.com/pojntfx/r3map) provides the device abstraction layer.
126+
- ["We ACTUALLY downloaded more RAM" by LTT](https://www.youtube.com/watch?v=minxwFqinpw) provided the original idea for this demo.
127+
128+
## Contributing
129+
130+
To contribute, please use the [GitHub flow](https://guides.github.com/introduction/flow/) and follow our [Code of Conduct](./CODE_OF_CONDUCT.md).
131+
132+
To build and start a development version of `ram-dl` locally, run the following:
133+
134+
```shell
135+
$ git clone https://github.com/pojntfx/ram-dl.git
136+
$ cd ram-dl
137+
$ make depend
138+
$ make && sudo make install
139+
$ ram-ul
140+
# In another terminal
141+
$ sudo ram-dl
142+
```
143+
144+
Have any questions or need help? Chat with us [on Matrix](https://matrix.to/#/#ram-dl:matrix.org?via=matrix.org)!
12145
13146
## License
14147
15148
ram-dl (c) 2023 Felicitas Pojtinger and contributors
16149
17-
SPDX-License-Identifier: Apache-2.0
150+
SPDX-License-Identifier: AGPL-3.0

‎cmd/ram-dl/main.go

+2
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,8 @@ func main() {
4444
}
4545
defer client.Close()
4646

47+
log.Println("Connected to", *raddr)
48+
4749
devPath, err := utils.FindUnusedNBDDevice()
4850
if err != nil {
4951
panic(err)

‎cmd/ram-dl-server/main.go ‎cmd/ram-ul/main.go

+1-1
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ func main() {
4242
knownBackendTypes,
4343
),
4444
)
45-
location := flag.String("location", filepath.Join(os.TempDir(), "ram-dl-server"), "Backend's directory (for directory backend) or file (for file backend)")
45+
location := flag.String("location", filepath.Join(os.TempDir(), "ram-ul"), "Backend's directory (for directory backend) or file (for file backend)")
4646
chunking := flag.Bool("chunking", true, "Whether the backend requires to be interfaced with in fixed chunks in tests")
4747

4848
verbose := flag.Bool("verbose", false, "Whether to enable verbose logging")

0 commit comments

Comments
 (0)