Skip to content

Commit 90518df

Browse files
committed
initial WasmDecoder
1 parent 3a0d407 commit 90518df

27 files changed

+2445
-276
lines changed

.gitignore

+2
Original file line numberDiff line numberDiff line change
@@ -6,3 +6,5 @@
66
/.nyc_output
77
/benchmark
88
/.vscode
9+
/wasm/sixel.wasm
10+
/src/wasm.json

README.md

+57-14
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,18 @@
22

33
SIXEL image decoding / encoding library for node and the browser.
44

5+
56
### Decoding
67

7-
For decoding the library provides a class `SixelDecoder` with the following properties:
8+
For decoding the library provides two decoder classes with slightly different semantics.
9+
10+
11+
#### DefaultDecoder
12+
13+
The `DefaultDecoder` is a general purpose decoder written in Typescript. It can decode level1 and level2
14+
SIXEL data with reasonable speed and without any further preparations. It currently only supports _printerMode_(_terminalMode_ with proper shared/private palette semantics is planned).
15+
16+
Properties of `DefaultDecoder`:
817

918
- `constructor(public fillColor: RGBA8888 = DEFAULT_BACKGROUND, public palette: RGBA8888[] = PALETTE_VT340_COLOR, public paletteLimit: number = 65536)`
1019
Creates a new SIXEL image. The optional `fillColor` (default black) is used to fill
@@ -63,6 +72,20 @@ For decoding the library provides a class `SixelDecoder` with the following prop
6372
`sx`, `sy` (offset in source) and `swidth`, `sheight` (area in source) for cropping/clipping. `fillColor` has the same
6473
meaning as in the constructor, explicit setting it to 0 will leave non encoded pixels unaltered (pixels, that were not colored in the SIXEL data). This can be used for a transparency like effect (background/previous pixel value will remain). Returns the altered `target`.
6574

75+
76+
#### WasmDecoder
77+
78+
The `WasmDecoder` is a level 2 only decoder written in C and compiled to WebAssembly. While it can decode image data much faster (see benchmarks below), it imposes several usage restrictions:
79+
80+
- limited to 1536 x 1536 pixels and 4096 palette colors (compile time settings)
81+
- level 2 only, needs proper pixel dimensions (can be obtained from raster attributes with `DimensionDecoder`)
82+
- always truncates images to pixel dimensions (not spec conform)
83+
84+
Other than the default decoder, `WasmDecoder` is meant to be re-used with follow-up images, which lowers the need to spawn webassembly instances.
85+
86+
TODO: document properties + canUsewasm + DimensionDecoder
87+
88+
6689
### Encoding
6790

6891
For encoding the library provides the following properties:
@@ -91,6 +114,7 @@ For encoding the library provides the following properties:
91114
Note that a SIXEL escape sequences changes the operation mode of a terminal,
92115
forgetting the finalizer might leave the terminal in an unrecoverable state.
93116

117+
94118
### Convenient Properties
95119

96120
Furthermore the library exposes some convenient properties:
@@ -133,28 +157,47 @@ The test image repeats the palette image 6 times to form a 640x480 image with 51
133157

134158
Results:
135159
```
136-
Context "lib/index.benchmark.js"
160+
Context "./lib/index.benchmark.js"
137161
Context "testimage"
138162
Context "pixel transfer"
139-
Case "toPixelData - with fillColor" : 10 runs - average runtime: 1.86 ms
140-
Case "toPixelData - without fillColor" : 10 runs - average runtime: 1.28 ms
141-
Context "decode"
142-
Case "decode" : 10 runs - average runtime: 4.22 ms
143-
Case "decodeString" : 10 runs - average runtime: 6.60 ms
144-
Case "decode + pixel transfer" : 10 runs - average runtime: 4.33 ms
163+
Case "toPixelData - with fillColor" : 20 runs - average runtime: 1.42 ms
164+
Case "toPixelData - without fillColor" : 20 runs - average runtime: 1.12 ms
165+
Context "decode (DefaultDecoder)"
166+
Case "decode" : 20 runs - average runtime: 4.34 ms
167+
Case "decodeString" : 20 runs - average runtime: 4.62 ms
168+
Case "decode + pixel transfer" : 20 runs - average runtime: 3.39 ms
169+
Context "decode (WasmDecoder)"
170+
Case "decode" : 20 runs - average runtime: 1.35 ms
171+
Case "decodeString" : 20 runs - average runtime: 1.81 ms
145172
Context "encode"
146-
Case "sixelEncode" : 10 runs - average runtime: 25.92 ms
147-
Context "decode - testfiles"
148-
Case "test1_clean.sixel" : 10 runs - average runtime: 17.96 ms
149-
Case "test2_clean.sixel" : 10 runs - average runtime: 8.01 ms
150-
Case "sampsa1_clean.sixel" : 10 runs - average runtime: 43.30 ms
173+
Case "sixelEncode" : 20 runs - average runtime: 25.10 ms
174+
Context "decode - testfiles (DefaultDecoder)"
175+
Case "test1_clean.sixel" : 20 runs - average runtime: 16.75 ms
176+
Case "test1_clean.sixel" : 20 runs - average throughput: 37.70 MB/s
177+
Case "test2_clean.sixel" : 20 runs - average runtime: 7.23 ms
178+
Case "test2_clean.sixel" : 20 runs - average throughput: 45.48 MB/s
179+
Case "sampsa_reencoded_clean.six" : 20 runs - average runtime: 16.53 ms
180+
Case "sampsa_reencoded_clean.six" : 20 runs - average throughput: 39.57 MB/s
181+
Case "FullHD 12bit noise" : 20 runs - average runtime: 228.52 ms
182+
Case "FullHD 12bit noise" : 20 runs - average throughput: 67.84 MB/s
183+
Context "decode - testfiles (WasmDecoder)"
184+
Case "test1_clean.sixel" : 20 runs - average runtime: 9.99 ms
185+
Case "test1_clean.sixel" : 20 runs - average throughput: 61.25 MB/s
186+
Case "test2_clean.sixel" : 20 runs - average runtime: 4.16 ms
187+
Case "test2_clean.sixel" : 20 runs - average throughput: 76.79 MB/s
188+
Case "sampsa_reencoded_clean.six" : 20 runs - average runtime: 10.54 ms
189+
Case "sampsa_reencoded_clean.six" : 20 runs - average throughput: 61.23 MB/s
190+
Case "FullHD 12bit noise" : 20 runs - average runtime: 100.77 ms
191+
Case "FullHD 12bit noise" : 20 runs - average throughput: 153.86 MB/s
151192
```
193+
`WasmDecoder` is roughly 1.5x - 2.3x faster than `DefaultDecoder`.
194+
TODO...
152195

153-
Note that encoding is much more expensive than decoding and prolly should be called within a webworker or child process.
154196

155197
### Status
156198
Currently beta, still more tests to come.
157199

200+
158201
### References
159202

160203
While being quite common in the DEC ecosystem in the 80s (even used for printer protocols), SIXEL references are very limited these days. The closest to a specification we have can be found in the Video Systems Reference Manual ([DEC STD 070](http://www.bitsavers.org/pdf/dec/standards/EL-SM070-00_DEC_STD_070_Video_Systems_Reference_Manual_Dec91.pdf#page=908), p. 908-933). Also see [Sixel Graphics](https://www.vt100.net/docs/vt3xx-gp/chapter14.html) on vt100.net, which gives a quick overview. For implementation the old usenet article "[All About SIXELs](https://www.digiater.nl/openvms/decus/vax90b1/krypton-nasa/all-about-sixels.text)" was very helpful.

index.html

+63-12
Original file line numberDiff line numberDiff line change
@@ -51,8 +51,12 @@
5151
<br><br>
5252
<span id="stats"></span>
5353
<br><br>
54+
Default decoder:<br>
5455
<canvas id="output" style="border: 1px solid black"></canvas>
5556
<br><br>
57+
L2 decoder (wasm):<br>
58+
<canvas id="output_wasm" style="border: 1px solid black"></canvas>
59+
<br><br>
5660
Reencoded with img2sixel:<br>
5761
<canvas id="output2" style="border: 1px solid black"></canvas>
5862
<script src="/dist/bundle.js"></script>
@@ -61,6 +65,39 @@
6165

6266
let drawHandle = null;
6367
let imgS = null;
68+
let imgWasm = null;
69+
let wasmDecoder = null;
70+
sixel.WasmDecoderAsync().then(dec => wasmDecoder = dec);
71+
72+
function decodeWasm(bytes) {
73+
const start = new Date();
74+
const dim = new sixel.DimensionDecoder().decode(bytes);
75+
console.log('L2 attrs:', dim);
76+
if (sixel.canUseWasm(dim.width, dim.height, 4096)) {
77+
wasmDecoder.init(
78+
dim.width,
79+
dim.height,
80+
sixel.toRGBA8888(...hexColorToRGB(document.getElementById('fillColor').value))
81+
);
82+
wasmDecoder.decode(bytes);
83+
}
84+
drawImageL2(wasmDecoder.data32, wasmDecoder.width, wasmDecoder.height);
85+
console.log('L2 conversion:', (new Date()) - start);
86+
}
87+
88+
function drawImageL2(data, width, height) {
89+
if (!height || !width) {
90+
return;
91+
}
92+
const canvas = document.getElementById('output_wasm');
93+
const ctx = canvas.getContext('2d');
94+
// resize canvas to show full image
95+
canvas.width = width;
96+
canvas.height = height;
97+
const target = new ImageData(width, height);
98+
new Uint32Array(target.data.buffer).set(data);
99+
ctx.putImageData(target, 0, 0);
100+
}
64101

65102
/**
66103
* example how to get the img data
@@ -75,7 +112,8 @@
75112
canvas.width = img.width;
76113
canvas.height = img.height;
77114
// grab imagedata
78-
const target = ctx.getImageData(0, 0, img.width, img.height);
115+
// const target = ctx.getImageData(0, 0, img.width, img.height);
116+
const target = new ImageData(img.width, img.height);
79117
img.toPixelData(
80118
// target metrics
81119
target.data, img.width, img.height,
@@ -90,21 +128,33 @@
90128
ctx.putImageData(target, 0, 0);
91129

92130
// test encoding by re-encoding the output above
93-
const reEncoded = sixel.image2sixel(target.data, img.width, img.height);
94-
const six2 = new sixel.SixelDecoder();
95-
six2.decodeString(reEncoded.slice(7, -2)); // strip off enclosing escape sequence
96-
const canvas2 = document.getElementById('output2');
97-
canvas2.width = six2.width;
98-
canvas2.height = six2.height;
99-
const ctx2 = canvas2.getContext('2d');
100-
const target2 = ctx.getImageData(0, 0, six2.width, six2.height);
101-
six2.toPixelData(target2.data, six2.width, six2.height);
102-
ctx2.putImageData(target2, 0, 0);
131+
//const reEncoded = sixel.image2sixel(target.data, img.width, img.height);
132+
//const dim = new sixel.DimensionDecoder().decodeString(reEncoded.slice(7, -2));
133+
//if (sixel.canUseWasm(dim.width, dim.height, 4096)) {
134+
// wasmDecoder.init(dim.width, dim.height, 0)
135+
// wasmDecoder.decodeString(reEncoded.slice(7, -2));
136+
// const canvas2 = document.getElementById('output2');
137+
// canvas2.width = dim.width;
138+
// canvas2.height = dim.height;
139+
// const ctx2 = canvas2.getContext('2d');
140+
// const target2 = ctx.getImageData(0, 0, dim.width, dim.height);
141+
// new Uint32Array(target2.data.buffer).set(wasmDecoder.data32);
142+
// ctx2.putImageData(target2, 0, 0);
143+
//} else {
144+
// const six2 = new sixel.SixelDecoder();
145+
// six2.decodeString(reEncoded.slice(7, -2)); // strip off enclosing escape sequence
146+
// const canvas2 = document.getElementById('output2');
147+
// canvas2.width = six2.width;
148+
// canvas2.height = six2.height;
149+
// const ctx2 = canvas2.getContext('2d');
150+
// const target2 = ctx.getImageData(0, 0, six2.width, six2.height);
151+
// six2.toPixelData(target2.data, six2.width, six2.height);
152+
// ctx2.putImageData(target2, 0, 0);
153+
//}
103154
}
104155

105156
function hexColorToRGB(color) {
106157
const value = parseInt(color.slice(1), 16);
107-
console.log();
108158
return [
109159
(value >> 16) & 0xFF,
110160
(value >> 8) & 0xFF,
@@ -132,6 +182,7 @@
132182
} else {
133183
const response = await fetch('/testfiles/' + s);
134184
const bytes = new Uint8Array(await response.arrayBuffer());
185+
decodeWasm(bytes);
135186
start = new Date();
136187
if (document.getElementById('slow').checked) {
137188
let localHandle = drawHandle = setInterval(() => drawImage(img), 100);

package.json

+13-14
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,8 @@
1111
"start": "webpack-cli && http-server",
1212
"prepublish": "npm run tsc",
1313
"coverage": "nyc --reporter=lcov --reporter=text --reporter=html npm test",
14-
"benchmark": "xterm-benchmark $*"
14+
"benchmark": "xterm-benchmark $*",
15+
"build-wasm": "bash -c wasm/build.sh"
1516
},
1617
"keywords": [
1718
"sixel",
@@ -27,24 +28,22 @@
2728
"dependencies": {},
2829
"devDependencies": {
2930
"@istanbuljs/nyc-config-typescript": "^1.0.1",
30-
"@types/chai": "^4.2.14",
31-
"@types/mocha": "^8.0.4",
32-
"@types/node": "^12.12.37",
33-
"canvas": "^2.6.1",
34-
"chai": "^4.2.0",
31+
"@types/mocha": "^8.2.2",
32+
"@types/node": "^14.17.3",
33+
"canvas": "^2.8.0",
3534
"http-server": "^0.12.3",
36-
"mocha": "^8.2.1",
35+
"mocha": "^9.0.1",
3736
"node-ansiparser": "^2.2.0",
3837
"nyc": "^15.1.0",
39-
"open": "^7.3.0",
38+
"open": "^8.2.1",
4039
"rgbquant": "^1.1.2",
4140
"source-map-support": "^0.5.19",
42-
"ts-loader": "^8.0.11",
43-
"ts-node": "^9.0.0",
41+
"ts-loader": "^9.2.3",
42+
"ts-node": "^10.0.0",
4443
"tslint": "^6.1.3",
45-
"typescript": "^4.0.5",
46-
"webpack": "^5.5.1",
47-
"webpack-cli": "^4.2.0",
48-
"xterm-benchmark": "^0.2.0"
44+
"typescript": "^4.3.4",
45+
"webpack": "^5.40.0",
46+
"webpack-cli": "^4.7.2",
47+
"xterm-benchmark": "^0.2.1"
4948
}
5049
}

0 commit comments

Comments
 (0)