Skip to content

Commit f663bf4

Browse files
committed
issue-193 TSDBs integration
Squashed commit of the following: commit 5ea4757fb592f4e98ebcdd077b4dbe404ebfb82c Author: mohamed.yacoubi <[email protected]> Date: Mon Nov 25 15:52:59 2024 +0100 feat: making tsdb example available through docker commit 152ad71521ad85f2197adda2ce767e60042bd6b2 Author: mohamed.yacoubi <[email protected]> Date: Mon Nov 25 14:33:52 2024 +0100 refactor: use cases rearrangement commit ed848566c4782fb34beeb15971892a3771ee2ead Author: mohamed.yacoubi <[email protected]> Date: Mon Nov 25 11:36:38 2024 +0100 fix: getting yarn.lock out of the .gitignore file commit e888c5f8ee99bd97b1d5159cbe709d683b938dce Author: mohamed.yacoubi <[email protected]> Date: Mon Nov 25 11:17:15 2024 +0100 fix: docker compose network_mode commit 6c91ea1e68d73ac1d029f04804fe71b36bd59d58 Merge: 48b01d6b 62219e0c Author: mohamed.yacoubi <[email protected]> Date: Wed Nov 20 12:02:52 2024 +0100 refactor: moving all services to one single folder refactor: upgrading docker compose and Dockerfile with nginx refactor: install/run-services.sh instead of install/run-local-monitoring-server.sh refactor: editing package.json with new pre scripts ci: setting all ci/cd jobs commit 62219e0c0832f37f97adfb76b31ea353e2452b4e Author: mohamed.yacoubi <[email protected]> Date: Tue Nov 19 16:31:21 2024 +0100 fix: deleting unnecessary volumes commit c4b5e7019fc60d68a367c49b3b90aded51a95974 Author: mohamed.yacoubi <[email protected]> Date: Tue Nov 19 15:24:09 2024 +0100 feat: adding local monitoring server inside main repo fix: configuring docker permissions and volumes to read system data commit 48b01d6bd79629f97c4bc196b61c7e7956fbf01e Author: mohamed.yacoubi <[email protected]> Date: Wed Oct 16 10:54:08 2024 +0200 refactor: mutli-viewport code refactoring increasing max-active-webgl-contexts commit 58b5a951993a0749e8d26bef235fb26f923fd21a Author: mohamed.yacoubi <[email protected]> Date: Mon Oct 14 16:27:56 2024 +0200 test: multiviewport benchmark commit 946e4b3f293202128b94b22a6829bec0d2294d77 Author: mohamed.yacoubi <[email protected]> Date: Thu Oct 10 15:21:10 2024 +0200 test: benchmark use cases commit 1116de247e4b4bd27e94f2b78ff8c897fe513730 Author: mohamed.yacoubi <[email protected]> Date: Thu Oct 10 15:19:33 2024 +0200 fix: build-storybook fix commit 2f66a11d735d9952642e7fad99f2b323ccea6ddb Author: mohamed.yacoubi <[email protected]> Date: Thu Oct 10 15:18:43 2024 +0200 refactor: deleting extra spaces commit c48b3e71b657b1a5e7472e93ba2654f9871b43f9 Author: mohamed.yacoubi <[email protected]> Date: Fri Oct 4 17:24:16 2024 +0200 refactor: adaptation with mutli-viewport commit f1b7caebd10df15ea5b43116dc0ea8bc89ee4990 Author: mohamed.yacoubi <[email protected]> Date: Fri Sep 27 15:01:24 2024 +0200 refactor: webWorkersHTTP on new declaration refactor: overall code refactoring commit 3505c1319af5bd9a5ad3bc8a8066f313063c0e0a Author: mohamed.yacoubi <[email protected]> Date: Fri Sep 27 10:44:04 2024 +0200 fix: useBackend param fix on update data function commit b30b992a363a232b4bdfb71e2cf9091aa1054258 Author: mohamed.yacoubi <[email protected]> Date: Thu Sep 26 17:15:37 2024 +0200 feat: multi viewport palindrome on storybook commit e11ce74cab13348a252f051d1d6475d79f5e8472 Author: mohamed.yacoubi <[email protected]> Date: Mon Sep 23 16:44:39 2024 +0200 ci: changing usecase commit 122f00c7a7e43427ff5742c10513522d863e8d73 Author: mohamed.yacoubi <[email protected]> Date: Mon Sep 23 16:32:43 2024 +0200 ci: setting benchmark to headless commit ef973adca06c25231dce345ced1fa15981d0060c Author: mohamed.yacoubi <[email protected]> Date: Mon Sep 23 16:25:18 2024 +0200 feat: mutli viewport palindrome feat: benchmark mutli viewport refactor: refactoring old external api data to match the current dev commit 228e27c56043271bfdbe501763f5d8f25a169fd7 Author: mohamed.yacoubi <[email protected]> Date: Wed Sep 11 16:55:35 2024 +0200 ci: recovering old jobs commit abbedff6daa9a6186b4e91f3d2142ac4b07f0823 Author: mohamed.yacoubi <[email protected]> Date: Wed Sep 11 16:42:06 2024 +0200 ci: refactoring remote-data-benchmark job commit 98063f19b99fd16060ae1b571c5e9ce640dbbf4b Author: mohamed.yacoubi <[email protected]> Date: Wed Sep 11 16:33:35 2024 +0200 ci: changing table title commit ac4a7d366d3f813020212500e12abca0d9277149 Author: mohamed.yacoubi <[email protected]> Date: Wed Sep 11 16:19:06 2024 +0200 feat: deleting mocky and serving local files commit 7c8ff70b1a57d147d909f38d4b760e62fab3c1d0 Author: mohamed.yacoubi <[email protected]> Date: Wed Sep 11 16:18:48 2024 +0200 ci: fixing benchmark commit 17fc96986c699dc670a4e889a3bd0b6f58339be1 Author: mohamed.yacoubi <[email protected]> Date: Wed Sep 11 12:12:35 2024 +0200 ci: adding backend to project repo and debugging commit 11394cb1c34373500fdd639edfcc573ef36fb1a6 Author: mohamed.yacoubi <[email protected]> Date: Wed Sep 11 10:25:09 2024 +0200 ci: adding debug messages commit c688b2df02b7edfa1787c9133791631c27923d58 Author: mohamed.yacoubi <[email protected]> Date: Wed Sep 11 09:50:55 2024 +0200 ci: exporting relevent env variables commit 7e17c3569434454f7210c374683ae0ac706a38d8 Author: mohamed.yacoubi <[email protected]> Date: Tue Sep 10 17:43:31 2024 +0200 ci: testing before_secript commit 263fff7c7f282100c3bc76e41bc4c46206bd1884 Author: mohamed.yacoubi <[email protected]> Date: Tue Sep 10 17:33:17 2024 +0200 ci: debugging env variable commit 18051af85208cca18e381b018cce9ccb19484785 Author: mohamed.yacoubi <[email protected]> Date: Tue Sep 10 17:19:09 2024 +0200 ci: testing with ghcr.io/puppeteer/puppeteer:latest commit 7289aee626165cb867102bfcd978550eb6986d24 Author: mohamed.yacoubi <[email protected]> Date: Tue Sep 10 17:17:11 2024 +0200 ci: testing puppeteer:latest commit 51776cfb5ce0d6d72cd816eb7960cdeeee3dca9c Author: mohamed.yacoubi <[email protected]> Date: Tue Sep 10 17:04:45 2024 +0200 ci: changing image to node:18 commit 4d89377e1bf0e24c52e8c11e771949e7b6466216 Author: mohamed.yacoubi <[email protected]> Date: Tue Sep 10 16:54:30 2024 +0200 ci: keeping the default image commit f0a992f19e27e6bf8289ad10a5b6d2e4d02c0250 Author: mohamed.yacoubi <[email protected]> Date: Tue Sep 10 16:22:50 2024 +0200 ci: adding sleep to debug commit 695c3b621cedf64116448b5a92d659b0daf45cb8 Author: mohamed.yacoubi <[email protected]> Date: Tue Sep 10 16:17:00 2024 +0200 ci: trying cimg/node:current-browsers commit 1c250353de216e7d8276800dabe60171aef95d77 Author: mohamed.yacoubi <[email protected]> Date: Tue Sep 10 16:09:59 2024 +0200 ci: ignore scripts on yarn install commit a09bd4315990962543e9c14a87536793c3a2e29f Author: mohamed.yacoubi <[email protected]> Date: Tue Sep 10 16:07:59 2024 +0200 fix: typo image name commit ac39b50934cd7318bd3db4e76d6d4d325f9990b3 Author: mohamed.yacoubi <[email protected]> Date: Tue Sep 10 16:06:03 2024 +0200 ci: debugging and trying new image commit 13a39ae2181279443ce511b1a13cfaba0f3e3730 Author: mohamed.yacoubi <[email protected]> Date: Tue Sep 10 15:39:53 2024 +0200 ci: fixing sleep commit fb767298633613adc9fd7e57b8a242fa8f961e0e Author: mohamed.yacoubi <[email protected]> Date: Tue Sep 10 15:37:02 2024 +0200 ci: sleep to access runner for debug commit 045aef10fd1b7192501cf22207be19f3b66da10f Author: mohamed.yacoubi <[email protected]> Date: Tue Sep 10 15:33:40 2024 +0200 ci: debugging puppeteer commit 0fcd4eb9ee83285af43ff1024dca2f7a11efd1e7 Author: mohamed.yacoubi <[email protected]> Date: Tue Sep 10 15:27:00 2024 +0200 ci: testing commit c89285d164668f1d7aca74e3ce8f4e09fe09bbd2 Author: mohamed.yacoubi <[email protected]> Date: Mon Sep 9 17:03:34 2024 +0200 feat: adding benchmarkInitialData commit b3ec8a30921562a93e6c8f7c25d9cde46bb4baff Author: mohamed.yacoubi <[email protected]> Date: Thu Sep 5 14:23:46 2024 +0200 feat: improving remote data benchmark commit 0dd7124fb81d2ebe68f1ad25a62ec97e4ecf20fb Author: mohamed.yacoubi <[email protected]> Date: Thu Sep 5 14:23:28 2024 +0200 feat: data validator implementation commit bab07773b6e96352b25a520374506a8df07f2a6b Author: mohamed.yacoubi <[email protected]> Date: Thu Aug 29 17:46:43 2024 +0200 feat: fetching in real time per metric, and per data provider commit f186f2723c9fc3ba20774aea23a7107e4a34a661 Author: mohamed.yacoubi <[email protected]> Date: Wed Aug 28 17:38:34 2024 +0200 feat: deleting old api and improving new api commit 2fa549096a41659bea968b9cc0f35a41bcdf95eb Author: mohamed.yacoubi <[email protected]> Date: Tue Aug 27 09:52:16 2024 +0200 feat: changing metrics declaration api / merging with standard api commit 9b92ce48afd48e327df60bd646bb0b10cb23348a Merge: 67293d35 1eb54882 Author: mohamed.yacoubi <[email protected]> Date: Tue Aug 20 13:39:44 2024 +0200 feat: remote data integration commit 67293d352af5bfb0fc0be52c05cb0012b446d863 Author: mohamed.yacoubi <[email protected]> Date: Tue Aug 13 15:03:58 2024 +0200 refactor: refactoring code commit 1eb548821a70efb3ec768160a7bed2dc41e1c152 Author: mohamed.yacoubi <[email protected]> Date: Mon Aug 12 16:19:41 2024 +0200 feat: json reconciliator prototype commit a3de30bc70ef105f74284ff282f5400dc8d962a7 Author: mohamed.yacoubi <[email protected]> Date: Thu Aug 8 16:53:27 2024 +0200 feat: native prometheus / influxdb support fix: displaying float numbers ci: fixing tsdbService location ci: updating paths ci: fixing relative paths delete: yarn.lock after failed ci job ci: refactoring remote data benchmark ci: adding Influx token refactor: small refactoring of scripts fix: refactoring gitlab-ci.yaml deleting unused Vagrantfile docs: updating readme.md test: adding use cases number constant ci: refactoring gitlab-ci.yaml
1 parent 7a8b36b commit f663bf4

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

72 files changed

+25224
-367
lines changed

Diff for: .babelrc

-3
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,5 @@
11
{
22
"plugins": [
33
"istanbul"
4-
],
5-
"ignore": [
6-
"**/webCollectors/**"
74
]
85
}

Diff for: .gitignore

+2-1
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,9 @@
55
node_modules
66
.cache
77
*log
8-
yarn.lock
98
.parcel-cache
109
.vagrant
10+
yarn.lock
1111

1212
# IDE
1313
*.swp
@@ -22,3 +22,4 @@ dist
2222
cypress/match-screenshots
2323
.nyc_output/*
2424
coverage/*
25+
**/snapshots/**

Diff for: .gitlab-ci.yml

+57-3
Original file line numberDiff line numberDiff line change
@@ -31,10 +31,17 @@ default:
3131
build-dev:
3232
stage: build
3333
before_script:
34-
- apk update
35-
- apk add bash
36-
- apk add git
34+
- export CI_BENCHMARK=remote_data
35+
- export PALINDROME_TYPE=basic
3736
script:
37+
- yarn install --ignore-scripts
38+
- |
39+
yarn parcel dev/index.html --no-cache &
40+
until curl -s http://localhost:1234 > /dev/null; do
41+
echo "Waiting for Parcel server..."
42+
sleep 1
43+
done
44+
- cd services/tsdbService
3845
- yarn install
3946
- yarn build
4047
artifacts:
@@ -74,6 +81,53 @@ cypress-test:
7481
- for i in 1 2; do yarn cypress:run; done
7582
allow_failure: false
7683

84+
.launch-benchmark-remote-data:
85+
stage: test
86+
image: ghcr.io/puppeteer/puppeteer:latest
87+
script:
88+
- |
89+
for file in data-examples/*; do
90+
if [ -f "$file" ]; then
91+
if grep -q "localhost" "$file"; then
92+
echo "Processing $file"
93+
sed -i "s/localhost/${INFLUX_PROMETHEUS_SERVER}/g" "$file"
94+
sed -i "s/'token'/'${INFLUX_TOKEN}'/g" "$file"
95+
fi
96+
fi
97+
done
98+
- yarn install --ignore-scripts
99+
- |
100+
yarn parcel dev/index.html --no-cache &
101+
until curl -s http://localhost:1234 > /dev/null; do
102+
echo "Waiting for Parcel server..."
103+
sleep 1
104+
done
105+
- cd services/tsdbService
106+
- yarn install
107+
- |
108+
yarn dev &
109+
until curl -s http://localhost:9009 > /dev/null; do
110+
echo "Waiting for palindrome service..."
111+
sleep 1
112+
done
113+
- cd ../..
114+
- node "$BENCHMARK_SCRIPT"
115+
allow_failure: false
116+
117+
initial_remote-data-benchmark:
118+
extends: .launch-benchmark-remote-data
119+
before_script:
120+
- export CI_BENCHMARK=remote_data
121+
- export PALINDROME_TYPE=basic
122+
- export BENCHMARK_SCRIPT=./scripts/remoteDataBenchmark/remoteDataBenchmark.js
123+
124+
remote-data-update-multiviewport-benchmark:
125+
extends: .launch-benchmark-remote-data
126+
before_script:
127+
- export CI_BENCHMARK=multi-viewports
128+
- export PALINDROME_TYPE=basic
129+
- export BENCHMARK_SCRIPT=./scripts/remoteDataBenchmark/multiViewportBenchmark.js
130+
77131
benchmark:
78132
image: ubuntu:20.04
79133
stage: test

Diff for: .storybook/main.js

+9
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
module.exports = {
22
"stories": ["../stories/**/*.stories.@(js|jsx|ts|tsx)"],
3+
"staticDirs": ['../static'],
34
"addons": ["@storybook/addon-links", "@storybook/addon-essentials", {
45
name: "storybook-addon-turbo-build",
56
options: {
@@ -10,5 +11,13 @@ module.exports = {
1011
postcss: false
1112
},
1213
framework: "@storybook/html",
14+
webpackFinal: async (config) => {
15+
config.optimization.minimizer.forEach((plugin) => {
16+
if (plugin.constructor.name === 'ESBuildMinifyPlugin') {
17+
plugin.options.target = 'esnext'; // or 'es2017'
18+
}
19+
});
20+
return config;
21+
},
1322
};
1423

Diff for: Dockerfile

+1-1
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ RUN yarn add global parcel-bundler
55
COPY package.json .
66
RUN yarn install --ignore-scripts
77
COPY . .
8-
RUN yarn build
8+
RUN yarn build:local
99

1010
# Build storybook stage
1111
FROM node:20 AS build-storybook-stage

Diff for: README.md

+27-7
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ Feedbacks are more than welcome !**_
1919

2020
## Motivation
2121

22-
Idea behind this project is to go one step further current monitoring and dashboards solutions, by enabling a scalable
22+
The idea behind this project is to go one step further current monitoring and dashboards solutions, by enabling a scalable
2323
and user oriented, 3D monitoring probe for multi-dimensional and heterogeneous sets of data points. Compatible use cases
2424
are various, from comparing system metrics with external indicators, to stacking up multi-tenancy informations groups
2525
for measuring differences or similarities, ```Palindrome.js``` can both be used as a live UI component for a larger BI
@@ -47,7 +47,7 @@ sudo apt install nodejs
4747
**Note:** An already built version of Palindrome.js is ready to be integrated without any Node.js setup, two files can be found inside the `/artifacts` folder: `palindrome.dev.js`, which includes a sidebar, and `palindrome.js`, which represents the basic version of Palindrome.js without a sidebar. The [documentation](../../wikis/Single-file-distribution) should be followed for more integration details.
4848
## Setup
4949

50-
This project uses ```three.js``` as its 3D renderer, ```yarn``` as the dependency manager, ```storybook``` for
50+
This project uses ```three.js``` as its 3D renderer, ```yarn``` as the dependency manager, ```Storybook``` for
5151
functional testing and ```parcel-bundler``` as the stand-alone packager.
5252

5353
First install the project dependencies :
@@ -56,6 +56,16 @@ First install the project dependencies :
5656
yarn install
5757
```
5858

59+
### TSDBs Setup
60+
61+
Palindrome.js includes use cases that require metrics data coming from `InfluxDB`, `Prometheus`, `Node Exporter`, and `Telegraf`. You can configure your remote data use cases by following the documentation [here](../../wikis/Remote-Data-Integration).
62+
63+
If you haven't configured your own TSDBs in Palindrome.js, you need to run the following command to launch the TSDBs with the default configuration:
64+
65+
```
66+
docker compose -f ./services/tsdbService/tsdbSetup/docker-compose.yml up -d
67+
```
68+
5969
## Default HTML
6070

6171
Then, run the default html session for an interactive session :
@@ -78,10 +88,18 @@ yarn storybook
7888

7989
## Docker
8090

81-
You can as well, run Palindrome.js on docker :
91+
Palindrome.js can also be run in Docker.
92+
93+
First, start the TSDB servers by running the following command (if you haven't configured your own TSDBs in Palindrome.js):
94+
95+
```
96+
docker compose -f ./services/tsdbService/tsdbSetup/docker-compose.yml up -d
97+
```
98+
99+
Next, launch Palindrome.js by running:
82100

83101
```
84-
docker-compose up
102+
docker compose up
85103
```
86104

87105
## Documentation
@@ -90,8 +108,10 @@ Click [here](../../wikis/home) to visit our wiki page containing the documentati
90108

91109
## Demo Environments
92110

93-
- Dev Environment : https://smile-sa.github.io/palindrome.js
94-
- Storybook Environment : https://smile-sa.github.io/palindrome.js/storybook
111+
You can explore the following demo environments:
112+
113+
- [Dev Environment](https://smile-sa.github.io/palindrome.js)
114+
- [Storybook Environment](https://smile-sa.github.io/palindrome.js/storybook)
95115

96116

97117
## Contribute
@@ -100,7 +120,7 @@ Simply open a pull request over the repository to describe your changes.
100120

101121
## Credits
102122

103-
- Mohamed Ali YACOUBI @yacoubii
123+
- Mohamed Ali Yacoubi @yacoubii
104124
- Koku Ulrich Gblokpo @koku-ulrich.gblokpo
105125
- Damien Gilles @gillesdami
106126
- JonRiv (author) @JonRiv

Diff for: Vagrantfile

-39
This file was deleted.

0 commit comments

Comments
 (0)