Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improvement-16125][dolphinscheduler-dist] Reduce binary distribution tarball file size #16179

Closed
wants to merge 11 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions deploy/docker/.env
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@
# specific language governing permissions and limitations
# under the License.
#
HUB=apache
TAG=3.2.2
HUB=ghcr.io/apache/dolphinscheduler
TAG=latest

TZ=Asia/Shanghai
DATABASE=postgresql
Expand Down
2 changes: 1 addition & 1 deletion deploy/kubernetes/dolphinscheduler/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ type: application

# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
version: 3.2.2
version: 3.1.0

# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application.
Expand Down
2 changes: 1 addition & 1 deletion deploy/kubernetes/dolphinscheduler/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ image:
# -- Docker image repository for the DolphinScheduler
registry: apache
# -- Docker image version for the DolphinScheduler
tag: 3.2.2
tag: latest
# -- Image pull policy. Options: Always, Never, IfNotPresent
pullPolicy: "IfNotPresent"
# -- Specify a imagePullSecrets
Expand Down
600 changes: 300 additions & 300 deletions docs/configs/docsdev.js

Large diffs are not rendered by default.

42 changes: 21 additions & 21 deletions docs/docs/en/contribute/release.md
Original file line number Diff line number Diff line change
Expand Up @@ -340,7 +340,7 @@ svn --username="${A_USERNAME}" commit -m "release ${VERSION}"

#### Check source package

Decompress `apache-dolphinscheduler-3.2.2-src.tar.gz` then check the following items:
Decompress `apache-dolphinscheduler-<VERSION>-src.tar.gz` then check the following items:

- Check whether source tarball is oversized for including nonessential files
- `LICENSE` and `NOTICE` files exist
Expand All @@ -353,7 +353,7 @@ Decompress `apache-dolphinscheduler-3.2.2-src.tar.gz` then check the following i

#### Check binary packages

Decompress `apache-dolphinscheduler-3.2.2-bin.tar.gz` to check the following items:
Decompress `apache-dolphinscheduler-<VERSION>-bin.tar.gz` to check the following items:

- `LICENSE` and `NOTICE` files exist
- Correct year in `NOTICE` file
Expand Down Expand Up @@ -389,23 +389,23 @@ Announce the vote result: send the result vote e-mail to `[email protected]
Title:

```txt
[VOTE] Release Apache DolphinScheduler 3.2.2
[VOTE] Release Apache DolphinScheduler <VERSION>
```

Body:

```txt
Hello DolphinScheduler Community,

This is a call for vote to release Apache DolphinScheduler version 3.2.2
This is a call for vote to release Apache DolphinScheduler version <VERSION>

Release notes: https://github.com/apache/dolphinscheduler/releases/tag/3.2.2
Release notes: https://github.com/apache/dolphinscheduler/releases/tag/<VERSION>

The release candidates: https://dist.apache.org/repos/dist/dev/dolphinscheduler/3.2.2/
The release candidates: https://dist.apache.org/repos/dist/dev/dolphinscheduler/<VERSION>/

Maven 2 staging repository: https://repository.apache.org/content/repositories/<STAGING.REPOSITORY>/org/apache/dolphinscheduler/

Git tag for the release: https://github.com/apache/dolphinscheduler/tree/3.2.2
Git tag for the release: https://github.com/apache/dolphinscheduler/tree/<VERSION>

Release Commit ID: https://github.com/apache/dolphinscheduler/commit/<SHA-VALUE>

Expand Down Expand Up @@ -436,13 +436,13 @@ Checklist for reference:
Title:

```txt
[RESULT][VOTE] Release Apache DolphinScheduler 3.2.2
[RESULT][VOTE] Release Apache DolphinScheduler <VERSION>
```

Body:

```txt
The vote to release Apache DolphinScheduler 3.2.2 has passed.Here is the vote result,
The vote to release Apache DolphinScheduler <VERSION> has passed.Here is the vote result,

4 PMC member +1 votes:

Expand Down Expand Up @@ -478,19 +478,19 @@ and then find DolphinScheduler in [apache staging repositories](https://reposito
### Update Document

Website should be present before you send the announce mail this section will tell you how to change the website. For example,
the release version is `3.2.2`, the following updates are required(note it will take effect immediately when the PR is merged):
the release version is `<VERSION>`, the following updates are required(note it will take effect immediately when the PR is merged):

- Repository **apache/dolphinscheduler-website**:
- `config/download.json`: add the download of the `3.2.2` release package
- `scripts/conf.sh`: Add new release version `3.2.2` key-value pair to variable `DEV_RELEASE_DOCS_VERSIONS`
- `config/download.json`: add the download of the `<VERSION>` release package
- `scripts/conf.sh`: Add new release version `<VERSION>` key-value pair to variable `DEV_RELEASE_DOCS_VERSIONS`
- Repository **apache/dolphinscheduler** (dev branch):
- `docs/configs/site.js`:
- `docsLatest`: update to `3.2.2`
- `docs0`: The `text` of two places of `en-us/zh-cn` needs to be updated to `latest(3.2.2)`
- `docs/configs/index.md.jsx`: Add `3.2.2: docsxyzConfig` and add new `import` for the new `docsxyzConfig`
- `docs/docs/en/history-versions.md` and `docs/docs/zh/history-versions.md`: Add new `3.2.2` release docs.
- `docsLatest`: update to `<VERSION>`
- `docs0`: The `text` of two places of `en-us/zh-cn` needs to be updated to `latest(<VERSION>)`
- `docs/configs/index.md.jsx`: Add `<VERSION>: docsxyzConfig` and add new `import` for the new `docsxyzConfig`
- `docs/docs/en/history-versions.md` and `docs/docs/zh/history-versions.md`: Add new `<VERSION>` release docs.
- `.github/ISSUE_TEMPLATE/bug-report.yml`: DolphinScheduler's GitHub [bug-report](https://github.com/apache/dolphinscheduler/blob/dev/.github/ISSUE_TEMPLATE/bug-report.yml)
issue template have **Version** selection bottom. So after released we should add the new `3.2.2` to
issue template have **Version** selection bottom. So after released we should add the new `<VERSION>` to
bug-report.yml

### Publish Docker Image and Helm Chart
Expand All @@ -511,30 +511,30 @@ Announcement e-mail template as below:
Title:

```txt
[ANNOUNCE] Release Apache DolphinScheduler 3.2.2
[ANNOUNCE] Release Apache DolphinScheduler <VERSION>
```

Body:

```txt
Hi all,

We are glad to announce the release of Apache DolphinScheduler 3.2.2. Once again I would like to express my thanks to your help.
We are glad to announce the release of Apache DolphinScheduler <VERSION>. Once again I would like to express my thanks to your help.

Dolphin Scheduler is a distributed and easy-to-extend visual workflow scheduler system,
dedicated to solving the complex task dependencies in data processing, making the scheduler system out of the box for data processing.


Download Links: https://dolphinscheduler.apache.org/en-us/download

Release Notes: https://github.com/apache/dolphinscheduler/releases/tag/3.2.2
Release Notes: https://github.com/apache/dolphinscheduler/releases/tag/<VERSION>

Website: https://dolphinscheduler.apache.org/

DolphinScheduler Resources:
- Issue: https://github.com/apache/dolphinscheduler/issues/
- Mailing list: [email protected]
- Documents: https://dolphinscheduler.apache.org/en-us/docs/3.2.2/about/introduction
- Documents: https://dolphinscheduler.apache.org/en-us/docs/<VERSION>/about/introduction
```

## News
Expand Down
4 changes: 2 additions & 2 deletions docs/docs/en/guide/expansion-reduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,9 @@ Attention: DolphinScheduler itself does not depend on Hadoop, Hive, Spark, but w
mkdir -p /opt
cd /opt
# decompress
tar -zxvf apache-dolphinscheduler-3.2.2-bin.tar.gz -C /opt
tar -zxvf apache-dolphinscheduler-<version>-bin.tar.gz -C /opt
cd /opt
mv apache-dolphinscheduler-3.2.2-bin dolphinscheduler
mv apache-dolphinscheduler-<version>-bin dolphinscheduler
```

```markdown
Expand Down
12 changes: 6 additions & 6 deletions docs/docs/en/guide/installation/kubernetes.md
Original file line number Diff line number Diff line change
Expand Up @@ -233,9 +233,9 @@ kubectl scale --replicas=6 sts dolphinscheduler-worker -n test # with test names
2. Create a new `Dockerfile` to add MySQL driver:

```
FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-<service>:3.2.2
FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-<service>:<version>
# For example
# FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-tools:3.2.2
# FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-tools:<version>

# Attention Please, If the build is dolphinscheduler-tools image
# You need to change the following line to: COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/tools/libs
Expand Down Expand Up @@ -284,9 +284,9 @@ externalDatabase:
2. Create a new `Dockerfile` to add MySQL or Oracle driver:

```
FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-<service>:3.2.2
FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-<service>:<version>
# For example
# FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker:3.2.2
# FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker:<version>

# If you want to support MySQL Datasource
COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/libs
Expand Down Expand Up @@ -316,7 +316,7 @@ docker build -t apache/dolphinscheduler-<service>:new-driver .
1. Create a new `Dockerfile` to install pip:

```
FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker:3.2.2
FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker:<version>
COPY requirements.txt /tmp
RUN apt-get update && \
apt-get install -y --no-install-recommends python-pip && \
Expand Down Expand Up @@ -351,7 +351,7 @@ docker build -t apache/dolphinscheduler-worker:pip .
1. Create a new `Dockerfile` to install Python 3:

```
FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker:3.2.2
FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker:<version>
RUN apt-get update && \
apt-get install -y --no-install-recommends python3 && \
rm -rf /var/lib/apt/lists/*
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/en/guide/installation/standalone.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ If you want to deploy DolphinScheduler in production, we recommend you follow [c
## Preparation

- JDK:download [JDK][jdk] (1.8+), install and configure environment variable `JAVA_HOME` and append `bin` dir (included in `JAVA_HOME`) to `PATH` variable. You can skip this step if it already exists in your environment.
- Binary package: download the DolphinScheduler binary package at [download page](https://dolphinscheduler.apache.org/en-us/download/3.2.2). <!-- markdown-link-check-disable-line -->
- Binary package: download the DolphinScheduler binary package at [download page](https://dolphinscheduler.apache.org/en-us/download/<version>). <!-- markdown-link-check-disable-line -->

## Start DolphinScheduler Standalone Server

Expand Down
10 changes: 5 additions & 5 deletions docs/docs/en/guide/start/docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Start DolphinScheduler with standalone-server Docker images is the easiest way t
you can learn DolphinScheduler's concepts and usage, with minimal cost.

```shell
$ DOLPHINSCHEDULER_VERSION=3.2.2
$ DOLPHINSCHEDULER_VERSION=<version>
$ docker run --name dolphinscheduler-standalone-server -p 12345:12345 -p 25333:25333 -d apache/dolphinscheduler-standalone-server:"${DOLPHINSCHEDULER_VERSION}"
```

Expand All @@ -37,11 +37,11 @@ be stored on disks after you change docker-compose configuration, and it is robu
DolphinScheduler in a long term. You have to install [docker-compose](https://docs.docker.com/compose/install/) before you
start servers.

After complete the installation, get the `docker-compose.yaml` file from [download page](https://dolphinscheduler.apache.org/en-us/download/3.2.2)
After complete the installation, get the `docker-compose.yaml` file from [download page](https://dolphinscheduler.apache.org/en-us/download/<version>)
form its source package, and make sure you get the right version. After download the package, you can run the commands as below.

```shell
$ DOLPHINSCHEDULER_VERSION=3.2.2
$ DOLPHINSCHEDULER_VERSION=<version>
$ tar -zxf apache-dolphinscheduler-"${DOLPHINSCHEDULER_VERSION}"-src.tar.gz
# Going to docker-compose's location
# For Mac or Linux users
Expand All @@ -68,7 +68,7 @@ $ docker-compose --profile all up -d
container when it up. You could start DolphinScheduler server separately if you want to reuse your exists services.

```shell
$ DOLPHINSCHEDULER_VERSION=3.2.2
$ DOLPHINSCHEDULER_VERSION=<version>
# Initialize the database, make sure database <DATABASE> already exists
$ docker run -d --name dolphinscheduler-tools \
-e DATABASE="postgresql" \
Expand Down Expand Up @@ -134,5 +134,5 @@ and use `admin` and `dolphinscheduler123` as default username and password in th

You can modify some environment variables to change configurations when you are starting servers through Docker. We have
an example in [using exists PostgreSQL ZooKeeper](#using-exists-postgresql-zookeeper) to change database and ZooKeeper configurations,
and you could find all environment variables in [all environment variables](https://github.com/apache/dolphinscheduler/blob/3.2.2/script/env/dolphinscheduler_env.sh) <!-- markdown-link-check-disable-line -->
and you could find all environment variables in [all environment variables](https://github.com/apache/dolphinscheduler/blob/<version>/script/env/dolphinscheduler_env.sh) <!-- markdown-link-check-disable-line -->
and change them if you want.
4 changes: 0 additions & 4 deletions docs/docs/en/history-versions.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,6 @@

#### Setup instructions, are available for each stable version of Apache DolphinScheduler below:

### Versions: 3.2.2

#### Links: [3.2.2 Document](../3.2.2/user_doc/about/introduction.md)

### Versions: 3.2.1

#### Links: [3.2.1 Document](../3.2.1/user_doc/about/introduction.md)
Expand Down
40 changes: 20 additions & 20 deletions docs/docs/zh/contribute/release.md
Original file line number Diff line number Diff line change
Expand Up @@ -347,7 +347,7 @@ svn --username="${A_USERNAME}" commit -m "release ${VERSION}"

#### 检查源码包的文件内容

解压缩`apache-dolphinscheduler-3.2.2-src.tar.gz`,进行如下检查:
解压缩`apache-dolphinscheduler-<VERSION>-src.tar.gz`,进行如下检查:

- 检查源码包是否包含由于包含不必要文件,致使 tarball 过于庞大
- 存在`LICENSE`和`NOTICE`文件
Expand All @@ -359,7 +359,7 @@ svn --username="${A_USERNAME}" commit -m "release ${VERSION}"

#### 检查二进制包的文件内容

解压缩`apache-dolphinscheduler-3.2.2-bin.tar.gz`进行如下检查:
解压缩`apache-dolphinscheduler-<VERSION>-bin.tar.gz`进行如下检查:

- 存在`LICENSE`和`NOTICE`文件
- 所有文本文件开头都有 ASF 许可证
Expand Down Expand Up @@ -394,23 +394,23 @@ DolphinScheduler 社区投票,发起投票邮件到`[email protected]
标题:

```txt
[VOTE] Release Apache DolphinScheduler 3.2.2
[VOTE] Release Apache DolphinScheduler <VERSION>
```

正文:

```txt
Hello DolphinScheduler Community,

This is a call for vote to release Apache DolphinScheduler version 3.2.2
This is a call for vote to release Apache DolphinScheduler version <VERSION>

Release notes: https://github.com/apache/dolphinscheduler/releases/tag/3.2.2
Release notes: https://github.com/apache/dolphinscheduler/releases/tag/<VERSION>

The release candidates: https://dist.apache.org/repos/dist/dev/dolphinscheduler/3.2.2/
The release candidates: https://dist.apache.org/repos/dist/dev/dolphinscheduler/<VERSION>/

Maven 2 staging repository: https://repository.apache.org/content/repositories/<STAGING.REPOSITORY>/org/apache/dolphinscheduler/

Git tag for the release: https://github.com/apache/dolphinscheduler/tree/3.2.2
Git tag for the release: https://github.com/apache/dolphinscheduler/tree/<VERSION>

Release Commit ID: https://github.com/apache/dolphinscheduler/commit/<SHA-VALUE>

Expand Down Expand Up @@ -441,11 +441,11 @@ Checklist for reference:
Title:

```txt
[RESULT][VOTE] Release Apache DolphinScheduler 3.2.2
[RESULT][VOTE] Release Apache DolphinScheduler <VERSION>
```

```txt
The vote to release Apache DolphinScheduler 3.2.2 has passed.Here is the vote result,
The vote to release Apache DolphinScheduler <VERSION> has passed.Here is the vote result,

4 PMC member +1 votes:

Expand Down Expand Up @@ -480,17 +480,17 @@ git push --delete "${GH_REMOTE}" "${VERSION}-prepare"

### 更新文档

官网应该在您发送通知邮件之前完成更新,本节将告诉您如何更改网站。假设发版的版本是 `3.2.2`,需要进行以下更新(注意,当修改 pull requests 被 merge 后就会生效):
官网应该在您发送通知邮件之前完成更新,本节将告诉您如何更改网站。假设发版的版本是 `<VERSION>`,需要进行以下更新(注意,当修改 pull requests 被 merge 后就会生效):

- **apache/dolphinscheduler-website** 仓库:
- `config/download.json`: 增加 `3.2.2` 版本发布包的下载
- `scripts/conf.sh`: 在变量 `DEV_RELEASE_DOCS_VERSIONS` 中增加版本为 `3.2.2` 的新键值对
- `config/download.json`: 增加 `<VERSION>` 版本发布包的下载
- `scripts/conf.sh`: 在变量 `DEV_RELEASE_DOCS_VERSIONS` 中增加版本为 `<VERSION>` 的新键值对
- **apache/dolphinscheduler** 仓库 (dev 分支):
- `docs/configs/site.js`:
- `docsLatest`: 更新为 `3.2.2`
- `docs0`: 两处 `en-us/zh-cn` 的 `text` 更新为 `latest(3.2.2)`
- `docs/configs/index.md.jsx`: 增加 `'3.2.2': docsxyzConfig,` 以及新的 `import`
- `docs/docs/en/history-versions.md` 和 `docs/docs/zh/history-versions.md`: 增加新的发版版本 `3.2.2` 的链接
- `docsLatest`: 更新为 `<VERSION>`
- `docs0`: 两处 `en-us/zh-cn` 的 `text` 更新为 `latest(<VERSION>)`
- `docs/configs/index.md.jsx`: 增加 `'<VERSION>': docsxyzConfig,` 以及新的 `import`
- `docs/docs/en/history-versions.md` 和 `docs/docs/zh/history-versions.md`: 增加新的发版版本 `<VERSION>` 的链接
- `.github/ISSUE_TEMPLATE/bug-report.yml`: DolphinScheduler 在 GitHub bug report 的 issue 中有版本选择,当有新的版本发版后,需要更新
[bug-report](https://github.com/apache/dolphinscheduler/blob/dev/.github/ISSUE_TEMPLATE/bug-report.yml) 中的 **Version** 部分。

Expand All @@ -510,30 +510,30 @@ git push --delete "${GH_REMOTE}" "${VERSION}-prepare"
标题:

```txt
[ANNOUNCE] Release Apache DolphinScheduler 3.2.2
[ANNOUNCE] Release Apache DolphinScheduler <VERSION>
```

正文:

```txt
Hi all,

We are glad to announce the release of Apache DolphinScheduler 3.2.2. Once again I would like to express my thanks to your help.
We are glad to announce the release of Apache DolphinScheduler <VERSION>. Once again I would like to express my thanks to your help.

Dolphin Scheduler is a distributed and easy-to-extend visual workflow scheduler system,
dedicated to solving the complex task dependencies in data processing, making the scheduler system out of the box for data processing.


Download Links: https://dolphinscheduler.apache.org/zh-cn/download

Release Notes: https://github.com/apache/dolphinscheduler/releases/tag/3.2.2
Release Notes: https://github.com/apache/dolphinscheduler/releases/tag/<VERSION>

Website: https://dolphinscheduler.apache.org/

DolphinScheduler Resources:
- Issue: https://github.com/apache/dolphinscheduler/issues/
- Mailing list: [email protected]
- Documents: https://dolphinscheduler.apache.org/zh-cn/docs/3.2.2/about/introduction
- Documents: https://dolphinscheduler.apache.org/zh-cn/docs/<VERSION>/about/introduction
```

## News
Expand Down
Loading
Loading