3.0.5

What's Changed

• fix: when auth is false and user || pwd null can't send alert email excection #13761
• fix: API usage of gantt graph in UI #13778
• fix: when update scheduler will execute workflow #13285
• fix: when the task details are not modified, the predecessor node cannot be modified #13422
• fix: NPE caused by the execution of workflow with startNode and forbidden task #13668
• fix: task failure continue #13155
• chore: Publish Docker images to docker hub when released #13631

3.1.4

What's Change

Bugfix

• Fix task wake up failed will block the event processing (#13466)
• Fix task instance will generate multiple times when retry interval is 0/s (#13571)
• [fix-13492]add single quotes (#13534)
• FIX: qrtz table use uppercase (#13494)
• Fix serial wait will go into block when there is no task need to submit (#13486)
• [Fix-13418][UI] Remove move-task-definition code (#13441)
• [fix][data quality] fix when running data quality task with Error "No main class set in JAR" (#13360)
• [BUG][install.sh] fix the install Error "Command not found rmr" & support ZooKeeper (3.4.6+) (#13163)
• [Bug-13309][TaskPlugin] fix k8s task run fail (#13348)
• Fix MessageSender may be null due to class initialize late #13446
• [Fix][Worker] alert email in sql task in K8S (#13387)

Document

• [Fix-13287][doc]Fix Chinese workflow-definition.md duplication description in execution-type (#13361)

Full Changelog: 3.1.3...3.1.4

2.0.8

What's Change

Bugfix

• [BUG-12361] complement data by serial execution, after a failed process instance, subsequent process instances are not generated (#12488)
• [Bug-12482] [Master] check for dependency task instance need failover is error (#12513)
• [Bugfix-12568] [Master] The retry task was submitted in advance will block other task (#12570)
• [Fix-12586] [Master] Repeat running the complement process instance will generate redundant complement instances (#12606)
• [Bugfix-12603] Dependency conditions for the last 24 hours of the dependent task did not take effect (#12862)
• [Bug-12960][Master] Fix that start parameters cannot update global variables (#13005)
• [Fix-13247] [Master] Kill yarn job failed due to NPE exception (#13164)
• [Fix-11903] [Worker] org.apache.dolphinscheduler.plugin.task.api.ProcessUtils: - kill task failed (#13345)
• [Fix-13292] [Api] When serial complement is a process configured with timing scheduling, an error will occur when selecting a time that is not within the timing range (#13356)
• [Fix-13244] [Master] Stopping a workflow does not update task status correctly (#13375)
• [Fix-13283][Master] Worker failover cause task repeat run (#13403)
• [Fix-13346] [Master] Pause a workflow instance that is complementing will automatically generate a new complement instance (#13411)
• [Fix-13413][Master]Empty dependency operation causes Master null pointer exception (#13420)
• [Fix-13272] [Master] resume failure processInstance state remain failure (#13425)
• [Fix-13347] [Master] Forced success will cause the workflow instance status to be judged as successful (#13426)
• [Fix-13409] [Worker] concurrent task response occasional cause status error (#13433)
• [Fix-12966] Failed to export and then import the project (#13455)
• [Improvement-13413] [Master] Empty dependency operation causes Master null pointer exception (#13457)
• [Fix-13496] [Worker] When the task request has not set the current execution status, an error will cause an NPE exception (#13497)
• [Fix-13505] [complement] enter startup parameters, the complement task is not started (#13510)

Full Changelog: 2.0.7...2.0.8

3.1.3

What's Change

Bugfix

• [3.1.3-release][SQL] Fix t_ds_fav table does not exist when rolling upgrade (#13337)
• fix mapper table name t_ds_fav_task (#13334)
• rename h2 and postgreSQL table t_ds_fav to t_ds_fav_task (#13333)
• Fix hive datasource connection leak (#12226)
• [fix][worker][bug] master/worker crash when registry recover from SUSPENDED to RECONNECTED (#13328)
• [Fix-13260][API] Ironed out the issue with lacking the value of user's name when querying the list of workflow definitions. #13261
• [Fix-13248][UI] Ironed out the issue with failing to query task instances by some state of task. #13251
• [Bug][DAO] Worked out the issue with throwing the sql exception.
• when delete workflow, delete related task #12681
• [fix][python] schedule unexpect offline each time update from api #13301
• Fix task group cannot release when kill task #13314
• fix sagemaker sync status (#13264)
• [Improvement][api]Start the downstream workflow when it is online (#13257)
• [Bug][Worker] Fix duplicated key bug when have out parameters of same name (#12930)
• [Fix-13144][UI] Worked out the issue with failing to query workflow instances by the incorrect workflow status. #13243
• [Fix-13187] refactor OpenmldbParameters #13190
• fix the issue with failing to save the settings of timeout alarm (#13234)
• Use bigint represent term in mysql registry to avoid time inaccurate #13082
• fix resource data authorization does not take effect. (#13318)
• Fix task group might be rob by multiple task #13115
• [Bug] [dolphinscheduler-api] in the task define list, when edit task, should show pre node list (#12695)
• [Fix-13160][UI] Ironed out the issue with failing to save the correct settings of dependent tasks. (#13179)
• fix retry task failure (#13077)

Document

• [fix][doc] Dead link for python api (#12848)

Full Changelog: 3.1.2...3.1.3

3.0.4

What's Change

Bugfix

• [Bug] wrong logic when master-server check data-quality result (#13065)
• [UI] Make download log button disabled when host is null (#13090)
• Throws the correct exception for the updateResource interface (#13120)
• Solve the deadlock problem caused by queuing (#13191)
• [fix][dml] Also query workflow without tenant by given name (#13075)
• [Bug] [Server] Once click online schedule time, task will be automatically scheduled (#13092)
• [Bug-12868][UI] Data target type is only HDFS (#12902)

Document

• [Improvement][doc] correct error in CONTRIBUTING.md (#13012)
• [Fix] Replaced dead links in the docs. (#13170)
• [Fix][Doc]Update pseudo-cluster.md (#13207)
• [Fix][Doc] Replace the links. (#13276)
• [doc] Correct install_env.sh file name (#13294)
• fix example checkFormula (#13110)

Full Changelog: 3.0.3...3.0.4

3.1.2

Bugfix

• [Improvement] change alert start.sh (#13100)
• [Fix] Add token as authentication for python gateway (#12893)
• [Fix-13010] [Task] The Flink SQL task page selects the pre-job deployment mode, but the task executed by the worker is the Flink local mode
• [Fix-12997][API] Fix that the end time is not reset when the workflow instance reruns. (#12998)
• [Fix-12994] [Worker] Fix kill process does not take effect (#12995)
• Fix sql task will send alert if we don't choose the send email #12984
• [Fix-13008] [UI] When using the complement function, turn on the dependent mode to generate multiple unrelated workflow instances (#13009)
• [Fix][doc] python api release link
• [Fix] Python task can not pass the parameters to downstream task. (#12961)
• [Fix] Fix Java path in Kubernetes Helm Chart (#12987)
• [Fix-12963] [Master] Fix dependent task node null pointer exception (#12965)
• [Fix-12954] [Schedule] Fix that workflow-level configuration information does not take effect when timing triggers execution
• Fix execute shell task exception no dolphinscheduler_env.sh file execute permission (#12909)
• Upgrade clickhouse jdbc driver #12639
• add spring-context to alert api (#12892)
• [Upgrade][SQL]Modify the table t_ds_worker_group to add a description field in the postgresql upgrade script #12883
• Fix NPE while retry task (#12903)
• Fix-12832][API] Fix update worker group exception group name already exists. #12874
• Fix and enhance helm db config (#12707)

Document

• [Fix][Doc] Fix sql-hive and hive-cli doc (#12765)
• [Fix][Alert] Ignore alert not write info to db (#12867)
• [Doc] Add skip spotless check during ASF release #12835
• [Doc][Bug] Fix dead link caused by markdown cross-files anchor #12357 (#12877)

Python API

• [Fix] python api upload resource center failed
• [Feature] Add CURD to the project/tenant/user section of the python-DS (#11162)
• [Chore][Python] Change name from process definition to workflow (#12918)
• [Feature] Support set execute type to pydolphinscheduler (#12871)
• [Hotfix] Correct python doc link
• [Improvement][Python] Validate version of Python API at launch (#11626)

Full Changelog: 3.1.1...3.1.2

3.0.3

Bugfix

• [Bug-12954] [Schedule] Fix that workflow-level configuration information does not take effect when timing triggers execution (#12955)
• [Fix][UI] download resource return 401 (#12566)
• Make sure all failed task will save in errorTaskMap (#12424)
• Fix timing scheduling trigger master service report to get command parameter null pointer exception (#12419)
• [Fix] Change the default unix shell executor from sh to bash (#12180)
• Fix-ldap-error (#12929)

Document

• [Feature-8030][docs] Add sqoop task doc (#12855)
• [Doc][fix] Add new alert doc Slack (#12567)
• [Improvement][Doc]Update architecture and process-start-flow pic (#12912)

Full Changelog: 3.0.2...3.0.3

3.0.2

Bugfix

Click to expand

• Fix default value for resource center in k8s mode (#12906)
• Fix send script alert NPE (#12495) (#12895)
• Fix procedure task output param (#12894)
• Fix memory leak in worker due to message retry map (#12878) (#12887)
• Fix flink sql cannot run due to missing main jar (#12705) (#12800)
• Fix sql error (#12717) (#12799)
• If the worker's clock early than master node, will ignore dispatch time (#12219) (#12798)
• Fix dependent task can not predicate the status of the corresponding task correctly (#12253) (#12792)
• Fix python class description error (#12790)
• Fix the duration in Workflow Instance page. (#12264) (#12788)
• Fix error problem on h2 startup data quality rule management page (#12108) (#12781)
• Fix hive datasource connection leak (#12226) (#12777)
• Fix constructing processInstance may NPE when master handling command (#12056) (#12776)
• Fix the bugs when upgrading ds from v1.3.9 to v3.0.0 ,such as file cannot be found or column not found etc (#11619) (#12770)
• Fix the bug which some scheduled tasks are not triggered on time (#12233) (#12767)
• Fix alert sending error (#11774) (#12762)
• Fix IPv4 Pattern (#11762) (#12739)
• Add unique key to process_definition_log avoid TooManyResultExpection (#12503) (#12670)
• Fix datax task data instance replay error
• Fix wrong env var name for alert in K8S (#12369) (#12635)
• Fix when the sql query result is empty, the email fails to send the attachment, and an exception will be reported (#12059) (#12633)

Document

Click to expand

• Add new alert doc Slack (#12567) (#12797)
• Update the readme content (#12500) (#12796)
• Add http header to avoid github 403 in dlc (#12509) (#12793)
• Change optional parameters --jar to --jars for Spark (#12385) (#12386) (#12791)
• Change release process base on new tool (#12324) (#12783)
• Update ZooKeeper minimum version requirements (#12284) (#12288) (#12769)
• Update slack invitation link (#12258) (#12754)
• Correct release export key step (#12228) (#12751)
• Polish docs for standalone deployment (#12181) (#12743)
• Do not change docsdev.js during releasing (#12151) (#12740)
• Fix link errors in release documentation
• Fix list param error when use sql task (#11285) (#12632)
• Fix the API usage of gantt graph (#11642) (#12631)
• Fix task switch branch not show in webui (#12120)

Improvement

Click to expand

• Script cannot contains ''' in params (#12067) (#12913)
• Add configmap resource permissions so config hot reload can work (#12572) (#12795)
• Only expose necessary actuator endpoints (#12571) (#12794)
• Correct spelling errors (#12174) (#12780)
• Avoid using search in for and start using testSaveTaskDefine (#11383) (#12773)
• Automatically convert resource storage type to upper case (#12281) (#12766)
• Delete useless exclusions (#12273) (#12761)
• Add chmod command after unzip DS tar.gz (#12752)
• Set the required to be reactive in the task modal. (#12225) (#12748)
• Improvement the error message when batch delete workflow (#11682) (#12747)
• Validate before deleting workflow or task used by other tasks (#10873) (#12731)
• Fix vulnerability in LDAP login (#11586) (#12730)
• Easier release: cherry-pick, changelog, contributor (#11478) (#12634)

3.1.1

Bugfix

Click to expand

• [fix-12675]edit workflow related task, workflow's task version change
• [Bug] Resource default auth function disabled false.
• [3.1.1][API]Fix updating workflow definition causing task definition data to be duplicated. (#12712)
• Fix flink sql cannot run due to missing main jar (#12705)
• Add pythonNodePort in config file (#12685)
• [Fix-12109]Fix the errors when starting 2 times with dolphinscheduler-daemon.sh (#12118)
• Fix the waiting strategy cannot recovery if the serverstate is already in running (#12651)
• Fix alert status spelling error #12592
• Add configmap resource permissions so config hot reload can work (#12572)
• [Fix][UI] download resource return 401 (#12566)
• [Bug] [API] Before deleting a worker group, check whether there is environment that reference the worker group. #12534
• [Fix-12518][Swagger]Fill up missing i18 properties (#12599)
• [Bug][Master] Add the aws-java-sdk-s3 jar package to the master module (#12259) (#12512)
• Remove equals in User to fix UT #12487
• [Bug] Set tenantDir permission #12486
• [Fix-12451][k8s] Read the kubeconfig from cluster conf (#12452)
• [Fix-12356][k8s] fix the null exception when submitting k8s task plugin (#12358)
• [fix#12439] [Alert] fix send script alert NPE #12495
• [Bug] [API] The workflow definition and the tenant in the workflow instance are inconsistent. (#12533)
• fix fix upgrade to 3.1.0 sql missing field (#12314) (#12315)
• [Fix-12425][API] Add rollbackFor setting.
• [Bug-12410] [API]Fix the worker list result in workflow definition only has default
• [BUG-12306][UI]Fix the password item always is disabled (#12437)
• [Fix][Task] Fix dependent task can not predicate the status of the corresponding task correctly (#12253)
• make sure all failed task will save in errorTaskMap (#12424)
• Fix timing scheduling trigger master service report to get command parameter null pointer exception (#12419)
• [Bug] [spark-sql] In spark-sql, select both SPARK1 and SPARK2 versions and execute ${SPARK_HOME2}/bin/spark-sql (#11721) (#12420)
• fix hdfs defaultFs not working (#11823) (#12418)
• source is not available in sh (#12413)
• fix datax NPE issue (#12388) (#12407)
• [BUG-12396][Schedule] Fixed that the workflow definition scheduling that has been online after the version upgrade does not execute

Doc

Click to expand

• [doc] Correct descriptions in glossary.md (#12282)
• [Hotfix][Docs] Fix 404 dead link
• update english oracle.md (#12332)

Improvement

Click to expand

• adjust the args of router in the dag (#12759)
• Change command file permission to 755 (#12678)
• beautify the dag (#12728)
• support to use the clearable button of components to search (#12668)
• Add worker-group-refresh-interval in master config #12601
• [Improvement][UI] Support to view the process variables on the page of DAG. (#12609)
• [Improvement] Merge spi.utils into common.utils (#12607)
• Add task executor threads full policy config in worker (#12510)
• Add mysql support to helm chart (#12517)
• Reorganize some classes in common module, remove duplicate classes #12321
• [DS-12131][master] Optimize the log printing of the master module acc… (#12152)
• [Improvement][WorkerGroup]Remove workerGroup in registry #12217
• [Improvement] remove log-server and server module #12206
• Refactor LogServiceClient Singleton to avoid repeat creation of NettyClient #11777
• [Improvement] Add remote task model #11767 (#12541)
• Use temurin Java docker images instead of deprecated ones (#12334)
• [DS-12154][worker] Optimize the log printing of the worker module (#12183)
• [Improvement-12372][k8s] Update the deprecated k8s api (#12373)
• [Improvement][Task Plugin] Modify the comment of 'deployMode'. (#12163)
• Remove the DataxTaskTest class because there is no junit5 package.
• [Improvement][API] When the workflow definition is copied, the operation user of the timed instance is changed to the current user
• [Improvement-12391][API] Workflow definitions that contain logical task nodes support the copy function

2.0.7

Bugfix:

Click to expand

• Monitoring: Added disk available capacity monitoring (#10030)
• fix Parameter transfer problem when recover failed task (#10694)
• fix copy bug (#10940)
• fix dependent task when the dependent on process have forbidden tasks (#10952)
• correct post body for http alert plugin (#11946)
• [bugfix-10116]fix complement data bug (#12376)
• [Fix-12167][MasterServer] The retry task was submitted in advance (#12291)
• [Bug][API] fix Circular dependencies are not checked in save process task relation (#12294)
• [Fix-12422][MasterServer] The task needing fault tolerance cannot be submitted (#12423)
• [doc] Correct Docker README URL (#11794)
• [Fix-10080]When the created tenant name is too long, an error message will be prompted (#10081)