Skip to content

Releases: apache/dolphinscheduler

3.1.0

28 Sep 04:39
Compare
Choose a tag to compare

Feature

Click to expand
  • [Feature-8252][doc] K8s and namespace manager docs and web page update (#9881)
  • [feat][task plugin] Add Jupyter task plugin (#9872)
  • [Feature-9474] [CI] Add cluster test script verify on shell script (#9997)
  • [Feature-10034][CI] Add postgresql cluster test in CI (#10035)
  • [feat] Add batch rerun and stop for process instance (#10013)
  • [Feat][UI] Add resource username for function and resource manage (#10086)
  • [Feature][plugin] Add k8s task in task plugin (#9425)
  • [Feature][Task Plugin] Add mlflow task plugin for MLops scenario. (#10036)
  • [Feature] [CI] Add alert server cluster test #10125
  • [python] Change distribute name avoid confusion with main pkg (#10134)
  • [Feature][MLops] Support MLflow Models to deploy model service (MLflow models serve and Docker) (#10150)
  • [Feature][CI] Schedule Dead Link Checker (#10000)
  • [Feature][MLops] Add MLflow Projects (custom project) component (#10175)
  • [Feature][Permission] Project management module authority control refactoring, authority abstract interface setting. (#10261)
  • [Feature][Datasource]replace DataSourceController API with string JSON and cast to DTO (#10276)
  • [Feature][API] Modify the permissions of project management, security center, data source center and data quality module. (#10289)
  • [Feature][Permission] Reconstruction of permissions of resource center and monitoring center. (#10307)
  • [feat][task] Add OpenMLDB task plugin (#10198)
  • [Feature][Task Plugin] Add support for dynamic form for zeppelin task plugin (#9977) (#10269)
  • [Feature-10290] [API] Refactor org.apache.dolphinscheduler.api.controller.AccessTokenController#createToken api (#10301)
  • [Feature] Should not add refresh logic in the e2e case. (#10331)
  • [Feature][Monitoring] Added disk available capacity monitoring (#10287)
  • [Feature-6758][Task] Add limit resource usage for tasks base on cgroup (#10373)
  • [Feature][metrics] Add master, worker metrics (#10326)
  • [Feature] [MLOps] support mlflow deploy with docker compose (#10217)
  • [Feature][Metrics] Enable prometheus to collect metrics in standalone mode demo (#10398)
  • [Feature][UI] Add manual input for scheduling dates. (#10396)
  • Add mysql registry plugin (#10406)
  • [CI] Add OWASP Dependency Check (#10058)
  • [Feature][Improvement] Support multi cluster environments - k8s type (#10096)
  • [Feature-10411] Add tenant api test (#10442)
  • [Feature-10363][server] S3 Resource center supports bucket customization (#10433)
  • [feature][task] Enable zeppelin schedule a whole zeppelin note (#10434)
  • [Feature] Enable users to switch and install conda env in jupyter task (#10337)
  • [Feature][Task Plugin] Add DVC task plugin for MLops scenario (#10372) (#10407)
  • [Feature] [Permission]Data permission module added and Query class api remove verification. (#10504)
  • [Feature] TaskGroup remove query func permission check. #10557
  • [python] Support using full-name definitions for resources (#10551)
  • [feat] ui build no inherit from maven proxy (#10590)
  • [Feature] Time function analysis extension. (#10624)
  • [feature][task-flink] Support Flink application mode (#9577)
  • [Feature][Master] Failover workflow instance base on priority (#10754)
  • [Feature] Enable users to create python env from requirements.txt (#10658)
  • [Feature-10629][Improvement] Support multi cluster environments - namespace use cluster (#10650)
  • [Feature][dependencies] Bump spring-core from 5.3.12 to 5.3.19 (#10857)
  • [Feature-10219][EMR] EMR supports use to add steps to an existing cluster (#10657)
  • [Feature-10871] add workflow executing data query (#10875)
  • [Feat][UI] Uniform search button size. (#10880)
  • [Feature][Metrics] Add resource download related metrics for workers (#10749)
  • [Feat][UI] Add title display to the table in the security center. (#10969)
  • feat: Added global trim method (#10975)
  • [Feature][Task Plugin] Enable users to switch endpoints in zeppelin tasks (#10925)
  • [Feature][Datax Task]Datax task plugin support custom parameters (#10950)
  • feat: Add trim method to all input (#10984)
  • [Feat][UI] Added LDAP function. (#11029)
  • [Feature][Task Plugin] Increase zeppelin task stability in production (#11010)
  • [Feature][Task Plugin] Add SageMaker Pipeline task plugin for MLOps scenario (#10826) (#10935)
  • [python] add import SageMaker task (#11121)
  • Add base loop task execotor and http template parser (#11137)
  • [Feature-10604] Task name in the copied and imported workflow without copy suffix (#10607)
  • [Feature][Api] Refactor org.apache.dolphinscheduler.api.controller.ProjectController (#10832)
  • [Feature-10837][Task Plugin] Sqoop Task Add --split-by Parameter (#10960)
  • [Feature] Add CURD to the project/tenant/user section of the API and add its tests. (#10913)
  • [Feature] Add new task type chunjun (#10937)
  • Write alert result into db (#11221)
  • [Python] Migrate exists method call in PythonGateway to corresponding service (#11188)
  • [Feature][style] Add spotless maven plugin for automatic style fix. (#11272)
  • [Python] Combine gateway.entry_point call in python api side (#11330)
  • [Feature-9978][UI] add node dependency type feature (#11235)
  • [Feature][Api] Refactor org.apache.dolphinscheduler.api.controller.QueueController (#11154)
  • [python] Add local dev mod runing python integate tests (#11362)
  • [Feature-10117][UI] Hide node execution when start from the workflow definition list page (#11372)
  • [Feature-11223] support stream task (#11350)
  • [python] Add multiple version of document (#11391)
  • [python] Add support interpreter version 310 and 311-dev (#11170)
  • [Feature-7024] Add waiting strategy to support master/worker can recover from registry lost (#11368)
  • [Feature][Style] Enable spotless to manage imports (#11458)
  • Support reading application properties from kubernetes config map (#11402)
  • [Feature-10182][task-datax] datax access kerberos authenticated bigdata cluster (#11387)
  • [Feature][Style] Make sure there is one and only one empty line at the end of each java file (#11520)
  • [Feature] New task plugin Pytorch (#11498)
  • [feature-11408] add hive option (#11493)
  • feat: Locations Formats the layout if it does not exist (#11535)
  • [Feature][UI] Enable highlight and auto-complete for SQL scripts (#11659)
  • [python] Auto location when create process definition (#11681)
  • [Feature][Task Plugin] Support hive cli task plugin (#11651)
  • [feature][API]feature: add task type list and enable user to add task type to fav (#11727)
  • Add Kubernetes configmap reload to all components (#11730)
  • [Feature-11530] add state history for process instance (#11757)
  • [DSIP-11][python] create workflows from YAML configuration (#11988)
  • [Feature][Registry] Support etcd as registry (#10981) (#11997)
  • [Feat][python] Add parameter environment to task (#11763) (#11989)
  • [feat][python] Support Pytorch task in python api (#11975)
  • [feat][python] Add sagemaker yaml example (#11925)
  • [feat][python] Support OpenMLDB task in python api (#11944)
  • [feat][python] Support MLflow task in python api (#11962)
  • [Feature][PyDolphinScheduler] Support DVC task in pyds #11922 (#11941)
  • [Feature-10273][Datasource] Add dataSource for AWS Athena (#11157) (#12039)
  • [Feat][UI] Update package. (#12046)
  • [python] Support SageMaker task type (#11002)
  • [Feature][Resource Center] Add support for Alibaba Cloud OSS as storage of resource center (#11708)

Improvement

Click to expand
  • [refactor][perf] cache compiled regex pattern (#10099)
  • [improve] Change Mysql Driver (#10220)
  • [Style][UI] Split multilingual modules. (#10256)
  • Clean up unused dependencies and packaging issues (#9944)
  • [Style][UI] Format code. (#10314)
  • Add dolphinscheduler-task-all in api module (#10350)
  • Move datasource params to datasouce plugin module (#10353)
  • issues-10354: upgrade commons-io to fix CVE (#10355)
  • issue-10356: upgrade logback to fix cve (#10357)
  • Remove quartz at WorkerServer (#10358)
  • Add dolphinscheduler-scheduler module (#10360)
  • [improve] Using create or replace function in sql task (#10170)
  • Optimize MasterServer, add MasterRPCService (#10371)
  • change to common lang3 and fix checkstyle (#10380)
  • Upgrade log4j in E2E (#10359)
  • Add some warning log in master (#10383)
  • [Improve] Enhance complement function transformation (#10376)
  • Update chart.yaml dependencies version (#10420)
  • [imporve] Add index to table t_ds_relation_process_instance (#10461)
  • [Improvement][Api] Interface and entity optimization to improve code scalability (#10466). (#10468)
  • [Improve] Add LDAP user not exsitst action config (#10451)
  • Modify obsolete swagger parameters in AuditLogController (#10494)
  • Remove the schedule thread in LowerWeightHostManager (#10310)
  • [Improvement][Metrics] Apply micrometer naming convention to metrics (#10477)
  • change import method access to protected (#10536)
  • [Improve]Remove the code logger.isDebugEnabled() #5918 (#10542)
  • [Improvement][Metrics] Update some metrics names in grafana-demo dashboards (#10432) (#10552)
  • Optimize master log, use MDC to inject workflow instance id and task instance id in log (#10516)
  • add datasource health check to the healthcheck endpoint (#10588)
  • dependent node log output optimization and fix checkstyle (#10593)
  • [Improvement][Master] batch remove TaskInstaceId and workflowInstanceId (#10620)
  • [Improvement][Worker] fixed naming of rpc package (#10614)
  • Add JDK11 on unit-test (#10364)
  • Validate master/worker config (#10649)
  • [Improvement]Delete the getSession in the Registry interface (#10633)
  • [Refactor][UI] Refactor yarn queue manage using NSpace component. (#10680)
  • [Refactor][UI] Refactor token manage using NSpace component. (#10681)
  • [Refactor][UI] Refactor k8s namespace manage using NSpace component. (#10682)
  • [Improvement-10617] Add comment in slot ...
Read more

3.0.1

22 Sep 08:39
Compare
Choose a tag to compare

Bugfix

Click to expand
  • [Improvement][UI] Unified local parameters UI #11190
  • [Fix][UI] Fix bug where icons are displayed in one line. #11320
  • [Improvement-11386][UI] Concise the logic available for task action buttons #11419
  • [Fix][UI] Fix the preTask options are unavailable when creating a tas… #11501
  • S3 Resource center supports bucket customization (#12022)
  • fix status check issue (#11580) (#12030)
  • [LDAP] Config value should return real null instead of 'null' string (#12031)
  • [3.0.1-preapre][cherry-pick]3.0.1 UI (#12020)
  • [Fix][UI] Fix bug where crontab special of month and year #11661
  • [UI] Enable highlight and auto-complete for SQL scripts #11659
  • [Fix][UI] Fix the problem that the pagination component is not centered in the environment management. #11646
  • [Fix][UI] Fix the port in the datasource edit. #11624
  • [Fix][UI] Fix the table header in the workflow instance. #11487
  • [Fix][UI] Fix bug where warn group name not display #11429
  • [Feature-10117][UI] Hide node execution when start from the workflow definition list page #11372
  • [Fix-11366][UI] Workflow instance should not support right-click running #11367
  • [fix-10961][alert server]Change the content of alert to an array (#11033)
  • [Fix][UI] Fix workflow name overlaps bug (#11862) (#12019)
  • [Bug] [Worker] Optimize the getAppId method to avoid worker OOM when kill task (#11994)
  • fix heartBeatTaskCount bug (#12024)
  • Fix kill task failed will cause the taskGroup cannot release and add taskGroup log (#11469) (#12013)
  • Workflow Definition Name Change Validation Optimization. (#11986)
  • add unit test ci timeout (#11655) (#12021)
  • [helm] fix worker template persistentVolumeClaim link (#11248) (#12018)
  • Set master's task running status in runTask to avoid the task group acquire failed, but the task status is in running (#11451) (#12011)
  • [python] Refactor structure to avoid cycle import (#11167)
  • [python] Add support interpreter version 310 and 311-dev (#11170)
  • [bug][python] Fix task switch error when default branch not defined last (#11606)
  • [fix][python] Sql pattern add truncate. (#11666)
  • Add dependent task instance log (#11541) (#12014)
  • If the task processor is not found need to throw error rather than exception (#11461) (#12012)
  • Fix workflow instance may failover many times due to doesn't check the restart time (#11445) (#12010)
  • Fix find last process instance in dependent task is not correct (#11424) (#12009)
  • Fix quartz threadPriority config name error (#11596) (#12015)
  • [Fix-11413] Cannot set task status to kill if the task is not in running(#11414) (#12007)
  • Clear thread name in BaseTaskProcessor (#11422) (#12008)
  • Bug Dependent downstream trigger error when schedule cycle not day. (#11987)
  • [CI] Fix cluster test abnormal (#11688) (#11985)
  • [Improvement-11380][scp-host.sh] Set StrictHostKeyChecking=no option to ssh (#11382) (#11995)
  • optimize the process instance query, change the date time range (#11719) (#11991)
  • [Fix-11051][Task]Fix the process exitValue exception when the process times out and th… (#11099) (#11983)
  • [fix-11404]: make the common.properties to configurable on values.yaml (#11441) (#11967)
  • ][fix-11452][Bug] [Logger Server] Incorrect password regular expression (#11993)
  • Remove logger header in task log file (#11555) (#11968)
  • [Bug] [API] The task priority and process instance priority might be null and cause NPE (#11969)
  • [Bug] [spark-sql] In spark-sql, select both SPARK1 and SPARK2 versions and execute /bin/spark-sql (#11971)
  • Update dolphinscheduler_ddl.sql (#11974)
  • [fix-#11815] fix ck column names contain special characters (#11973)
  • [Bug][Workflow Definition] fix ordinary users can not create depend task (#11961) (#11976)
  • [Fix-11877][UI] Fix the problem that the environment cannot be delete (#11934)
  • [fix-10938]: use dot to replace source and make the default env work in shell (#11937)
  • fix hdfs defaultFs not working (#11823) (#11936)
  • [Quartz] cron did not work as expected (#11932)
  • [Bug] [Master] WorkflowExecuteRunnable will face a infinite loop #11838 #11864 (#11949)
  • [Bug-#11650][worker] #11650 fix SQL type task, stop task does not take effect (#11938)
  • [Fix][db] fix init&upgrade mysql-meta-schema bugs #11887 (#11933)
  • Process definition length too long when copy or import #10621 (#11893)

Documentation

Click to expand
  • [improvement-#11630]fix document about common.properties (#11653)
  • update PyDolphinScheduler documentation link. (#11474)
  • [doc] Correct E2E Doc, fix WorkerGroupPage typo (#11629)
  • Fix homepage email subscription link (#11622)
  • [DOC] should notice that need setting the full path when call the resource file #11620 (#11621)
  • [Doc][Bug] Fix resource center docs for standalone mode (#11600)
  • feat: update slack (#11578)
  • [Improvement-11550] [Doc]Document content update (#11577)
  • [Doc][Security] Update instructions on worker groups (#11483)
  • [Doc][DSIP] Move DSIP docs to the right place (#11442) (#11443)
  • [Doc][Resources] Instruct users to use local storage if they have remote storage mounted to local (#11435)
  • feat: Modifying Slack Links (#11416)
  • [python] Add multiple version of document (#11391)
  • [doc] Refine the deployment documentation (#11361)
  • [Fix-11217] [Doc] add postgresql config in doc: datasource-setting (#11326)
  • [doc] Improve the document introduction of complement (#11275)
  • [DOC] improve zk digest doc (#11703) (#11992)
  • [Doc] Remove re-upload file function in the 3.0.0's doc (#11804) (#11984)
  • [doc] Add how obtain IP address from network card in FAQ (#11311) (#11982)
  • fix doc about sub-process's child node describe (#11972)
  • [fix][doc] Update the registry related configuration in values.yaml (#11444) (#11980)

3.0.0

05 Aug 11:00
Compare
Choose a tag to compare

Feature

Click to expand
  • Add Task group feature to limit the concurrency of taskInstance (#6422)
  • [UI Next] A new version of the UI proposal (#7332)
  • [UI Next] Data Quality (#8275)
  • Support custom timezone (#8310)
  • Dependent tasks can re-run automatically in the case of complement (#8373)
  • Optimize workflowlieage query speed (#8499)
  • Add dataSource for Amazon Redshift (#8527)
  • Refactor cli config dict mechnism (#8593)
  • E2E test adapt M1 chip (#9039)
  • Support output parameters transfer from parent workflow to child work flow. (#9396)
  • [python] Add mechanism cli with simple command (#8516)
  • [python] Add config mechanism and cli subcommand config (#8585)
  • [Feature- 9837][plugin/ui] support FlinkSQL Task (#9840)
  • [Feature-9177][Task] The sql task supports configuring segmentation notation to provide execution of multiple statements (#9917)
  • [Feature][UI] Disabled statistics. (#9935)
  • [Feature][Log]Add timezone information in log output (#9913)
  • [Feature] Remove old ui. (#9909)
  • [Feature][UI Next][V1.0.0-Beta] Add hints to the password repeat. (#9888)
  • [Feature][UI Next][V1.0.0-Beta] Rename SeaTunnel. (#9839)
  • [Feature][UI Next] Add front-end for Apache Zeppelin task plugin (#9597) (#9810)
  • [Feature][Deploy] Add missing default values in dolphinscheduler_env.sh (#9733)
  • [Feature]The utils configuration files are centrally managed under common (#9706)
  • [Feature]Unified exposure method class (#9698)
  • [Feature][UI][V1.0.0-Beta] Modify timeout from 10s to 15s of axios (#9644)
  • [Feature][UI][V1.0.0-Beta] Add dependent task status in dependent task. (#9663)
  • [Feature][Task-Plugin]Add zeppelin task-plugin to support Apache Zeppelin (#9327)
  • [Feature][UI Next][V1.0.0-Beta] Monaco Editor supports theme switching function. (#9521)
  • [Feature][UI Next][V1.0.0-Beta] Change the css style usage of the blue number flopper in the alarm center to css variables. (#9511)
  • [python] Add task decorator for python function (#9496)
  • [python] Support read config in env variable (#9517)
  • [Feature][Task] K8s namespace auth manager (#9303)
  • [Feature][API] Refactor get dependent node information api (#9591)
  • [Feature][UI Next]Added the method of downloading files (#9605)
  • [Feature] [ALERT-9406]enrich alert info when insert new row into DB (#9445)
  • [Feature] [ALERT-9406]add new properties to alert class (#9408)
  • [Feature-9204][alert] Implement alert send status (#9208)
  • [Improve][API] Allowed the non-root user to create the task group. (#9523)
  • [DS-9263][Improvement][master]optimize failover (#9281)
  • [Improvement] change method access (#9390)
  • [improve][python] Specify warning group when deploy process-definition (#9773)
  • [improve][api] Support to upload file without file type suffix (#9553)
  • [Improvement][server] varPool support syntax #{setValue(key=value)} (#9586)
  • [Improvement-9609][Worker]The resource download method is selected according to the configurati… (#9636)
  • [Python] Supports creating or editing resources. (#10823)
  • [Feature-10871] add workflow executing data query (#10875)

Improvements

Click to expand
  • Task back-end plugin optimization, new plugins only need to modify the plugin's own module (#8584)
  • Verify the endtime and starttime when submit/create a cron under workflow (#8874)
  • Dependent The global project can be selected when adding dependencies (#9016)
  • AlertSender optimization and gracefully close, such as MasterServer (#9221)
  • implement use the slot to scan the database (#9228)
  • Slim dist package by migrate python gatewar into apiserver (#9230)
  • [python] Migrate pythonGatewayServer into api server (#9372)
  • [python] Add missing doc about config and connect remote server (#9443)
  • [Master/Worker] Change the task ack to runnning callback (#8719)
  • [Master] add task event thread pool (#9293)
  • [Improvement-10617] Add comment in slot check (#10618)
  • Optimize MasterServer, add MasterRPCService (#10371)
  • [python] Integrate test run strategy change (#10825)
  • Optimize master log, use MDC to inject workflow instance id and task instance id in log (#10516)
  • Add some warning log in master (#10383)
  • [Improvement] Resource authority module database table unique index increased. (#10807)
  • [Improvement] [Alert Server] Add default datasource url in alert server application.yaml for consistency (#10790)
  • [Improvement] heartbeat info improvement (#10699)
  • Add JDK11 on unit-test (#10364)
  • Update SparkRuntimeEnvironment.java (#10549)
  • add datasource health check to the healthcheck endpoint (#10588)
  • [Improvement][UI] Improve monaco-editor component in ts. (#11274)
  • [python] Add directly install without clone source code (#11179)

Bugfix

Click to expand
  • Task back-end plugin optimization, new plugins only need to modify the plugin's own module (#8584)
  • Verify the endtime and starttime when submit/create a cron under workflow (#8874)
  • Dependent The global project can be selected when adding dependencies (#9016)
  • AlertSender optimization and gracefully close, such as MasterServer (#9221)
  • implement use the slot to scan the database (#9228)
  • Slim dist package by migrate python gatewar into apiserver (#9230)
  • [python] Migrate pythonGatewayServer into api server (#9372)
  • [python] Add missing doc about config and connect remote server (#9443)
  • [Master/Worker] Change the task ack to runnning callback (#8719)
  • [Master] add task event thread pool (#9293)
  • [Bug] init docker compose DATABASE env (#10244)
  • [fix] Deploy scp-hosts use fullpath avoid dangling (#10249)
  • [fix][deploy] File overwrite when specific value set to installPath (#10241)
  • [Fix][UI] Set task group name empty of default value (#10243)
  • [Fix#9923] The alarm instance management interface is always in the loading state (#9924)
  • [fix] Task group queue query all list error (#10237)
  • [improve] Change Mysql Driver (#10220)
  • [Fix][UI] Fix issue with treemap depth in workflow relationships. (#10229)
  • [fix-10207] When the registered alarm plugin is empty, the error message is not clear (#10224)
  • [Fix][UI] Fix the problem of multi-language support of workflow instance host in Chinese state. (#10223)
  • [Fix-10103][k8s]Fix k8s Change DataSource Error (#10128)
  • [Fix][UI] add assignment of totalPage (#10165)
  • [python] Fix process definition attr release state not work (#10151)
  • fix#10214 Delete online workflows in batches, prompt information is accurate (#10215)
  • [Bug][Deploy]Fix worker-server path in script scp-hosts.sh (#10208) (#10209)
  • [Fix-10199] [Workflow/Workflow Definition] After deleting messages in batches, the error message is repeated (#10201)
  • [Fix][UI] Fix data-quality component multi-language support. (#10202)
  • [refactor][perf] cache compiled regex pattern (#10099)
  • [Fix-10076] queryUserByToken use server timezone (#10177)
  • fix spcial week init bug (#10183)
  • [Fix][UI] Add seatunnel icon (#10178)
  • [Fix-10057][api] api parameter name misspelling (#10063)
  • [Fix][UI] Fix the problem that the failure strategy field in the time management form is multilingual. (#10169)
  • [Fix]Change the HTTP or HTTPS regular expression (#10166)
  • [Fix][UI] Limit task group resource capacity to only positive integers. (#10158)
  • [Fix][UI] Add the task status of pending. (#10129)
  • [python] Change distribute name avoid confusion with main pkg (#10134)
  • Fix alert server curator health check not work (#10111)
  • [Build][UI] Solve vite config throws terser warning. (#10105)
  • [BUG][TaskGroup] Task group does not take effect (#10093)
  • [fix] Enhance name pre checker in resource center (#10094)
  • [Fix][UI] fulfill missing fields in DataQuality (#10098)
  • fix issue-10061 (#10062)
  • [Feat][UI] Add resource username for function and resource manage (#10086)
  • [Fix-10049] Conditions Task branch flow failed (#10077)
  • [Fix-10082]Resource userName is null and udf function manager add a username (#10064)
  • [Fix][Docker] Support Multi-Architecture Docker Images (#10052)
  • Create process definition with task group and task group priority cannot be save into db (#10078)
  • [Fix-10080]When the created tenant name is too long, an error message will be prompted (#10081)
  • [Bug][Master] fix master task failover (#10065)
  • [Fix][UI] Fix bug where username is empty in file manage (#10070)
  • [Fix][UI] Fix in the Chinese state, there is a problem with the multi-language support of the form. (#10066)
  • [Fix][UI]Modify error message (#10054)
  • [python] Fix pre_submit_check error when local params in tasks (#10032)
  • [Bug] fix run on docker and k8s (#10026)
  • [Fix-10039] Flink run command when perfecting Python jobs (#10042)
  • [dataquality] Fix task commnd null bug (#9974)
  • [fix-9991][worker]fix statement is closed before resultSet.getMetaData() (#10014)
  • [Fix][UI] Fix errorOutputPath column in dataquality page (#10015)
  • [Fix][UI] Support only one file upload on the file manage page. (#10007)
  • [Fix-9975] The selected task instance was recreated when the Master service fail… (#9976)
  • [Fix][UI] Fix the task name validator error. (#10008)
  • [Fix-10002] Fix some bugs in datasource list (#10005)
  • [Bug] [MASTER-9811]fix cmd param to overwrite global param when executing complement (#9952)
  • [Fix][UI][V1.0.0-Beta] Fix the problem of packing failure. (#9962)
  • [Build][UI] Upgrade dependencies. (#9956)
  • [chore] Combine version 206 and 210 sql upgrade to 300 (#9959)
  • [Fix][UI][V1.0.0-Beta] Fix the problem of packing failure. (#9966)
  • [Fix #9925]: fix ellipsis bug in table column (#9936)
  • fix miss fixvalue input (#9933)
  • [Fix][UI Next][V1.0.0-Beta] Fix console bug after deleting a node from the context menu. (#9932)
  • [Fix] add resource capacity required rule (#9922)
  • [Fix][UI Next][V1.0.0-Beta] Fix data quality data not displayed. (#9930)
  • [Fix]Added requ...
Read more

2.0.6

04 Jul 13:09
Compare
Choose a tag to compare
  • [Fix-8645][Api] Resource Re Upload, name duplicate (#8811)
  • [Feature][Dag] Jump back to the list page after the workflow is saved. (#8989)
  • [Fix-8929][DAO]Description Failed to upload the file because the full name was too long (#9020)
  • [Fix-8544][API] The folder's size can't be updated when creating, updating or deleting a resource file int the folder. (#9107)
  • [BUG][ALERT-SERVER]validate script before alert script (#9834)
  • [Fix-10386]LDAP user create with status enable version 2.0.6 (#10508)
  • [Fix-10517] Some important bugs in version 2.0.5 (#10541)
  • [Fix][UI] Fix bytesToSize function calculation error. (#10628)
  • to #9801: fix delete job of quartz not work (#10645)
  • to #9595: fix param lost when execute failed task (#10646)
  • [Improvement] heartbeat info improvement (#10699)
  • [Fix-10672] Dependent task retry bug (#10707)
  • [Fix-10443][server]fix dependent task always running (#10708)
  • [Fix-10673] Workflow recovery bug with failed tasks (#10722)
  • [Fix-10762] [bug] Dependent tasks wait for the workflow to end (#10765)

3.0.0-beta-2

28 Jun 15:43
Compare
Choose a tag to compare

Bugfix

  • [Fix][UI] fix status statistics card height is inconsistent(#10268)
  • [Fix] fix the line.separator problem for alert-plugin test case (#10277)
  • [Fix][UI] add mapping_columns filed for the data_quality (#10286)
  • [Fix] fix database initialize late in standalone (#10296)
  • [Style][UI] split multilingual modules (#10256)
  • [Fix][UI] display the result page when the DB page has no data.
  • [Style][UI] format code. (#10314)
  • [Fix-10317] [UI] fix the tips of startTimeout in the jupyter form (#10319)
  • [Fix][UI] fix the issue that the status statistics on the project overview page were inconsistent(#10328)
  • [Fix-10318][ui] fix EMR Task UI custom parameters Is not correct (#10334)
  • [Build][UI] upgrade dependencies(#10346)
  • [Feature] add dolphinscheduler-task-all in api module (#10350)
  • [Improvement] upgrade commons-io to fix CVE (#10355)
  • [Improvement] upgrade logback to fix cve (#10357)
  • [Improvement] Upgrade log4j in E2E (#10359)
  • [Fix][Deploy] fix run on kubernetes bug (#10379)
  • [Bug][Tools] add datasource url for mysql profile in tools application.yaml (#10399)
  • [Fix-10412] [Standalone] failed to start standalone server according to the official document
  • [Improvement] update chart.yaml dependencies version (#10420)
  • [Fix-10386] LDAP user create with status enable (#10426)
  • [Fix][Python] support same task name in project defferent process definition
  • [Fix-10425] recovery LDAP Config (#10429)
  • [Fix][UI] add loading before file modification data is returned. (#10450)
  • [Fix][Python] correct variable type of SQL type (#10464)
  • [Fix] fix rename form not init bug in file manange (#10471)
  • [Fix-10305][common] AWS config Constant definitions are repeated (#10311)
  • [Fix-10522] [API] fix get executable time null pointer exception(#10524)
  • [Fix] [Worker] fix worker will hang if fails to start (#10501)
  • [Fix] could not find or load main class org.apache.zookeeper.ZooKeeperMain (#10535)
  • [Fix] change K8s postgresql password key name (#10545)
  • [Feature] ui build no inherit from maven proxy (#10590)
  • [Fix][dolphinscheduler-spi] json parse support LocalDateTime (#10440)
  • [Fix] fix standalone start error (#10577)
  • [Fix-10400] fix K8s deployment bitnami version error (#10567)

Document

  • [Doc] Refactor context parameter document (#10227)
  • [Doc] Delete SPRING_DATASOURCE_DRIVER_CLASS_NAME in doc (#10228)
  • [Doc] Upgrade screenshot to latest version in dev document (#10242)
  • [Doc] Update the development environment to build the software version (#10246)
  • [Doc] Add switch task demo (#10247)
  • [Doc] Added local file configuration guide for resource center (#10264)
  • [Doc] Fix api standard error (#10292)
  • [Doc] Add how to use resource in shell task (#10308)
  • [Doc] Upgrade license. (#10365)
  • [Doc] Add how to set the alarm mailbox in faq (#10401)
  • [Doc] Fix wrong class name in doc (#10438)
  • [Doc] Correct installPath when using related path (#10597)

3.0.0-beta-1

27 May 01:25
Compare
Choose a tag to compare

Feature

  • [Feature- 9837][plugin/ui] support FlinkSQL Task (#9840)
  • [Feature-9177][Task] The sql task supports configuring segmentation notation to provide execution of multiple statements (#9917)
  • [Feature][UI] Disabled statistics. (#9935)
  • [Feature][Log]Add timezone information in log output (#9913)
  • [Feature] Remove old ui. (#9909)
  • [Feature][UI Next][V1.0.0-Beta] Add hints to the password repeat. (#9888)
  • [Feature][UI Next][V1.0.0-Beta] Rename SeaTunnel. (#9839)
  • [Feature][UI Next] Add front-end for Apache Zeppelin task plugin (#9597) (#9810)
  • [Feature][Deploy] Add missing default values in dolphinscheduler_env.sh (#9733)
  • [Feature]The utils configuration files are centrally managed under common (#9706)
  • [Feature]Unified exposure method class (#9698)
  • [Feature][UI][V1.0.0-Beta] Modify timeout from 10s to 15s of axios (#9644)
  • [Feature][UI][V1.0.0-Beta] Add dependent task status in dependent task. (#9663)
  • [Feature][Task-Plugin]Add zeppelin task-plugin to support Apache Zeppelin (#9327)
  • [Feature][UI Next][V1.0.0-Beta] Monaco Editor supports theme switching function. (#9521)
  • [Feature][UI Next][V1.0.0-Beta] Change the css style usage of the blue number flopper in the alarm center to css variables. (#9511)
  • [python] Add task decorator for python function (#9496)
  • [python] Support read config in env variable (#9517)
  • [Feature][Task] K8s namespace auth manager (#9303)
  • [Feature][API] Refactor get dependent node information api (#9591)
  • [Feature][UI Next]Added the method of downloading files (#9605)
  • [Feature] [ALERT-9406]enrich alert info when insert new row into DB (#9445)
  • [Feature] [ALERT-9406]add new properties to alert class (#9408)
  • [Feature-9204][alert] Implement alert send status (#9208)
  • [Improve][API] Allowed the non-root user to create the task group. (#9523)
  • [DS-9263][Improvement][master]optimize failover (#9281)
  • [Improvement] change method access (#9390)
  • [improve][python] Specify warning group when deploy process-definition (#9773)
  • [improve][api] Support to upload file without file type suffix (#9553)
  • [Improvement][server] varPool support syntax #{setValue(key=value)} (#9586)
  • [Improvement-9609][Worker]The resource download method is selected according to the configurati… (#9636)

Bugfix

  • [Bug] init docker compose DATABASE env (#10244)
  • [fix] Deploy scp-hosts use fullpath avoid dangling (#10249)
  • [fix][deploy] File overwrite when specific value set to installPath (#10241)
  • [Fix][UI] Set task group name empty of default value (#10243)
  • [Fix#9923] The alarm instance management interface is always in the loading state (#9924)
  • [fix] Task group queue query all list error (#10237)
  • [improve] Change Mysql Driver (#10220)
  • [Fix][UI] Fix issue with treemap depth in workflow relationships. (#10229)
  • [fix-10207] When the registered alarm plugin is empty, the error message is not clear (#10224)
  • [Fix][UI] Fix the problem of multi-language support of workflow instance host in Chinese state. (#10223)
  • [Fix-10103][k8s]Fix k8s Change DataSource Error (#10128)
  • [Fix][UI] add assignment of totalPage (#10165)
  • [python] Fix process definition attr release state not work (#10151)
  • fix#10214 Delete online workflows in batches, prompt information is accurate (#10215)
  • [Bug][Deploy]Fix worker-server path in script scp-hosts.sh (#10208) (#10209)
  • [Fix-10199] [Workflow/Workflow Definition] After deleting messages in batches, the error message is repeated (#10201)
  • [Fix][UI] Fix data-quality component multi-language support. (#10202)
  • [refactor][perf] cache compiled regex pattern (#10099)
  • [Fix-10076] queryUserByToken use server timezone (#10177)
  • fix spcial week init bug (#10183)
  • [Fix][UI] Add seatunnel icon (#10178)
  • [Fix-10057][api] api parameter name misspelling (#10063)
  • [Fix][UI] Fix the problem that the failure strategy field in the time management form is multilingual. (#10169)
  • [Fix]Change the HTTP or HTTPS regular expression (#10166)
  • [Fix][UI] Limit task group resource capacity to only positive integers. (#10158)
  • [Fix][UI] Add the task status of pending. (#10129)
  • [python] Change distribute name avoid confusion with main pkg (#10134)
  • Fix alert server curator health check not work (#10111)
  • [Build][UI] Solve vite config throws terser warning. (#10105)
  • [BUG][TaskGroup] Task group does not take effect (#10093)
  • [fix] Enhance name pre checker in resource center (#10094)
  • [Fix][UI] fulfill missing fields in DataQuality (#10098)
  • fix issue-10061 (#10062)
  • [Feat][UI] Add resource username for function and resource manage (#10086)
  • [Fix-10049] Conditions Task branch flow failed (#10077)
  • [Fix-10082]Resource userName is null and udf function manager add a username (#10064)
  • [Fix][Docker] Support Multi-Architecture Docker Images (#10052)
  • Create process definition with task group and task group priority cannot be save into db (#10078)
  • [Fix-10080]When the created tenant name is too long, an error message will be prompted (#10081)
  • [Bug][Master] fix master task failover (#10065)
  • [Fix][UI] Fix bug where username is empty in file manage (#10070)
  • [Fix][UI] Fix in the Chinese state, there is a problem with the multi-language support of the form. (#10066)
  • [Fix][UI]Modify error message (#10054)
  • [python] Fix pre_submit_check error when local params in tasks (#10032)
  • [Bug] fix run on docker and k8s (#10026)
  • [Fix-10039] Flink run command when perfecting Python jobs (#10042)
  • [dataquality] Fix task commnd null bug (#9974)
  • [fix-9991][worker]fix statement is closed before resultSet.getMetaData() (#10014)
  • [Fix][UI] Fix errorOutputPath column in dataquality page (#10015)
  • [Fix][UI] Support only one file upload on the file manage page. (#10007)
  • [Fix-9975] The selected task instance was recreated when the Master service fail… (#9976)
  • [Fix][UI] Fix the task name validator error. (#10008)
  • [Fix-10002] Fix some bugs in datasource list (#10005)
  • [Bug] [MASTER-9811]fix cmd param to overwrite global param when executing complement (#9952)
  • [Fix][UI][V1.0.0-Beta] Fix the problem of packing failure. (#9962)
  • [Build][UI] Upgrade dependencies. (#9956)
  • [chore] Combine version 206 and 210 sql upgrade to 300 (#9959)
  • [Fix][UI][V1.0.0-Beta] Fix the problem of packing failure. (#9966)
  • [Fix #9925]: fix ellipsis bug in table column (#9936)
  • fix miss fixvalue input (#9933)
  • [Fix][UI Next][V1.0.0-Beta] Fix console bug after deleting a node from the context menu. (#9932)
  • [Fix] add resource capacity required rule (#9922)
  • [Fix][UI Next][V1.0.0-Beta] Fix data quality data not displayed. (#9930)
  • [Fix]Added request header validation (#9915)
  • [Fix-9906] After the serial wait execution strategy stops the running workflow instance, the instance will be woken up and executed if there is a wait instance. (#9907)
  • [Fix][UI-NEXT] Disable file name (#9893)
  • [Fix][UI Next][V1.0.0-Beta] Fix jdbc connect parameter validate bug (#9891)
  • [Fix][UI Next][V1.0.0-Beta] Fix the task group queue display error. (#9890)
  • [Fix][UI Next][V1.0.0-Beta] Fix edge deletion unhandled and remove the useless nextNodes. (#9875)
  • [Fix][UI Next][V1.0.0-Beta] Fix some errors on the datasource page. (#9877)
  • [Fix][UI Next][V1.0.0-Beta] Fix the page input error on the token manage page. (#9879)
  • [Fix][UI Next][V1.0.0-Beta] Fix bug where logo disappeared when format dag. (#9864)
  • [Fix-9856]Failed to upload the file but get misleading error message (#9861)
  • [Fix-9868] A task flow definition isolates the runs of different execution strategies by version numbers. (#9869)
  • [Bug] [worker] fix CommandType TASK_EXECUTE_RUNNING_ACK don't consumed (#9849) (#9850)
  • [Fix][UI Next][V1.0.0-Beta] Fix the problem that there is no prompt for selecting users. (#9858)
  • [fix] fix query access token for specified user failed (#9855)
  • [Fix][UI Next][V1.0.0-Beta] Add validation rules to dependency of the dependent task. (#9844)
  • Serial wait for subsequent execution (#9847)
  • [Bug-9829][Api] Fix schedule timezone (#9830)
  • [Fix]fix the search bar echoes abnormally (#9835)
  • [BUG][ALERT-SERVER]validate script before alert script (#9834)
  • [Fix] json object serialization error (#9828)
  • [Fix][UI][V1.0.0-Beta] Fix the problem of wrong workflow name. (#9833)
  • Task queue status update. (#9832)
  • [Fix][UI] Hide the version control button in workflow instance (#9825)
  • The node ID under serial is greater than the current ID. (#9820)
  • [Fix][UI] Fix timing modal start time and end time bug (#9815)
  • [Bug] [Website-9764]add http alert doc and change body param to unnecessary (#9792)
  • [Fix-9796] The task queue can be executed normally. (#9797)
  • [Fix][UI Next][V1.0.0-Beta] Add a return button and set the form to disabled on the file detail page. (#9802)
  • [Fix][Next-UI][V1.0.0-Beta] Fix placeholder error on the task queue page. (#9805)
  • [Fix-9352][datasource-api]data source client uses a cache with an expiration time (#9353)
  • revert delete dumplicate task_definition_log ddl (#8567)
  • [Bug][dep] Bump httpclient version to 4.5.13 (#9631)
  • [bug][db] Table relation_project_user have duplicate record (#9536)
  • [fix-9785][UI Next][dev] dynamic change height in viewlog when full screen (#9788)
  • [feat] Update README (#8261)
  • [Bug][Api] Failed to rerun the workflow instance (#9795)
  • [Bug]Fix log request errors (#9791)
  • Task group queue supports fuzzy search. (#9784)
  • [Bug] [Alert-9786] Http alert Get method not working (#9787)
  • [Bug] [Website-9764]add script doc to the alert function (#9765)
  • A task instance that normally queries the serial wait state. (#9777)
  • [Bug]Fix groupId initialization value problem (#9775)
  • [Fix][UI Next][V1.0.0-Beta] Add the 'startNodeList' field to the 'start-process-instance' interface. (#9769)
  • [Bug]Fix right-click menu function error (#9766)
  • [Fix][UI Next][V1.0.0-Beta] Fix cannot select the workload scheduling ...
Read more

3.0.0-alpha

18 Apr 11:11
Compare
Choose a tag to compare

[Feature]
[#6422] Add Task group feature to limit the concurrency of taskInstance
[#7332] [UI Next] A new version of the UI proposal
[#8275] [UI Next] Data Quality
[#8310] Support custom timezone
[#8373] Dependent tasks can re-run automatically in the case of complement
[#8499] Optimize workflowlieage query speed
[#8527] Add dataSource for Amazon Redshift
[#8593] Refactor cli config dict mechnism
[#9039] E2E test adapt M1 chip
[#9396] Support output parameters transfer from parent workflow to child work flow.
[#8516] [python] Add mechanism cli with simple command
[#8585] [python] Add config mechanism and cli subcommand config

[Improvements]
[#8584] Task back-end plugin optimization, new plugins only need to modify the plugin's own module
[#8874] Verify the endtime and starttime when submit/create a cron under workflow
[#9016] Dependent The global project can be selected when adding dependencies
[#9221] AlertSender optimization and gracefully close, such as MasterServer
[#9228] implement use the slot to scan the database
[#9230] Slim dist package by migrate python gatewar into apiserver
[#9372] [python] Migrate pythonGatewayServer into api server
[#9443] [python] Add missing doc about config and connect remote server
[#8719] [Master/Worker] Change the task ack to runnning callback
[#9293] [Master] add task event thread pool

[BUG]
[#7236] Failed to create tenant using S3a Minio
[#7416] Text file busy
[#7896] When the project is authorized, it will generate a duplicate authorized project
[#8089] start server failed because can't connect to postgresql
[#8183] message:datasource plugin 'spark' is not found.
[#8202] MapReduce generated command built-in parameter location is wrong
[#8751] Change param user, queue do no work in ProcessDefinition
[#8756] Process using the dependence component cannot migrate between test and prod environment
[#8760] Resource file deletion conditions
[#8791] Rectify the issue with affecting the original node's data when editing the form of the copied node.
[#8951] Worker resources are exhausted and cause downtime
[#9243] Some types of alarms can't display project name

3.2.0

30 Aug 12:28
Compare
Choose a tag to compare
3.2.0 Pre-release
Pre-release

Feature

Click to expand

Improvement

Click to expand
  • [Improvement-11773][api] Optimize the log printing of the api module according… (#11782) @guowei-su
  • [Improvement][Test] Give an example on replacing Powermock with Mockito (#11588) @EricGao888
  • [Improvement-11857][Spark] Remove the spark version of spark task (#11860) @rickchengx
  • [Improvement][SQL Task]use default sql segment separator (#10869) @zhuxt2015
  • [Improvement][Test] Remove powermock in dolphinscheduler-master module (#12143) @EricGao888
  • [Improvement][Test] Remove powermock in service and server modules (#12164) @EricGao888
  • [Improvement-12144] Remove powermock in dolphinschduler-api (#12169) @rickchengx
  • [Improvement-11880][Hivecli] Improve the UI of HiveCli (#11882) @rickchengx
  • [Improvement][Test] Remove the usage of powermock in datasource-plugin module (#12194) @EricGao888
  • [Improvement][Test] Remove powermock from DatasyncTaskTest (#12270) @EricGao888
  • [Improvement][Test] Fully remove the usage of powermock from the whole project (#12244) @EricGao888
  • [Improvement][Test] Migrate all UT cases from jUnit4 to jUnit5 in task-plugin module as an example (#12299) @EricGao888
  • [Improvement-12296][Test] Close static mock (#12300) @rickchengx
  • [Improvement-12289][k8s] update the common.properties in values.yaml (#12290) @rickchengx
  • [Migrate][Test] Migrate all UT cases from jUnit 4 to jUnit 5 in alert and api module (#12337) @EricGao888
  • [improvement] Migrate all UT cases from jUnit4 to jUnit5 in dao module (#12328) @fuchanghai
  • [Migrate][Test] Migrate all UT cases from jUnit 4 to 5 in datasource, registry and data-quality modules (#12352) @EricGao888
  • [Migrate][Test] Migrate all UT cases from jUnit 4 to 5 in common, service, and spi modules (#12363) @DarkAssassinator
  • [Improvement][Test] Block the usage of powermock and move mockito dependencies from sub-modules to root pom (#12311) @EricGao888
  • [Migrate][Test] Migrate all UT cases from jUnit 4 to 5 in master, worker, and remote modules (#12362) @DarkAssassinator
  • [Migrate][Test] Migrate other UT cases from jUnit 4 to 5, change @ignore to @disable (#12382) @DarkAssassinator
  • [Improvement] add presto in datax (#12371) @fuchanghai
  • [Migrate][Test] Add Spotless step to check and block jUnit4 imports (#12398) @EricGao888
  • [Improvement][Style] Fix remaining style errors and wildcard imports (#12412) @EricGao888
  • [Improvement-12333][Test] Migrate all UT cases from jUnit 4 to jUnit 5 in microbench and e2e module (#12348) @rickchengx
  • [Improvement-12446][CI] Run mvn spotless apply (#12447) @DarkAssassinator
  • [Improvement][Style] Update spotless junit4 check scope (#12450) @EricGao888
  • [Refactor] Migrate all command-related interface functions from ProcessServiceImpl (#12474) @DarkAssassinator
  • Update the project introduction (#12501) @davidzollo
  • [Improvement] add springdoc-openapi-ui (#12379) @fuchanghai
  • [Refactor]migrate some task dao functions from ProcessServiceImpl to dao modules. (#12505) @DarkAssassinator
  • [Refactor] Migrate all workergroup-related interface functions from ProcessServiceImpl (#12493) @DarkAssassinator
  • Fix alert status spelling error (#12592) @qingwli
  • [Improvement-12536][k8s] Support the command for the container in k8s task plugin (#12538) @rickcheng...
Read more

2.0.5-release

07 Mar 03:23
Compare
Choose a tag to compare

[Improvements]
[#8110] The fault-tolerant process of the worker is optimized, so that when the server is under too much pressure, the worker service is interrupted, and the task can be transferred to other workers to continue.
[#8445] Optimize the page display flag of forbidden tasks to distinguish the display of normally executed tasks
[#8441] Add a prompt on the task box to display all the long task names.
[#8187] The function of re uploading files is added in the resource center. When users need to modify the execution script, they do not need to reconfigure the task parameters to realize the function of automatically updating the execution script.
[#8169] After the modification workflow is optimized, the page remains in the DAG page and will not jump to the list page.
[#8285] In the dingtalk alarm plugin, add the markdown information type in the alarm content.

[BUG]
[#8213] Fixed the task runs incorrectly when the worker group contains uppercase letters.
[#8347] Fixed the workflow cannot be stopped when the task fails and retries
[#8135] Fixed JDBC connection parameter not input '@'
[#8367] Fixed complement may not end normally
[#8170] Fix the sub workflow that cannot be entered from the page

2.0.3-release

25 Jan 05:20
Compare
Choose a tag to compare

Download:
https://dolphinscheduler.apache.org/en-us/download/download.html

Feature
[#7804] DingTalk alert plugin adds signature.
[#7857] Problems caused by modifying the administrator user information.
[#7675] Supports whether SQL is placed in one session
[#7571] Add dependent tips about success or failure

Improvement
[#7990] zkRoot in conf/config/install_ config does not take effect
[#7713] The data source password in the log is not encrypted

Bug:
[#8131] add delete workflow instance when delete process definition
[#8098] udf sub resource manage edit modal cant close
[#8081] process is always running: netty writeAndFlush without retry when failed leads to worker response to master failed
[#7986] After deleting the running workflow, the master keeps brushing the error log
[#8056] Edit the bug of worker grouping in environment management.
[#7512] dependent node ui dislocation
[#7913] Error in querying historical version information of workflow
[#7982] common task log in master no need to print separately
[#7962] The global parameters of the sub_process node are not passed to the associated workflow task
[#7953] query log can not show contents when task log in master on k8s
[#7815] There are duplicate processes in the process definition list
[#7698] process instance is always running: task is failure when process instance FailureStrategy.END
[#7732] field ‘is_directory’ in t_ds_resources table has error type in PostgreSQL database
[#7883] Repair JDBC connection of Oracle
[#7538] when there is a forbidden node in dag, the execution flow is abnormal
[#7789] querySimpleList return wrong projectCode