Releases: apache/linkis
Releases · apache/linkis
Apache Linkis(incubating) Release-1.1.2-Incubating
Apache Linkis(incubating) Release-1.1.2-Incubating-RC2
Release-1.1.2-Incubating-RC2
Release Notes:https://linkis.apache.org/docs/1.1.2/release-notes-1.1.2
Apache Linkis(incubating) Release-1.1.2-Incubating-RC1
Release-1.1.2-Incubating-RC1
Release Notes:https://linkis.apache.org/docs/1.1.2/release-notes-1.1.2
Apache Linkis(incubating) Release-1.1.1
Apache Linkis(incubating) Release-1.1.0
Apache Linkis(incubating) Release-1.0.3-Incubating
Linkis 1.0.2 release(Not Apache Release)
Linkis-1.0.2 includes all of Project Linkis-1.0.2.
This release mainly introduces Flink-support into Linkis ecosystem.
The following key features are added:
- Flink-EngineConn which offers solid support for Flink jobs. Executing, debugging and monitoring Flink SQL or applications are now available, together with SQL-enhancement ability powered by Linkis Orchestrator.
- LinkisManagerClient which enables direct access to LinkisManager. Submitting and managing OnceJob rely on this feature.
Abbreviations:
CGS: Computation Governance Services
PES: Public Enhancement Services
MGS: Microservice Governance Services
New Feature
EngineConn
- Linkis-936 [CGS-LinkisOnceEngineconn] supports OnceEngineExecutor
EnginePlugin
- Linkis-935 [CGS-EngineConnPlugin-Flink] supports Flink EngineConn
- Linkis-947 [CGS-EngineConnPlugin-Flink] supports executing Flink SQL and Flink applications
- Linkis-948 [CGS-EngineConnPlugin-Flink] multiple-datasource support for Flink EngineConn
- Linkis-949 [CGS-EngineConnPlugin-Flink] monitoring Flink Metrics
ComputationClient
- Linkis-937 [CGS-LinkisComputationClient] supports OnceEngineExecutor client
Enhancement
- Linkis-953 [CGS-LinkisManager] label supports '-' in hostname
- Linkis-925 [MGS-LinkisServiceGateway] fix weak password in linkis gateway
- Linkis-950 [CGS-LinkisEngineConnManager] support both ip address and hostname for service discovery
- Linkis-967 [CGS-LinkisEntrance] remove instance-label client dependency, Solve the host name and ip judgment abnormality in the gateway router, exclude the pom dependency to pentaho-aggdesigner-algorithm jar.
- Linkis-963 [PES-LinkisBmlServer] default download user changed to jvm user, and supports to set default download user by configuration.
Bugs Fix
- Linkis-938 [CGS-LimkisMnagager] fixes a serial execution bug
- Linkis-952 [CGS-LinkisEngineConn] fixes a redundant thread bug
- Linkis-943 [CGS-EngineConnPlugin-Hive] fixes a Hive3.0 compilation error
- Linkis-961 [CGS-EngineConnPlugin-Flink] fixes a Flink-EnginePlugin compilation bug
- Linkis-966 [CGS-EngineConnPlugin-Hive] [CGS-EnginePlugin-Spark] Solve Spark and hive compatibility issue
Credits
The release of Linkis 1.0.2 is inseparable from the contributors of the Linkis community. Thanks to all the community contributors!
Compiled Version
- Tencent Cloud:
Linkis 1.0.1 release(Not Apache Release)
Linkis-1.0.1 includes all of Project Linkis 1.0.1.
This release mainly fixes performance bugs in high concurrency scenarios, optimizes the resource management module, and optimizes the Spark engine to support the use of FATE (Federated Machine Learning), and fixes multiple discovered bugs.
Abbreviations:
CGS: Computation Governance Services
PES: Public Enhancement Services
MGS: Microservice Governance Services
Enhancement
- Linkis-906 [Commons] HTTPClient supports setting contentType when uploading files.
- Linkis-897 [Commons] HttpClient supports Response whose content is empty.
- Linkis-877 [Commons] HDFS FileSystem object supports caching.
- Linkis-862 [Commons] Optimize ShutdownHook to be compatible with windows operating system.
- Linkis-889 [Orchestrator] ioClient supports isolating a separate Orchestrator.
- Linkis-880 [PES-JobHistory] JobHistory supports recording the original code of the task submitted by the user.
- Linkis-879 [PES-InstanceLabel] InstanceLabel module support to modify and remove instance label information.
- Linkis-864 [PES-ErrorCode] Optimize the initialization sql of the error code, remove the default value of the ID field.
- Linkis-872 [PES-Configuration] Management console configuration supports real-time modification of cached values.
- Linkis-844 [PES-Configuration] The management console supports setting memory unit, the default unit is GB.
- Linkis-869 [CG-Entrance] Optimize task submission record client IP for link analysis.
- Linkis-848 [CG-LimkisMnagager] The tenant label of ECM should be retained to facilitate the restart.
- Linkis-863 [EnginePlugin-Spark] Spark EngineConn startup parameters support the setting of native parameters to support the use of Fate, such as setting the python version.
Bugs Fix
- Linkis-900 [Commons] Logging is recommended to be optimized, because the user may pass in Null and thus cause an exception.
- Linkis-851 [Commons] The version of the jackson-core-asl package is incorrect after linkis1.0 is compiled with the CDH version.
- Linkis-896 [PES-UDF] UDFClient has a bug that can only get UDF but cannot get method functions.
- Linkis-873 [PES-BML] Optimize the execution order of the BML EngineConnHook to fix the bug of not being able to read resource files.
- Linkis-850 [PES-Configuration] Failed to save IDE-JDBC parameter configuration in Linkis1.0 management console.
- Linkis-847 [CG-Entrance] When a task contains thousands of SQL, the execution may not be correct.
- Linkis-842 [CG-Entrance] Task execution fails when the sql limit setting is greater than 5000.
- Linkis-841 [CG-Entrance] Log is not fully recorded in the stress test of 10,000 tasks.
- Linkis-840 [CG-Entrance] During the stress test of 10,000 tasks, the task status is in the Inited/Running/Scheduled status and no longer updated.
- Linkis-846 [CG-LinkisMnaager] When the parameter of the number of engine is set to 2 in management console, however, 18 engines is started when schedulis is scheduling.
- Linkis-845 [CG-LinkisMnaager] The value of concurrent label may be null in ECM management .
- Linkis-843 [EnginePlugin-Hive] When killing a complex hive task, Hive MR of Yarn is not simultaneously killed.
Credits
The release of Linkis 1.0.1 is inseparable from the contributors of the Linkis community. Thanks to all the community contributors!
Compiled Version
- Tencent Cloud:
Linkis 1.0.0 release(Not Apache Release)
Linkis-1.0.0 includes all of Project Linkis 1.0.0.
This release mainly brings in Linkis-Orchestrator, optimizing lots of architecture code of 'Computation Governance', adding support for OnceEngineConn, which used for batch jobs and streaming jobs, and simplifies the deployment of Linkis.
Meanwhile, the /docs directory is removed, documents are moved to the Linkis-Doc repository.
New Feature
Orchestrator
- Linkis-746 Add the core module of Linkis Orchestrator, and provide top-level architecture and extension interfaces for the Orchestrator module.
- Linkis-739 Add the Linkis OLAP scene task Orchestrator module.
- Linkis-733 Add linkis-code-orchestrator module to support OLAP job requests.
EngineConn
- Linkis-799 Add linkis-once-engineconn module for Linkis1.0 to support for OnceEngineConn, which used for batch jobs and streaming jobs.
Web
- Linkis-757 Add Linkis front-end web module, provide Global history, Resource management, Parameter configuration, Global variables and other modules.
Client
- Linkis-754 Add Linkis-Cli module, allowing submitting codes through shell command.
Enhancement
- Linkis-818 Rename Application EngineConn in Eureka to Linkis-CG-EngineConn to follow the naming convention.
- Linkis-807 Enhance the Linkis1.0.0 common module, and provide lots of tools classes for other modules.
- Linkis-805 Use Utils.tryCatch to catch exception, instead of try...catch directly.
- Linkis-797 Add Chinese and English support for all logs.
- Linkis-789 Constant substitution.
- Linkis-788 Optimize some magic values in DESUtil class.
- Linkis-782 Optimize some magic values in Linkis codes.
- Linkis-781 Optimize error code definition.
- Linkis-777 Merge publicservice with other publicenhancement microservices, and add the combined packaging method.
- Linkis-775 Optimize Linkis computation governance module and make it adapt to Linkis1.0's new architecture.
- Linkis-768 Remove all commented code and optimize HttpBmlClient .
- Linkis-767 Increase the regularity of the code, remove all @author and @Date at the top of each class.
- Linkis-752 Optimize Linkis engineconn module and make it adapt to Linkis1.0's new architecture.
- Linkis-751 Optimize Linkis spring cloud services module and make it adapt to Linkis1.0's new architecture.
- Linkis-749 Add a new linkis-io-file-client module, modify some comments and add some license.
- Linkis-748 Optimize the public-enhancements module and make it adapt to Linkis1.0's new architecture.
Bugs Fix
- Linkis-825 Solve the recursive calling problem in SparkPythonExecutor's close method.
- Linkis-816 Shutdown all executors instead of only closing the default one for multi-executor EngineConns.
- Linkis-815 Fix Orchestrator AsyncExec ResultSet wait didn't get notified bug and remove Orchestrator limit 5000 constraint.
- Linkis-804 Fix the bug where "limit 5000" would not be automatically added if the user submit a sql like 'select*'.
- Linkis-803 Fix the thread safety issues of SSOUtils, which may cause OOM exception.
- Linkis-787 Fix the package conflict problem caused by hive version upgrade.
- Linkis-780 Fix the value conversion problem of RM module in capacity scheduler.
- Linkis-729 Solve the problem that cookies does not take effect after login, and cookies added to Action is not delivered to gateway by the HttpClient.
Credits
The release of Linkis 1.0.0 is inseparable from the contributors of the Linkis community. Thanks to all the community contributors!
Compiled Version
- Tencent Cloud:
Contributors
Date and author
Assets 2
Linkis 1.0.0-RC1 release(Not Apache Release)
Linkis 1.0.0-RC1 is an important release that marks the start of the 1.0 version, including all of Project Linkis 1.0.0 RC1.
With the design of supporting full-stack computation/storage engines, this release brings a brand new implemention for the architecture of computation governance, provides a variety of new features and Provides a strong compatibility guarantee for later 1.x version.
New Feature
- [Linkis-572] Add the EngineConn management plug-in in the Entrance module.
- [Linkis-576] Add a new module EngineConnPlugin for the Linkis1.0 architecture to simplify the user's implementation of a new Linkis underlying computation storage engine.
- [Linkis-584] Add a new LinkisManager Common module to collect all commonly used interfaces in LinkisManager.
- [Linkis-585] Add a new LinkisManager Monitor module to monitor the status of instances managed by LinkisManager.
- [Linkis-587] Add the common module of EngineConn of Linkis1.0 architecture.
- [Linkis-591] Add the Common Module for Computation Governance.
- [Linkis-600] Add the linkis-manager-service-common module, it can be used by AppManager and ResourceManager.
- [Linkis-602] Add the Persistence Module for Linkis Manager to provide the persistence capability.
- [Linkis-604] Add the ability of microservice tagging, and support the tagging of each micro-service.
- [Linkis-599] Add the Linkis ResourceManager Common Module.
- [Linkis-609] Add a universal label module to provide basic support for linkis label capability.
- [Linkis-630] Add the launch module of EngineConn of Linkis1.0 architecture.
Enhancement
- [Linkis-569] Enhance the processing capacity of Linkis RPC, and backward compatibility with Linkis0.x RPC.
- [Linkis-570] Enhance the ability of RM and integrate it into Linkis Manager.
- [Linkis-574] Enhance the label management function of Linkis, so it can manage the Label of EngineConnManager and EngineConn.
- [Linkis-580] Enhance SparkEngineConn module, adapts to Linkis1.0's new architecture.
- [Linkis-583] Enhance ShellEngine, adapts to Linkis1.0's new architecture.
- [Linkis-590] Enhance the core module of EngineConn, provide the abstractions and interfaces of the core functions.
- [Linkis-592] Enhance JDBCEngine, adapts to Linkis1.0's new architecture.
- [Linkis-594] Enhance IOEngine, adapts to Linkis1.0's new architecture.
- [Linkis-596] Enhance HiveEngine, adapts to Linkis1.0's new architecture.
- [Linkis-597] Enhance PipelineEngine, adapts to Linkis1.0's new architecture.
- [Linkis-608] Enhance scritps for starting and shutting down services, optimize the steps of construction.
- [Linkis-618] Enhance the Context Service module of Public Enhancements and adapt it with Linkis 1.0.
- [Linkis-638] Enhance the encryption function of linkis-common.
- [Linkis-643] Enhance the Linkis DataSource Module to adapt to the structure of Linkis 1.0.
- [Linkis-645] Enhance the linkis-hadoop module to support the new feature of multi hadoop cluster.
- [Linkis-648] Enhance linkis-httpclient module to support PUT and Delete requests.
- [Linkis-650] Optimize linkis-mybatis module to integrate with mybatis-plus.
- [Linkis-652] Enhance the module of linkis-protocol, adapts to Linkis1.0's new architecture.
- [Linkis-654] Enhance the Linkis Gateway Module to support forwarding requests by labels in Linkis 1.0.
- [Linkis-655] Enhance the Linkis JobHistory Module to support task cache, label persistent and manager of job history.
- [Linkis-656] Optimize the implementation of the serializer of the linkis-rpc module.
- [Linkis-660] Enhance the linkis-scheduler module to automatically clean up idle Consumers.
- [Linkis-662] Enhance the linkis-user-control Module, adapts to Linkis1.0's new architecture.
- [Linkis-663] Enhance the linkis-entrance module, adapts to Linkis1.0's new architecture.
- [Linkis-666] Optimize directory hierarchy structures and remove useless directories.
- [Linkis-668] Enhance the linkis-storage module to support viewFS and multi hadoop cluster.
- [Linkis-674] Enhance the PythonEngineConn, adatps to Linkis1.0 architecture.
- [Linkis-675] Optimize directories and files, update the version of all pom files.
Credits
The release of Linkis 1.0.0-RC1 is inseparable from the contributors of the WeDataSphere community. They selflessly contributed their time and made lots of valueable suggestions for the document construction, deployment simplification and user experience perfection. Without their help, Linkis 1.0.0-RC1 can not be released so smoothly. Thank you all Contributors of the community!
Upgrade Wizard
In principle, Linkis 1.0.0-RC1 is backward compatible with all the interfaces of Linkis0.X, so the upgrade has little effect to the upper layer applications.
As a computation middleware, except for the related user configuration data of PublicService, other data in the Linkis database has no need for migration.
For the migration of the user configuration data of PublicService, please refer to: Linkis1.0.0-RC1 Upgrade Document