Dinky-1.0.1 Release Note

1.0.1 is a BUG repair version, no database upgrade changes, can be directly upgraded

About SCALA version: The release version uses Scala-2.12. If your environment must use Scala-2.11, please compile it yourself. Please refer to Compile Deployment and change the scala-2.12 in the profile. for scala-2.11

New Feature

• Add some Flink Options classes to trigger shortcut prompts
• Implement automatic scrolling of console logs during data development

Fix

• Fixed the problem that the SMS alarm plug-in was not packaged
• Fixed NPE exception and some other issues when creating UDF
• Fixed job type rendering exception when creating tasks
• Fixed the issue of page crash when viewing Catalog during data development
• Fixed parameter configuration problem when using add jar with s3
• Fix some issues with rs voluntary agreement
• Fixed the routing error jump problem in the quick navigation in data development
• Fixed the issue that the console was not closed when selecting UDF task type
• Fixed the issue where the decimal data type exceeds 38 digits (more than 38 digits will be converted to string)
• Fixed the problem that some pop-up boxes could not be closed
• Fixed the problem that global variables cannot be recognized in application mode
• Fixed the problem of array out-of-bounds when obtaining container in application mode
• Fix the problem that add file cannot be parsed

Optimization

• Optimize some front-end request URLs into agreed constants
• Optimize the startup script and remove the FLINK_HOME environment variable loading
• Optimize the prompt message when the password is incorrect
• Optimize tag display of data development tasks
• Turn off automatic preview in the data development editor
• Optimize the expression variable definition method, changing from file definition to system configuration definition
• Optimize the prompt message that query statements are not supported in application mode
• Optimize the rendering effect of FlinkSQL environment list
• Optimize the environment check exception prompt when building GIT projects
• Optimize the cluster for NPE issues that may occur during heartbeat detection

Document

• Added built-in variable documents for synchronization of the entire library
• Optimize document version
• Add EXECUTE JAR task DEMO
• Optimize some copywriting tips when creating cluster configurations
• Optimize some paths in the entire database synchronization document

Dinky-1.0.0 Release Note

Upgrade Instructions

• Dinky 1.0 is a refactored version that restructures existing functions, adds several enterprise-level functions, and fixes some limitations of 0.7. There is currently no direct upgrade from 0.7 to 1.0. It is recommended to redeploy version 1.0.
• Starting from Dinky 1.0, the Dinky community will no longer maintain all versions before 1.0.
• Starting from Dinky version 1.0, the Dinky community will provide support for Flink 1.14.x and above, and will no longer maintain Flink versions below 1.14. At the same time, Flink has added some new features, which Dinky will gradually support.
• Dinky 1.0 and later versions, every time Flink adds a new major version, Dinky will also add a new major version, and at the same time, a Dinky-Client version will be eliminated depending on the situation. Deleted versions may be subject to a vote, and the results of the vote determine the deleted version.
• Four RC versions have been released successively during the reconstruction process. The RC version can be upgraded, but it is still recommended to redeploy the 1.0-RELEASE version. Avoid some location issues.
• Users of Dinky version 0.7 can continue to use version 0.7, but no maintenance and support will be provided. It is recommended to install version 1.0 as soon as possible.

The changes from version 0.7 to version 1.0 are relatively large, and there are some incompatible changes. Users using version 0.7 cannot directly upgrade to version 1.0. It is recommended to redeploy version 1.0.

Incompatible changes

• CDCSOURCE dynamic variable definition changed from ${} to #{}
• Global variables such as _CURRENT_DATE_ are removed and replaced by expression variables
• Flink Jar task definition is changed from form to EXECUTE JAR syntax
• The definition of dinky-app-xxxx.jar in Application mode is moved to the cluster configuration
• The database DDL part is not compatible with upgrades
• The type attribute of Dinky's built-in Catalog is changed from dlink_catalog to dinky_catalog

Refactoring

• Reconstruct data development
• Reconstruct the operation and maintenance center
• Reconstruct the registration center
• Reconstruct the Flink task submission process
• Reconstruct the Flink Jar task submission method
• Reconstruct CDCSOURCE entire library synchronization code architecture
• Reconstruct Flink task monitoring and alarming
• Reconstruct permission management
• Reconstruct system configuration to online configuration
• Refactor push DolphinScheduler
• Reconstruct the packaging method

new function

• Data development supports code snippet prompts
• Support real-time printing of Flink table data
• Console real-time printing task submission log
• Support Flink CDC 3.0 entire database synchronization
• Support custom alarm rules and custom alarm templates
• Support Flink k8s operator submission
• Support proxy Flink webui access
• Added Flink task Metrics to monitor custom charts
• Support Dinky jvm monitoring
• Added resource center functions (local, hdfs, oss) and expanded rs protocol
• Added Git UDF/JAR project hosting and overall construction process
• Supports full-mode Flink jar task submission
• Added ADD CUSTOMJAR syntax to dynamically load dependencies
• Added ADD FILE syntax to dynamically load files
• openapi supports custom parameter submission
• Permission system upgrade to support tenants, roles, tokens, and menu permissions
• Support LDAP
• Added new widget function to the data development page
• Support pushing dependent tasks to DolphinScheduler
• Implement the Flink instance stopping function
• Implement CDCSOURCE synchronization of the entire database and ordering of data under multiple degrees of parallelism
• Implement configurable alarm retransmission prevention function
• Implement ordinary SQL that can be scheduled and executed by DolphinScheduler
• Added the ability to obtain dependent JARs loaded in the system and group them into groups to facilitate troubleshooting JAR related issues
• Implement cluster configuration test connection function
• Support H2, Mysql, Postgre deployment, the default is H2

New syntax

• CREATE TEMPORAL FUNCTION is used to define temporary table functions
• ADD FILE is used to dynamically load class/configuration and other files
• ADD CUSTOMJAR is used to dynamically load JAR dependencies
• PRINT TABLE for real-time preview of table data
• EXECUTE JAR is used to define Flink Jar tasks
• EXECUTE PIPELINE is used to define Flink CDC 3.x entire library synchronization tasks

Fix

• Fixed the problem of missing extends path in CLASS_PATH of auto.sh
• Fixed the problem that the job list life cycle status value was not re-rendered after release/offline
• Fixed Flink 1.18 set syntax not working and producing null error
• Fixed the save point mechanism issue of submission history
• Fixed the problem of creating views in Dinky Catalog
• Fixed Flink application not throwing exception
• Fixed incorrect rendering of alarm options
• Fixed job life cycle issues
• Fixed the problem that k8s YAML cannot be displayed in cluster configuration
• Fixed a time-consuming formatting error in the operation and maintenance center job list
• Fixed the problem of Flink dag prompt box
• Fixed checkpoint path not found
• Fixed node location error when pushing jobs to Dolphin Scheduler
• Fixed the problem that job parameters did not take effect when the set configuration contained single quotes
• Upgrade jmx_prometheus_javaagent to 0.20.0 to resolve some CVEs
• Fixed checkpoint display problem
• Repair job instance is always running
• Fixed the problem of missing log printing after Yarn Application failed to submit a task
• Fixed the problem that job configuration cannot render yarn prejob cluster
• Fixed URL misspelling causing request failure
• Fixed the problem of inserting the same token value when multiple users log in
• Fixed alarm instance form rendering issue
• Fixed the problem that FlinkSQLEnv could not be checked
• Fixed the problem that set statement could not take effect
• Fixed the problem of invalid yarn cluster configuration, customized Flink and hadoop configuration
• Fixed the problem that the checkpoint information of the operation and maintenance center cannot be obtained
• Fixed the problem that the status cannot be detected after the Yarn Application job is completed
• Fixed the problem of no printing in the console log when yarn job submission failed
• Fixed the issue where Flink instances started from cluster configuration cannot be selected in job configuration
• Fixed RECONNECT status job status recognition error
• Fixed an issue with FlinkJar tasks being submitted to PreJob mode
• Fixed Dinky startup detection pid problem
• Fixed the problem that caused conflicts when the built-in Paimon version was inconsistent with the user integrated version (implemented using shader)
• Fixed the problem that the CheckPoint parameter of the FlinkJar task does not take effect in Application mode
• Fixed the issue where the name and remark information were updated incorrectly when modifying the Task job
• Fixed the issue where password is required when registering data source
• Fixed the problem of incorrect heartbeat detection of cluster instances
• Fixed the problem that Jar task submission cannot use set syntax
• Fixed an issue where data development->job list cannot be folded in some cases
• Fixed the problem of repeated sending of alarm information under multi-threading
• Fixed the problem of tag height of data development->open job
• Fixed the problem that the jobmanager log of the operation and maintenance center job details could not be displayed normally in some cases
• Fixed Catalog NPE issues
• Fixed the problem of incorrect prejob task status
• Fixed add customjar syntax problem
• Fixed the problem that the jar task could not be monitored
• Fixed Token invalid exception
• Fixed a series of problems caused by statement delimiters and removed the system configuration
• Fixed the problem of task status rendering in the operation and maintenance center
• Fixed the problem of failure to delete tasks when the job instance does not exist
• Fixed duplicate exception alarm
• Fixed some issues submitted by PythonFlink
• Fixed the problem that Application Mode cannot use global variables
• Fixed the problem that K8s task could not start due to uninitialized resource type
• Fixed the pipeline acquisition error of the Jar task causing the front end to not work properly
• Fix SqlServer timestamp conversion to string
• Fixed NPE issue when publishing tasks with UDF
• Fixed the problem of Jar task being unable to obtain execution history
• Fixed the problem of front-end crash caused by NPE when Doris data source obtains DDL and queries

Optimization

• Added key width for job configuration items
• Optimize query job directory tree
• Optimize Flink on yarn app submission
• Optimize Explainer class to use builder pattern to build results
• Optimize document management
• Implement operator via SPI
• Optimize document form pop-up layer
• Optimize type rendering of Flink instances
• Optimize the data source details search box
• The method of obtaining the version is optimized to be returned by the backend interface
• Optimize CANCEL job logic, and can forcefully stop the lost connection job
• Optimize the detection reference logic when part of the registration center is deleted
• You can specify a job template when creating an optimization job
• Optimize Task deletion logic
• Optimize some front-end internationalization
• Optimize automatic switching between console and result tag during execution preview
• Optimize the UDF download logic of K8S
• Optimize the synchronization of the entire database and sub-databases and tables
• Optimize the registration center->data source list jump logic to the details page
• Optimization of job configuration logic (job configuration cannot be edited when the job has been released)
• Optimize the cluster instance rendering logic of job configuration in data development
• Optimize the heartbeat detection of Flink cluster
• Optimize the data source to obtain data anomalies without feedback to the front-end problem
• Optimize program shutdown strategy to graceful shutdown
• CDCSOURCE supports earliest offset and timestamp scanStartupMode
• Cancel the uniqueness restriction of the task table save point path
• Optimize CDCSOURCE light_schema_change from Mysql to Doris
• Optimize startup script classpath and add FLINK_HOME
• Optimize some front-end absolute paths to relative paths
• Change the default admin account password to a strong password

Document

• Improve the cluster instance list document of the registration center
• Improve the alarm documentation of the registration center
• Improve the Git project documentation of the registration center
• Modify domain name
• Improve the documentation of the registration center and certification center
• Improve contributor development documentation
• Add parameter description debezium.* to CDCSOURCE
• Modify the official website document structure
• Add some data development related documents
• Removed some deprecated/wrong documentation
• Add quick start document
• Add deployment documentation
• Optimize the documentation of sub-databases and tables
• Optimize general deployment documents
• Added documents related to alarm re-sending
• Optimize openapi documentation
• Add HDFS HA configuration document
• Optimize LDAP related documents
• Fixed some document word errors
• Fixed the version error issue in the integrated DolphinScheduler document

Security

• CVE-2023-2976
• CVE-2020-8908

Other

• Add some automated actions

Feature:

• Realize the synchronization of the entire database and orderly data under multiple degrees of parallelism
• Implement HDFS HA in the resource center
• Implement permission control for global configuration in the configuration center
• Implement configurable alarm anti retransmission function
• Implement DB SQL that can be scheduled by DolphinSchedule
• New resource center synchronization directory based on configured resource storage type (currently implemented as oss)

Fix:

• Fix the issue of incorrect heartbeat detection in cluster instances
• Fix delimiter issues
• Fix the issue of Jar task submission not being able to use set syntax
• Fix the issue of NPE when obtaining user related information through LDAP
• Fix the issue of being unable to carry the higher-level ID when assigning menu permissions
• Fix the issue of version history not being updated properly when switching keys in data development
• Fix some default value issues in PG SQL files
• Fix Dinky's inability to start due to resource configuration errors
• Fix the issue of default route redirection in permission control
• Fix the issue of data development ->task list section not being able to fold
• Fix the issue of duplicate sending of alarm information under multithreading
• Fix the issue of tag height in data development ->open job
• Fix authentication related issues when integrating Gitlab
• Fix the issue where jobmanager logs in the operation and maintenance center's job details cannot be displayed properly
• Fix issues with CataLog NPE
• Fix the issue of Yarn's port being 0
• Fix front-end form status issues with data sources
• Fix the issue with Kubeconfig acquisition
• Fix the issue of pre job task status errors
• Fix syntax issues with add customjar
• Fix some web NPE exceptions
• Fix a bug in enabling SSL when the alarm instance is of email type
• Fix the issue of inability to monitor jar tasks

Optimization & Improve:

• Optimize the UDF download logic of K8S
• Optimize CDC3.0 related logic
• Optimize the synchronization of the entire database by sub database and sub table
• Optimize and integrate LDAP logic
• Optimize the logic of redirecting the data source list to the details page in the registry ->Data source list
• Optimization of homework configuration logic (homework configuration cannot be edited in the published status)
• Optimize the cluster instance rendering logic for job configuration in data development
• Optimize the startup script to enable configuration of environment variables for startup

Document：

• Optimize documents for database and table partitioning
• Optimize regular deployment documents
• Add relevant documents on alarm anti resend
• Optimize OpenAPI documentation
• Add HDFS HA configuration document

Contributors

@aiwenmo @gaoyan1998 @izouxv @JiaLiangC @kylinmac @leechor @yangzehan @yqwoe @zackyoungh

New Feature

• The default Flink startup version is modified to 1.16
• Implement CodeShow component line break button
• Implement Flink instance stopping function
• Realize deletion of defined task monitoring layout

Optimization

• The method of obtaining the version is optimized to be returned by the backend interface
• Optimize the CANCEL job logic, and can forcefully stop the lost connection job
• Optimize the detection reference logic when part of the registration center is deleted
• You can specify a job template when creating an optimization job
• Optimize Task deletion logic
• Optimize some front-end internationalization
• Optimize Dinky process PID detection logic
• Optimize automatic switching between console and result tag during execution preview

Fix

• Fixed alarm instance form rendering issue
• Fixed the problem that FlinkSQLEnv could not be checked
• Fixed the problem that set statement could not take effect
• Fixed the problem of invalid yarn cluster configuration, customized Flink and hadoop configuration
• Fix some problems in Prejob mode
• Fixed the problem that the checkpoint information of the operation and maintenance center cannot be obtained
• Fixed the problem that the status cannot be detected after the Yarn Application job is completed
• Fixed the problem that the console log failed to print when yarn job submission failed.
• Fix the problem of getting savepoint list 404
• Fixed the issue where Flink instances started from the cluster configuration could not be selected in the job configuration.
• Fixed RECONNECT status job status recognition error
• Fixed the problem that the end time of the operation and maintenance center list is 1970-01-01
• Fixed the problem of submitting FlinkJar tasks to PreJob mode
• Fixed the repeated introduction of dependencies in the alarm module, causing conflicts
• Fix the problem of Dinky startup detection pid
• Fix the problem of conflict caused by inconsistent version of built-in Paimon and user integration (implemented using shader)
• Fix the syntax regular issue of execute jar
• Fixed the problem that the CheckPoint parameter does not take effect in the Application mode of the FlinkJar task
• Fixed the issue where the name and remark information were updated incorrectly when modifying Task operations
• Fixed the problem that password is required when registering data source

Document

• Add some data development related documents
• Optimize some documents of the registration center
• Remove some deprecated/wrong documentation
• Adjust some document structures
• Add quick start document
• Add deployment documents

@aiwenmo @drgnchan @gaoyan1998 @gitfortian @gitjxm @leechor @leeoo @Logout-y @MaoMiMao @Pandas886 @yangzehan @YardStrong @zackyoungh @Zzm0809

Fix: [Fix-2739] Fix bug that complete the missing path in auto.sh's CLASS_PATH [Fix-2740] Fixed issue of re-rendering task list after publishing or offline [Fix] Fix flink 1.18 set operator not work and configure null error [Fix] Fix the bug of save_point_strategy in submission history [Fix] Fix the bug of print flink table [Fix] Fix the bug of create view to ddl catalog [Fix] Fix flink application not throw exception [Fix] Fix the alert option is incorrect [Fix] Fix the bug of job life cycle [Fix-2754] Fix the YAML of K8s form in the cluster is not displayed [Fix-2756] Fix the devops job list duration formate error [Fix-2777] Fix flink dag tooltip [Fix-2782]Fix checkpoint path not found [Fix] Fix the locations bug in pushing task to DolphinScheduler [Fix-2806] The job parameters are not effective when the set parameters key and value contain single quotes [Fix-2811] Upgrade jmx_prometheus_javaagent to 0.20.0 to fix some CVE [Fix-2814] Fix checkpoint overview error [Fix] Fix Flink catalog does not take effect with add_jar [Fix] Fix some devops bug [Fix-2832] Fix h2 driver no default packaging problem [Fix] Fix sql bug [Fix] Fixed jobInstance was always in the running state [Fix-2843] Fix Yarn Application mode submission task failed and lack of log printing [Fix] Fix the bug of udf in h2 [Fix-2823] Fix jobconfig cannot render yarn prejob cluster [Fix] Fix URL misspelling causing the request to fail [Fix-2855] Fix savepoint table params bug [Fix-2776] Fix multi user login with the same token value insert error

Optimization & Improve: [Improve] Improve extract yaml from execute pipeline command [Optimization] Add key width for job configure item [Optimization] Add dinky port configure in PrintNetSink [Improve] Improve query catalog tree [Optimization-2773] Optimize the data source directory tree has two scroll bars [Optimization-2822] Optimize metrics page tips [Optimization] Optimize Flink on yarn app submit [Optimization] Optimize explainer class use user builder for result [Optimization] Optimize document management [Optimization] Implement operator with SPI [Improve] Improve document form layout [Optimization-2757] Optimize Flink instance render type [Optimization-2755] Optimize datasource detail search box [Optimization] Add resource implement for DinkyClassLoader

Document： [Document] Improve the cluster instance list document for the registration center [Document] Improve the alert document for the registration center [Document] Improve the git project document for the registration center [Document] Improve the k8s document for the quick start [Document] Modify domain name [Document] Improve documents in registration center and authentication center [Document] Improve documents in developer guide [Document] Add parameter description in CDCSOURCE and example for debezium.* [Document-2830] Update download [Document] Modify document struct

Contributors: @aiwenmo @gaoyan1998 @gitfortian @leeoo @leechor @stdnt-xiao @yangzehan @zackyoungh @Zzm0809

Introduction

Dinky is a data development platform based on Apache Flink, which enables agile data development and deployment.

Upgrade instructions

Dinky 1.0 is a refactored version that restructures existing functions, adds several enterprise-level functions, and fixes some limitations of 0.7. Currently, it is not possible to directly upgrade from 0.7 to 1.0. An upgrade plan will be provided in the future.

Function

Its main functions are as follows:

• FlinkSQL data development: automatic prompt completion, syntax highlighting, statement beautification, syntax verification, execution plan, MetaStore, lineage analysis, version comparison, etc.
• Support FlinkSQL multi-version development and multiple execution modes: Local, Standalone, Yarn/Kubernetes Session, Yarn Per-Job, Yarn/Kubernetes Application
• Support Apache Flink ecosystem: Connector, FlinkCDC, Paimon, etc.
• Support FlinkSQL syntax enhancement: whole database synchronization, execution environment, global variables, statement merging, table value aggregation function, loading dependencies, row-level permissions, Jar submission, etc.
• Support FlinkCDC real-time warehousing of the entire database into the lake: multi-database output, automatic table creation, model evolution, sub-database and sub-table
• Supports SQL job development and metadata browsing: ClickHouse, Doris, Hive, Mysql, Oracle, Phoenix, PostgreSql, Presto, SqlServer, StarRocks, etc.
• Support Flink real-time online debugging preview TableData, ChangeLog, Operator, Catalog
• Support Flink job custom monitoring statistical analysis and custom alarm rules.
• Support real-time task operation and maintenance: online and offline, job information (supports obtaining checkpoint), job log, version information, job snapshot, monitoring, SQL lineage, alarm record, etc.
• Support real-time job alarms and alarm groups: DingTalk, WeChat business account, Feishu, email, SMS, etc.
• Supports automatically hosted SavePoint/CheckPoint recovery and triggering mechanisms: latest, earliest, specified, etc.
• Supports multiple resource management: cluster instances, cluster configurations, data sources, alarms, documents, global variables, Git projects, UDFs, system configurations, etc.
• Support enterprise-level management: tenants, users, roles, menus, tokens, data permissions

New Feature

• Added new homepage signboard
• Data development supports code tips
• Supports real-time printing of Flink table data
• The console supports real-time printing task submission log
• Support Flink CDC 3.0 entire database synchronization
• Support customized alarm rules and customized alarm information templates
• Comprehensive revision of the operation and maintenance center
• k8s and k8s operator support
• Support proxy Flink webui access
• Support Flink task monitoring
• Support Dinky jvm monitoring
• New resource center function and expanded rs protocol
• New Git UDF/JAR project hosting and overall construction process
• Supports full-mode application mode custom jar submission
• openapi supports custom parameter submission
• Permission system upgrade, supporting tenants, roles, tokens, and menu permissions
• LDAP authentication support
• New widget function on data development page
• Support pushing dependent tasks to DolphinScheduler

Feature: [Feature-2370][connect] Add jdbc scan filter in flink-connector-jdbc-1.16 [Feature-2381][udf] AGGTABLE supports udtaf in Flink 1.16

Fix: [Fix][udf] Fix UDF's submission to submission with Checkpoint failed [Fix-2392][metadata] Fix listColumnsSortByPK infinite recursion

Optimization & Improve: [Optimization-2361][web] Optimize the display of job which unable to obtain the latest status [Optimization][core] Optimize flink task query use enum value

Contributors: @aiwenmo @Wintle @zackyoungh

Feature: [Feature-1930][client] Analyze custom functions from Flink SQL [Feature-2126][connector] Add SQLServer Simple Connector in Flink 1.16 [Feature][cdc] Doris auto create table in CDCSOURCE task

Fix: [Fix][metadata] Fix the issue of no data type when MySQL automatically generates DDL statements [Fix][cdc] Fix issues with CDCSOURCE [Fix-1958][cdc] Fix data misalignment occurs after synchronizing data hen PK do not declare at the table head [Fix-2087][UDF] Fix udf cannot run on flink1.16 and flink 1.17 [Fix][metadata] Fix clickhouse Nullable columns [Fix-2147][metadata] Fix missing precision and scale when generating MySql DDL [Fix][metadata] Fix mssql tinyint Type conversion error [Fix-2176][admin] Fix unknown jobid bug [Fix-2108][admin] Fix updating the savepoint policy of the job during checkpoint recovery

Optimization & Improve: [Optimization][cdc] Optimize CDCSOURCE in Flink 1.16 [Optimization-2089][flink] Upgrade Flink version

Contributors: @aiwenmo @HamaWhiteGG @leeoo @LiuHao0606 @seawenc @tan-zhuo @xiaofan2022 @zhengwenhao2017

Feature: [Feature-1709][core] Support use FlinkSQL CTE to preview data [Feature-1774][flink] Support Apache Flink 1.17

Fix: [Fix-797][web] Fix no fragment parameters were passed when getting lineage [Fix-1731][gateway] Fix deploy flink k8s session error [Fix-1787][metadata] Fix error in automatic table creation using CDCSOURCE Mysql field unsigned type problem [Fix-1805][admin] Fix NPE occurred when publishing the job [Fix][cdcsource] Fix data sending and receiving issues with synchronizing multiple Sinks in CDCSOURCE [Fix][SqlSegment] Fix the issue of SQL configuration parsing failure in specific scenarios [Fix-1784][admin] Fix task is deleted but task instances and history are not deleted consecutively [Fix-1849][client] Fix CDCSOURCE kafkaSink transactionalIdPrefix NullPointerException [Fix-1847][metadata] Fix postgreSql get DDL script error [Fix][metadata] Fix generating MySQL DDL statements without quotes for normal default values

Optimization & Improve: [Optimization-1715][admin] Optimize job monitor reconnect [Optimization-1776][core,metadata] Fix typo [Optimization-1657][client] Optimize MysqlJsonDebeziumDeserializationSchema use shaded lib in flink 1.13 [Optimization][client] Support scan.incremental.snapshot.chunk.size [Optimization][client] Analyze lineage from the original relNode, not the optimized relNode, and add unit tests [Optimization-1851][flink] Upgrade all versions of Flink to the latest version

Contributors: @aiwenmo @AntChen27 @cygnusdark @HamaWhiteGG @LiuHao0606

Feature: [Feature-1593] Add CDCSOURCE support sink catalog [Feature-1608] Support get task lineage by taskid with openapi [Feature-1641] Add Flink row level permissions [Feature-1647] Yarn session support applicationId and resource manager config [Feature-1547] Flink Jar support alert

Fix: [Fix-1615] Fix submit task error but return succeed when using dolphinscheduler [Fix-1624] Fix lineage api in data explore module returns empty result [Fix-1652] Fix the task using the FlinkSQLEnv cannot obtain lineage [Fix-1662] Fix OpenApi notLoginException [Fix] Fix wrong Python type [Fix-1672] Fix failed to get metadata due to jdbc session timeout in CDCSOURCE [Fix-1638] Fix dlink web could not correct display Boolean type data [Fix-1686] Fix ZipFile got wrong in K8s native app mode

Optimization & Improve: [Optimization-1631] Optimize auto.sh and remove useless files [Optimization-1666] Add job reconnecting status and optimize job monitor [Optimization-1682] Optimize setting named tinyInt1isBit for mysql metadata

Contributors: @aiwenmo @liaotian1005