https://github.com/apache/flink

sort by:
Revision Author Date Message Commit Date
fd35c48 Commit for release 1.12.5 15 July 2021, 02:07:10 UTC
8f0e13c [FLINK-22547][tests] Harden OperatorCoordinatorHolderTest. Ensure that the 'FutureCompletedAfterSendingEventsCoordinator' cannot exit before it has completed the triggered checkpoint (completed the checkpoint future). 14 July 2021, 14:16:01 UTC
adfd85f [FLINK-23312][ci] speed up compilation for e2e tests The "compile" builder already applies all checks so we can use -Dfast here; also, the web UI is not actually needed in the E2E tests. 14 July 2021, 08:07:57 UTC
331ae2c [FLINK-22545][coordination] Fix check during creation of Source Coordinator thread. The check was meant as a safeguard to prevent re-instantiation after fatal errors killed a previous thread. But the check was susceptible to thread termination due to idleness in the executor. This updates the check to only fail if there is in fact an instantiation next to a running thread, or after a previously crashed thread. 13 July 2021, 22:31:43 UTC
2c3f8f6 [FLINK-23233][runtime] Ensure checkpoints confirmed after all the failed events processed for OepratorCoordinator This closes #16432. 13 July 2021, 12:30:16 UTC
6292777 [FLINK-23223] Notifies if there are available data on resumption for pipelined subpartition 12 July 2021, 09:55:20 UTC
86cc6a1 [FLINK-23184][table-runtime-blink] Fix compile error in code generation of unary plus and minus This closes #16435 12 July 2021, 08:25:09 UTC
da53f93 [FLINK-23076][tests] Harden DispatcherTest.testWaitingForJobMasterLeadership The test DispatcherTest.testWaitingForJobMasterLeadership is unstable because it uses a 10 ms timeout to decide whether the JobMasterService has been created or not. On our CI infrastructure this 10 ms can be too short to complete the future with JobStatus.INITIALIZING which indicates that the JobMasterService has not been created. This commit hardens the test by increasing the timeout to 200 ms and to increase the number of retries from 5 to 50. This closes #16444. 12 July 2021, 07:56:27 UTC
e599aa2 [FLINK-23164][tests] Harden JobMasterTest.testMultipleStartsWork The JobMasterTest.testMultipleStartsWork fails because we don't wait for the JobMaster to finish its suspension before restarting the JobMaster. This can lead to ignoring the underlying control START control message. This again can lead to a situation where we are in the STOPPED state and try to send UnfencedMessage(startJobExecution) in JobMaster.start. This commit fixes this problem by waiting on the suspension future to complete before restarting the JobMaster. This closes #16442. 12 July 2021, 07:53:25 UTC
2eaabca [FLINK-20321][formats] Fix NPE when using Avro/Json/Csv formats to deserialize null input (#16394) Co-authored-by: Xue Wang <xzwxpop@hotmail.com> 12 July 2021, 03:47:41 UTC
e280421 [FLINK-22819][yarn] Remove 'yarn.am.liveness-monitor.expiry-interval-ms' override for tests. This is an interval between AM container allocation and actually running RM Client inside this container, which can take longer in resource limited environment such as CI. If heartbeats between AM and RM don't work and a test relies on this, then it will fail a bit later because the default value is 5 minutes. This closes #16413. 09 July 2021, 14:04:20 UTC
09ac6e5 [FLINK-23074][connector-hive] Shade parquet class in hive-exec to prevent conflict with flink-parquet module This closes #16424 09 July 2021, 04:17:17 UTC
4812b42 [FLINK-22443][streaming-java] Fix overflow in MultipleInputSelectionHandler 08 July 2021, 06:33:49 UTC
bb29f5e [FLINK-21445][clients] Adds configuration to ClassPathPackagedProgramRetriever 07 July 2021, 18:11:08 UTC
4ef1427 [hotfix][tests] Close SubtaskCheckpointCoordinator in tests This is to reveal any async failures occurring for example when writing channel state. 06 July 2021, 20:45:16 UTC
ae24c97 [FLINK-22105][tests] Fix checkpoint id in testForceAlignedCheckpointResultingInPriorityEvents 06 July 2021, 20:45:16 UTC
4d017d3 [FLINK-23182][connectors/rabbitmq] Fix connection leak in RMQSource 06 July 2021, 16:45:36 UTC
701d20e [FLINK-23248][datastream] Close SinkWriter when calling dispose Without this change the SinkWriter was only closed when the operator finishes and not during failing. Now, the SinkWriter is closed on dispose which is called on finish and on failure. 06 July 2021, 08:15:20 UTC
5186060 [FLINK-23052][ci] Improve stability of maven snapshot deployment This closes #16331 30 June 2021, 09:30:55 UTC
83a26d5 [FLINK-20888][runtime] Close outputs in OperatorChain. OperatorChain creates the outputs and owns them, so that it should also close them. Specific operators should not close the outputs. Also, ChainingOutput should never close the chained operator; it's not owning the operator. 30 June 2021, 07:49:19 UTC
2a8cc56 [hotfix][datastream] Remove raw casts in ContinuousFileReaderOperator. 30 June 2021, 07:49:19 UTC
ca999b2 [FLINK-22964][connector/common] Exclude flink-core from connector-common dependencies. Connectors get shaded into the user jar and as such should contain no unnecessary dependencies to flink. However, connector-base is exposing `flink-core` which then by default gets shaded into the user jar. Except for 6MB of extra size, the dependency also causes class loading issues, when `classloader.parent-first-patterns` does not include `o.a.f`. 30 June 2021, 07:44:01 UTC
ab04b11 [FLINK-18182][kinesis] Updating to latest AWS SDK for Kinesis connector 29 June 2021, 07:25:55 UTC
d1b8c5f [FLINK-22157][table-planner-blink] Fix join & select a portion of composite primary key will cause ArrayIndexOutOfBoundsException This closes #16305 (cherry picked from commit 92fbe7f1fe5f0eade036b4184cdbab8f9b791647) 29 June 2021, 02:37:29 UTC
169c967 [FLINK-23166][python] Fix ZipUtils to handle properly for softlinks This closes #16309. 28 June 2021, 23:15:59 UTC
b577808 [FLINK-23009][kinesis] Include guava:failureaccess This wasn't included in the flink-sql-connector-kinesis jar and this caused an e2e test failure in another release branch. The other release branch is fixed in PR 16297. Applying the same changes here. 28 June 2021, 10:20:19 UTC
91d9004 [FLINK-23133][python] Properly handle the dependencies when mixing use of Python Table API and Python DataStream API This closes #16272. 27 June 2021, 07:07:28 UTC
d77c51d [FLINK-23120][python] Fix ByteArrayWrapperSerializer.serialize to use writeInt to serialize the length This closes #16258. 27 June 2021, 02:14:29 UTC
f3c22a8 [hotfix][tests] Correct variable name in RunnablesTest.testExecutorService_uncaughtExceptionHandler 25 June 2021, 15:48:02 UTC
872a45b [FLINK-23045][tests] Harden RunnablesTest by not relying on timeout This commit hardens the RunnablesTest.testExecutorService_uncaughtExceptionHandler to not rely on timeouts to check whether the uncaught exception handler was called. This closes #16263. 25 June 2021, 15:47:45 UTC
8861251 [hotfix] Let RunnablesTest extend TestLogger 25 June 2021, 15:47:39 UTC
3a7552a [FLINK-23010][hive] HivePartitionFetcherContextBase shouldn't list folders to discover new partitions This closes #16182 25 June 2021, 06:55:27 UTC
1619bbb [FLINK-23119][python] Throw exceptions when compiling the job at places where Python UDAF is not supported This is to give a more helpful message if Python UDAF is used in places where it's not supported. This closes #16270. 24 June 2021, 12:13:35 UTC
a21baa4 [FLINK-21229][docs] Update avro-confluent docs This closes #15808. 24 June 2021, 06:57:14 UTC
5bafade [FLINK-21229][avro-confluent-registry] Add Confluent schema registry SSL support 24 June 2021, 06:57:14 UTC
cdec22e [FLINK-22927][python] Fix the bug of JobStatus This closes #16146. 24 June 2021, 03:01:14 UTC
20da2a1 [FLINK-22886][state] Fix thread leak and add unit test in RocksIncrementalSnapshotStrategy 22 June 2021, 18:24:58 UTC
3d93d67 [FLINK-22788][table-planner-blink] Support equalisers for many fields When working with hundreds of fields, equalisers can fail to compile because the method body grows beyond 64kb. With this change, instead of generating all code into one method, we generate a dedicated method per field and then call all of those methods. This doesn't entirely remove the problem, but supports roughly a factor of 10 more fields and is currently deemed sufficient. This closes #16213. 22 June 2021, 17:15:43 UTC
4b4b614 [backport-1.12][FLINK-22698][connectors/rabbitmq] Add deliveryTimeout to RabbitMQ source This change enables setting the message delivery timeout in the RabbitMQ queueing consumer, allowing to properly stop the job in cases when no new message is available on the queue. Changes: - Add the ability to setDeliveryTimeout on the RMQConnectionConfig and its builder - Change default message delivery timeout in the RMQSource to 30 seconds (previously there was no timeout) - Extend RabbitMQ source unit tests 22 June 2021, 11:32:58 UTC
1e2a1ec [FLINK-22312][yarn][test] Fix test instabilities due to expected heartbeat exceptions in log This closes #16232 22 June 2021, 09:59:53 UTC
38bf63b [FLINK-23030][network] PartitionRequestClientFactory#createPartitionRequestClient should throw when network failure 21 June 2021, 15:11:32 UTC
710b5f6 [FLINK-21952] Make all the "Connection reset by peer" exception wrapped as RemoteTransportException 21 June 2021, 07:26:58 UTC
2b26df9 [FLINK-22987][ci] Fix scala suffix check * [FLINK-22987][ci] Fix scala suffix check * temporarily disable avro registry * Add basic precondition that at least 1 clean&infected module must be found * Invert exit handling 21 June 2021, 07:06:59 UTC
68961a7 [FLINK-23009][kinesis] Upgrade Guava for flink connector kinesis Also make flink-sql-connector-kinesis use the Guava library coming transitively from the connector-kinesis dependency. 18 June 2021, 14:10:36 UTC
949db6f [FLINK-22946][runtime] Recycle floating buffer outside the lock to avoid deadlock 18 June 2021, 12:52:13 UTC
654e2a6 [FLINK-22890][hive] HiveTestUtils should create partition after the data file is ready This closes #16099 16 June 2021, 06:40:16 UTC
7b4d6b8 [FLINK-22963][doc] correct the description of taskmanager.memory.task.heap.size. This closes #16152 15 June 2021, 07:25:29 UTC
faf7cc4 [FLINK-22015][table-planner-blink] Exclude IS NULL from SEARCH operators (#16140) 11 June 2021, 10:19:37 UTC
11c2030 [FLINK-22908][tests] Wait until job has started before testing shutdown 11 June 2021, 10:07:35 UTC
e9e8d1c [FLINK-22419][coordination][tests] Rework RpcEndpoint delay tests This closes #16136 11 June 2021, 07:25:39 UTC
aa3cd4b [FLINK-22952][azure] Use Ruby 2.5 10 June 2021, 14:48:10 UTC
6e86ccb Adding gcs documentation. Connecting flink to gcs. 09 June 2021, 18:05:23 UTC
e794574 [FLINK-22833][metrics] Set checkpointStartDelay in checkpoint metrics for SourceTasks This commit is fixing a bug of checkpointStartDelay being not set in CheckpointMetrics of SourceStreamTask and SourceOperatorStreamTask. 09 June 2021, 13:33:30 UTC
30637ef [hotfix][test] Deduplicate test case between SourceStreamTask and SourceOperatorStreamTask 09 June 2021, 13:32:08 UTC
2c0e3f6 [hotfix][task] Rename triggerCheckpoint to better reflect a difference compared to other similarly named methods 09 June 2021, 13:22:39 UTC
277965b [FLINK-22939][azure] Generalize JDK switch 09 June 2021, 09:26:00 UTC
82431f1 [hotfix] Replace deprecated import org.junit.Assert.assertThat with org.hamcrest.MatcherAssert.assertThat 08 June 2021, 15:57:29 UTC
4c05456 [FLINK-22496][tests] Include log output from started process in ClusterEntrypointTest.testCloseAsyncShouldBeExecutedInShutdownHook This closes #16111. 08 June 2021, 15:57:29 UTC
b7b324d [FLINK-22496][tests] Harden ClusterEntrypointTest.testCloseAsyncShouldBeExecutedInShutdownHook Hardens the ClusterEntrypointTest.testCloseAsyncShouldBeExecutedInShutdownHook by increasing the timeout from 3s to 10s. 08 June 2021, 15:57:28 UTC
9831b46 [FLINK-22856][Azure] Upgrade to ubuntu-20.04 08 June 2021, 10:00:06 UTC
1575e53 [FLINK-22815][checkpointing] Remove SubtaskStateMapper.DISCARD_EXTRA_STATE which does not work The mapping implemented via DISCARD_EXTRA_STATE is not supported on the network level. At the same time there is no use for that mapping. It's better to remove the mapper for now, so that it is not used by mistake. 08 June 2021, 06:30:40 UTC
e6ab966 [FLINK-21945][streaming] Add the possibility to force certain exchanges to be aligned. Introduces the AlignmentType to CheckpointOptions that includes a FORCED_ALIGNED. This barrier acts as an aligned barrier for certain exchanges. The flag is then later used to restore the unaligned characteristic in SubtaskCheckpointCoordinatorImpl for downstream exchanges. 08 June 2021, 06:30:40 UTC
bdf4399 [FLINK-22815][checkpointing] Disable unaligned checkpoints for broadcast partitioning Broadcast partitioning can not work with unaligned checkpointing. There is no guarantees that records are consumed at the same rate in all channels. This can result in some tasks applying state changes corresponding to a certain broadcasted event while others don't. In turn upon restore it may lead to an unconsistent state. 08 June 2021, 06:30:40 UTC
68f1812 [FLINK-22683][runtime] Fix the null or incorrect value of total Flink/process memory in creating TaskExecutorMemoryConfiguration This closes #16067 04 June 2021, 06:20:05 UTC
08bf3e6 [hotfix][coordination] Adjust TaskExecutor resource to reasonably large value for local execution This fix is to avoid arithmetic overflow in calculating the resource overview of the minicluster. 04 June 2021, 06:20:04 UTC
31f4099 [FLINK-22820] Allow storing SUSPENDED jobs in FileArchivedExecutionGraphStore 03 June 2021, 07:29:22 UTC
9f1de31 [FLINK-22756] Ignore leadership future termination during testing This changes hardens the DispatcherTest#testJobStatusIsShownDuringTermination by ignoring the JobMaster shutdown exception when querying for the expected job status. 02 June 2021, 14:00:51 UTC
aa590c8 [FLINK-22814][metrics] Calculate checkpointStartDelay for FLIP-27 sources 01 June 2021, 12:34:20 UTC
d215329 [FLINK-22774][sql-connector-kinesis] Update Kinesis SQL connector's Guava to 27.0-jre Avoids that security tools complain about an outdated Guava version in the Kinesis SQL connector. This closes #16005. 26 May 2021, 20:31:56 UTC
b0b7dfc [FLINK-22613][tests] Fix FlinkKinesisITCase.testStopWithSavepoint. The test relies on a few elements remaining pending after stop-with-savepoint, however that can only be guaranteed heuristically: We cannot block task thread or else the respective savepoint will not succeed but we also cannot add infinite input as it's an IT test against Kinesis. Here, the fix is to add much more elements to Kinesis stream. An optimized sendMessage on the test client ensures timely setup. 26 May 2021, 14:50:41 UTC
cc885c4 [FLINK-22747] Upgrade to commons-io 2.8.0 This closes #15989. 25 May 2021, 16:44:28 UTC
a58b516 Update japicmp configuration for 1.12.4 21 May 2021, 14:51:02 UTC
5052dc3 [FLINK-22434] Retry in clusterclient if job state is unknown Since the dispatcher now exposed the SUSPENDED job state it needs to be handled in the ClusterClient. We decided to not expose this state because it would be an unexpected change for users relying on the old behavior. In summary, the RestClusterClient will retry to receive a different state for jobs in SUSPENDED state. This closes #15973. 21 May 2021, 12:49:01 UTC
083fe40 [FLINK-22434] Store suspended execution graphs on termination to keep them accessible Before this change as soon as a cluster termination was initiated every job was suspended and not accessible anymore until recovery. By storing the execution graph they will appear as SUSPENDED they are now accessible until the cluster is fully shutdown. 21 May 2021, 12:49:00 UTC
c35df7c [FLINK-22708][config] Propagate savepoint settings from StreamExecutionEnvironment to StreamGraph 21 May 2021, 09:02:51 UTC
1e5d808 [FLINK-22721][ha] Add default implementation for HighAvailabilityServices.cleanupJobData In order to not break the HighAvailabilityServices interface, this commit adds a default implementation for the HighAavilabilityServices.cleanupJobData method which does nothing. Users of this interface are recommended to override this method and add a specific implementation in order to clean up job specific HA services after a job completion. This closes #15971. 20 May 2021, 15:53:14 UTC
7b9351e [FLINK-22704][tests] Harden ZooKeeperHaServicesTest.testCleanupJobData This commit hardens the ZooKeeperHaServicesTest.testCleanupJobData by making sure that all zNodes have been created before checking for their existence. This is done by waiting for the leader information for the ResourceManager and the JobMaster. This closes #15956. 20 May 2021, 08:17:47 UTC
e09c919 [FLINK-21329][tests] Increase timeout and delay in local test_local_recovery_and_scheduling.sh Back-pressure is likely to occur without injection delay at the source which may lead to delaying checkpoint triggering at the sources which may lead to a timeout. To prevent this, test timeout is increased from 10m to 15m and injection delay of 100ms is added. 19 May 2021, 12:27:58 UTC
7e6fb13 [hotfix][runtime] Log checkpoint processing delay if above threshold 19 May 2021, 12:27:58 UTC
a153976 [FLINK-22502][checkpointing] Don't tolerate checkpoint retrieval failures on recovery Ignoring such failures and running with an incomplete set of checkpoints can lead to consistency violation. Instead, transient failures should be mitigated by automatic job restart. 18 May 2021, 20:16:40 UTC
ee34945 [hotfix][tests] Move test checkpoint component classes to upper level 18 May 2021, 20:16:40 UTC
6b53f8b [FLINK-22494][runtime] Adds PossibleInconsistentStateException handling to CheckpointCoordinator 18 May 2021, 09:07:27 UTC
5aec1bb [FLINK-22494][runtime] Refactors CheckpointsCleaner to handle also discardOnFailedStoring 18 May 2021, 09:07:27 UTC
b81887e [FLINK-22494][ha] Introduces PossibleInconsistentState to StateHandleStore 18 May 2021, 09:07:27 UTC
95bd043 [FLINK-22494][ha] Refactors TestingLongStateHandleHelper to operate on references The previous implementation stored the state in the StateHandle. This causes problems when deserializing the state creating a new instance that does not point to the actual state but is a copy of this state. This refactoring introduces LongStateHandle handling the actual state and LongRetrievableStateHandle referencing this handle. 18 May 2021, 09:07:27 UTC
a53a1f3 [FLINK-22494][kubernetes] Introduces PossibleInconsistentStateException We experienced cases where the ConfigMap was updated but the corresponding HTTP request failed due to connectivity issues. PossibleInconsistentStateException is used to reflect cases where it's not clear whether the data was actually written or not. 18 May 2021, 09:07:26 UTC
814fc65 [FLINK-20695][ha] Clean ha data for job if globally terminated At the moment Flink only cleans up the ha data (e.g. K8s ConfigMaps, or Zookeeper nodes) while shutting down the cluster. This is not enough for a long running session cluster to which you submit multiple jobs. In this commit, we clean up the data for the particular job if it reaches a globally terminal state. This closes #15910. 18 May 2021, 07:20:23 UTC
60093e4 [FLINK-22592][runtime] numBuffersInLocal is always zero when using unaligned checkpoints This closes #15915 17 May 2021, 03:16:19 UTC
9541b4c [FLINK-21469][runtime] Implement advanceToEndOfEventTime for MultipleInputStreamTask For stop with savepoint, StreamTask#advanceToEndOfEventTime() is called (in source tasks) to advance to the max watermark. This PR implments advanceToEndOfEventTime for MultipleInputStreamTask chained sources. 12 May 2021, 12:20:04 UTC
531158f [FLINK-17857][test] Make K8s e2e tests could run on Mac This closes #15890. 11 May 2021, 12:11:20 UTC
8c7a1c8 [hotfix][e2e] Output and collect the logs for Kubernetes IT cases 11 May 2021, 12:11:18 UTC
9e5b0d4 [FLINK-22408][sql-parser] Fix SqlDropPartitions unparse Error This closes #15788 11 May 2021, 10:06:34 UTC
2b1d475 [FLINK-22566][test] Adds log extraction for the worker nodes We struggled to get the logs of the node manager which made it hard to investigate FLINK-22566 where there was a lag between setting up the YARN containers and starting the TaskExecutor. Hopefully, the nodemanager logs located on the worker nodes will help next time to investigate something like that. 11 May 2021, 08:23:41 UTC
cd425a2 [hotfix][test] Adds -e flag to interpret newline in the right way 11 May 2021, 08:23:41 UTC
ca9d2cb Revert "[FLINK-22566][e2e] Disable Running Kerberized YARN application on Docker test (default input) fails with no resources" This reverts commit 59f764ba504df53e5bd6226c6d8e668f1f964ba3. 11 May 2021, 08:23:41 UTC
4fd1344 [FLINK-21181][runtime] Wait for Invokable cancellation before releasing network resources 10 May 2021, 12:14:10 UTC
daaa01d [FLINK-22597] Make JobMaster restartable The problem is that the DefaultLeaderRetrievalService for the ResourceManager is reused by the JobMaster. Since the DefaultLeaderRetrievalService is only usable once, this means that the JobMaster can only be started once even though the JobManagerRunnerImpl restarts it in case of a regained leadership. This closes #15854. 10 May 2021, 10:17:55 UTC
ed9965c [FLINK-22577][tests] Harden KubernetesLeaderElectionAndRetrievalITCase This commit introduces closing logic to the TestingLeaderElectionEventHandler which would otherwise forward calls after the KubernetesLeaderElectionDriver is closed. This closes #15857. 10 May 2021, 10:08:38 UTC
87555d8 [FLINK-22555][build][python] Exclude leftover jboss files 09 May 2021, 18:23:27 UTC
b377569 [FLINK-22560][build] Add dedicated name to flink-dist shade-plugin execution 09 May 2021, 18:23:27 UTC
back to top