https://github.com/apache/flink

sort by:
Revision Author Date Message Commit Date
460b386 Commit for release 1.14.0 22 September 2021, 06:39:40 UTC
a595fbb [FLINK-24347][connectors/kafka] Keep idle source readers if parallelism is higher than partitions in KafkaSource Before this commit the enumerator signalled the leftover source readers without a partition to finish. This caused that checkpointing was not possible anymore because it is only supported if all tasks are running or FLIP-147 is enabled. This closes #17327 21 September 2021, 21:51:36 UTC
40b0a31 [FLINK-24340] Only print exception on job failure/suspension 21 September 2021, 13:35:40 UTC
36c1b5b [FLINK-24303][coordination] Failure when creating a source enumerator lead to full failover, not JobManager failure. Instead of letting exceptions during the creation of the Source Enumerator bubble up (and utimately fail the JobManager / Scheduler creation), we now catch those exceptions and trigger a full (global) failover for that case. 21 September 2021, 08:30:14 UTC
30d7200 [release] Create 1.14 release-notes This closes #17182 21 September 2021, 07:35:31 UTC
69b6e60 [FLINK-24197] Guard against CLRF being split across chunks This commit adds a defense mechanism against https://github.com/netty/netty/issues/11668. If the CRLF prefix of a multipart delimiter is split across 2 chunks an exception is thrown because Netty incorrectly treats CR as data. If CR is the last byte of the current chunk, then we pessimistically assume that this case occurred. We exclude the trailing CR from the current chunk, and add it to the front of the next received chunk. 20 September 2021, 08:38:12 UTC
6f2fa53 [hotfix][test] Derive expected files from getFilesToUpload This simplifies debugging if the number of files is modified. 20 September 2021, 08:38:02 UTC
026b0cb [hotfix][docs] fix link from upsert-kafka to kafka Table(!) connector 20 September 2021, 08:33:27 UTC
cc19997 [FLINK-24277][connector/kafka] Add OffsetsInitializerValidator interface for validating offset initializer in KafkaSourceBuilder 18 September 2021, 13:44:00 UTC
984cfe9 [FLINK-24277][connector/kafka] Remove auto-generated group id in Kafka table source 18 September 2021, 13:44:00 UTC
677caa8 [FLINK-24277][connector/kafka] Add configuration for committing offset on checkpoint and disable it if group ID is not specified 18 September 2021, 13:44:00 UTC
791c1b9 [FLINK-24300] SourceOperator#getAvailableFuture reuses future Callers of SourceOperator#getAvailableFuture might call the method multiple times even if the returned future does not complete. Before the commit each we were creating a new combined future from the SourceReader#isAvailable and the forcedStop one. If the underlying SourceReader#isAvailable has not changed this operation is unnecessary. What is even worse each such operation adds another entry onto the source reader's availability future stack which caused performance regression. The commit reuses the combined future if the underlying SourceReader#isAvailable future has not changed. This closes #17303 17 September 2021, 19:19:42 UTC
63fa7d0 [FLINK-24233][runtime] Ignore message about new buffer size if the reader doesn't ready yet 17 September 2021, 18:16:54 UTC
64902ea [FLINK-24129][connectors-pulsar] Harden TopicRangeTest.rangeCreationHaveALimitedScope. 17 September 2021, 17:56:29 UTC
a2b612a [FLINK-23969][connector/pulsar] Create e2e tests for pulsar connector. (cherry picked from commit b6086203d5fc0a08a330dd0069fbe1359ceac97a) 17 September 2021, 14:32:29 UTC
d818ccb [hotfix][connectors/kafka] Rename EventDeSerializer to EventDeSerializationSchema in examples 17 September 2021, 13:30:56 UTC
06d4828 [FLINK-24292][connectors/kafka] Use KafkaSink in examples instead of FlinkKafkaProducer 17 September 2021, 13:30:56 UTC
5dd99ed [FLINK-24281][connectors/kafka] Migrate all format tests from FlinkKafkaProducer to KafkaSink 17 September 2021, 13:19:38 UTC
83fd46f [FLINK-24281][connectors/kafka] Only allow KafkaSinkBuilder creation with KafkaSink.builder() 17 September 2021, 13:19:38 UTC
587e677 [hotfix][connectors/kafka] Remove unused code from KafkaDynamicSink 17 September 2021, 13:19:38 UTC
c05dd39 [FLINK-24317][python][tests] Optimize the implementation of Top2 in test_flat_aggregate This closes #17309. 17 September 2021, 11:13:32 UTC
9c4262e [hotfix] Update frontend-maven-plugin This backports part of a commit to update the frontend-maven-plugin. Doing so prevents Angular's compatibility compiler output to be logged on ERROR level. 17 September 2021, 09:58:36 UTC
1ad53cf [hotfix] Fix a typo in 'estimatedTimeToConsume(r)BuffersMs' 16 September 2021, 12:20:25 UTC
4cb7624 [FLINK-24133][core] Network failure test replaced by comment in the code due to high expense of stabilizing such low important test 16 September 2021, 08:12:02 UTC
4ebd057 [FLINK-24305][python] Limit the protobuf version<3.18 This closes #17298. 16 September 2021, 07:40:36 UTC
8d148a8 [FLINK-24059][Connectors/Common][test] Allow SourceReaderTestBase.NUM_SPLITS to be overridden (#17064) 15 September 2021, 12:40:50 UTC
86f96cd [hotfix][docs] Add glossary entry for 'Table program' 15 September 2021, 07:49:42 UTC
02040c9 [FLINK-21589][docs] Document table pipeline upgrades This closes #17260 15 September 2021, 07:49:34 UTC
5f2ec45 [FLINK-24283][connector/pulsar] Use stick key consumer in Key_Shared subscription. This would make sure Pulsar won't treat the flink reader as a shared consumer. This fix https://github.com/apache/pulsar/pull/12035 14 September 2021, 19:52:55 UTC
8c1033d [FLINK-24282][connectors/kafka] Make topic selector for KafkaSink serializable It is possible to calculate the target topic per record. Therefore users can provide a lambda when constructing the KafkaSink. Before this commit the lambda was not marked as serializable and could not be transferred to the workers. 14 September 2021, 19:52:22 UTC
d008496 [FLINK-23607][state/changelog] Document checkpointing.changelog in pom.xml 14 September 2021, 13:28:31 UTC
2169b79 [FLINK-23607][state/changelog] Cleanup dependencies of DSTL DFS 14 September 2021, 13:28:31 UTC
e7ba160 [FLINK-23607][state/changelog] Make Changelog Backend a transitive dependency ...of flink-test-utils instead of direct dependency of each module. 14 September 2021, 13:28:31 UTC
918fde8 [hotfix][docs] Fix output of test job Previously, the output listed here was the one for `head` and not `tail` as it's described in the steps to execute the test. This closes #17261. 14 September 2021, 07:36:28 UTC
753947a [FLINK-24266][checkpoint] Log improvement for aborting checkpoint due to tasks are finishing This closes #17250. 14 September 2021, 07:03:51 UTC
83ab9eb [hotfix][connector] Use null to replace empty pending transaction in 2pc sink This closes #17251. 14 September 2021, 07:03:51 UTC
574e80f [FLINK-24276][python] Avoid confusing output when executing in loopback mode This closes #17274. 14 September 2021, 06:38:51 UTC
beb7009 [FLINK-24267][python][docs] Update the example tutorial to use latest API This closes #17273. 14 September 2021, 06:36:44 UTC
b26f7e7 [FLINK-23773][connector/kafka] Mark empty splits as finished to cleanup states in SplitFetcher 14 September 2021, 05:34:48 UTC
52a26c8 [hotfix][connector/common] Expose splitFinishedHook in SplitFetcher for fine-grained testing 14 September 2021, 05:34:48 UTC
da82cb1 [FLINK-24155][documentation] Sync Chinese version of documentation to configure CheckpointFailureManager (#17173) (#17252) 14 September 2021, 03:09:40 UTC
742ba0a [FLINK-24212][k8s]fix the problem that kerberos krb5.conf file is mounted as empty directory, not the expected file This closes #17198 14 September 2021, 01:58:01 UTC
28677f3 [FLINK-24206][connector/pulsar] Close the pulsar client properly. Pulsar Consumer would be closed in SplitReader if it reaches the end of split. The registered Consumer would also be closed in PulsarClient if we call PulsarClient.close(). This would cause race condition. We have to use PulsarClient.shutdown() which don't close Consumer instead. This closes #17255 14 September 2021, 01:40:05 UTC
1e3a923 [hotfix][docs] Adding new sources on overview pages This closes #17266 13 September 2021, 16:02:17 UTC
dc94e85 [FLINK-24098] Document FLIP-147 capabiliites and limitations This closes #17135 13 September 2021, 13:00:40 UTC
b7e4550 [FLINK-24243][python] Cleanup code to use latest API to avoid warnings This closes #17237. 13 September 2021, 12:22:23 UTC
daad7e2 [FLINK-23458][docs] Added the network buffer documentation along with the buffer debloat doc 13 September 2021, 10:16:29 UTC
23d63ed [FLINK-24245][python] Fix the problem caused by multiple jobs sharing the loopback mode address stored in the environment variable in PyFlink This closes #17239. 13 September 2021, 07:28:54 UTC
79a801a [FLINK-24184][task] Introduce lock to guard against race conditions around shouldInterruptOnCancel Previously there was a potential race condition in disabling interrupts while closing resources. It used be guarded by a volatile variable, but there might have been a race condition when: 1. interrupter thread first checked the shouldInterruptOnCancel flag 2. shouldInterruptOnCancel flag switched to false as Task/StreamTask entered cleaning up phase 3. interrupter issued an interrupt while Task/StreamTask are closing/releasing resources, potentially causing a memory leak 13 September 2021, 07:24:41 UTC
b3a97b1 [FLINK-24244][python] Logging whether it's executed in loopback mode This closes #17234. 13 September 2021, 03:21:32 UTC
ed022d2 [FLINK-18880][python] Respect configurations defined in flink-conf.yaml and environment variables when executing in local mode This closes #17216. 13 September 2021, 03:17:51 UTC
22e7617 [FLINK-24065][connector] Upgrade the state of TwoPhaseCommitSink to support empty transaction after finished 13 September 2021, 02:38:55 UTC
7b213e4 [hotfix] Improve language for error message when checkpoint is declined due to some tasks being finished already. 10 September 2021, 14:08:18 UTC
b32e6a6 [FLINK-23864][docs] Add pulsar connector document (Chinese & English). 10 September 2021, 12:53:14 UTC
982db57 [FLINK-23864][connector/pulsar] Remove PULSAR_AUTO_UPDATE_PARTITIONS option. 10 September 2021, 12:53:14 UTC
d40cfc6 [FLINK-23864][connector/pulsar] Release Pulsar Message if user enable poolMessage option. 10 September 2021, 12:53:14 UTC
77012db [FLINK-23864][docs] Add flink-connector-pulsar module to flink-docs, auto generate the config document. 10 September 2021, 12:53:14 UTC
df3b723 [FLINK-24161] Fix interplay of stop-with-savepoint w/o drain with final checkpoints This closes #17200 10 September 2021, 09:18:24 UTC
d9e6abf [FLINK-24123][python] Optimize the python operator instances of the same job to share one python environment resources in the same jvm This closes #17180. 10 September 2021, 06:05:01 UTC
4e68c45 [FLINK-23944][connector/pulsar] Enable PulsarSourceITCase.testTaskManagerFailure after test framework was fixed. This closes #17201 10 September 2021, 03:00:18 UTC
4e732ad [FLINK-23848][connector/pulsar] Make sure the topic would only be sought once. This closes #17199 10 September 2021, 02:58:42 UTC
1a7bc49 [FLINK-23848][connector/pulsar] Fix the consumer not found. 10 September 2021, 02:58:41 UTC
3621b00 [FLINK-23345][python] Limits the version requests to 2.26.0 or above This closes #17219. 10 September 2021, 02:01:34 UTC
c56ecf3 [FLINK-23527][core] Clarify semantics of SourceFunction.cancel() with respect to thread interruptions. 09 September 2021, 15:18:44 UTC
9a26fe9 [hotfix][core] Make the example for checkpointed SourceFunction a bit less strange. 09 September 2021, 15:18:37 UTC
18e94e4 [hotfix] Deduplicate JavaDocs in SourceFunction 09 September 2021, 15:18:25 UTC
814f278 [hotfix] Remove outdated JavaDocs in SourceFunction about deprecated TimeCharacteristic 09 September 2021, 15:18:13 UTC
2af65df [hotfix][connectors] Deduplicate config default in FutureCompletingBlockingQueue. 09 September 2021, 15:17:17 UTC
a76854f [FLINK-22358][connector base] Add stability annotations to connector base and iterator sources. 09 September 2021, 15:17:04 UTC
2a13b24 [hotfix][core] Improve/fix JavaDocs in SplitEnumeratorContext class. 09 September 2021, 15:12:27 UTC
4aa457d [FLINK-22357][core] Upgrades API stability of unified source API to @Public (= stable). 09 September 2021, 15:12:23 UTC
2459a3c [FLINK-24213][qs] Use single lock in ServerConnection 09 September 2021, 12:37:13 UTC
a2185c0 [FLINK-24213][qs] Introduce factory for established connection 09 September 2021, 12:37:06 UTC
2db50d6 [FLINK-24137][python] Fix the issue that tests planned to run in process mode was actually executed in loopback mode This closes #17214. 09 September 2021, 12:25:11 UTC
4d131d5 [FLINK-24137][python] Fix the cases that Python boot process exits before Python SDK harness connects to the Java Operator in lookback mode This closes #17214. 09 September 2021, 12:25:07 UTC
e859af6 [FLINK-22889][tests] Increase timeouts in JdbcExactlyOnceSinkE2eTest ...to tolerate occasional long XA PREPARE executions in MySQL. 09 September 2021, 08:47:59 UTC
8e982b7 [FLINK-23462] [docs-zh] Translate the abfs documentation to chinese (cherry picked from commit 19804811351c5a66503bf2c6d3b626d1fd1f8c65) 09 September 2021, 08:08:27 UTC
adc9d43 [hotfix] Add .eslintcache to .gitignore The file only exists on master; the entry is just added for convenience. 09 September 2021, 07:48:28 UTC
e7766f7 [FLINK-24031] Extend vcs.xml 09 September 2021, 07:45:37 UTC
b56041b [FLINK-24160][tests] Use FS checkpoint storage in PartiallyFinishedSourcesITCase This closes #17165. 08 September 2021, 16:04:17 UTC
28109d0 [FLINK-24068][checkpoint] Also check for alignment start when received EndOfPartition from pending channels This closes #17131. 08 September 2021, 15:50:41 UTC
f907ef2 [FLINK-24034] Upgrade commons-compress to 1.21 08 September 2021, 14:24:36 UTC
21fd895 [FLINK-23807][connector/testing-framework] Use RestClient to detect TaskManager failure in test environments 08 September 2021, 13:51:55 UTC
b434f0a [FLINK-24126][connector/kafka] Use increment of bytes consumed/produced for updating numBytesIn/Out in Kafka connector (cherry picked from commit 0dc46ca03933890bc14b9025702b56e32ee8b40f) 08 September 2021, 13:51:09 UTC
7818495 [FLINK-23961][kubernetes] Hint user about missing 'kubernetes.cluster-id' config option 08 September 2021, 08:41:55 UTC
adb9009 [FLINK-24170][build] Use maven-directory-plugin:directory-of goal 08 September 2021, 08:30:58 UTC
a4cd43b [FLINK-24183][legal] Add aws-java-sdk-s3 to source NOTICE 08 September 2021, 08:16:35 UTC
602a177 [FLINK-23949][checkpoint] Fix first incremental checkpoint after a savepoint degenerate into a full checkpoint this fix #16969. Co-authored-by: wangfeifan <zoltar9264@163.com> Co-authored-by: jinghaihang <jinghaihang_hrbeu@163.com> 08 September 2021, 07:18:33 UTC
3a8a6b0 [FLINK-24163][test] Increase the checkpoint timeout for PartiallyFinishedSourcesITCase This closes #17170. 08 September 2021, 03:12:23 UTC
c10de14 [FLINK-24115][docs] Fix outdated SQL Temporal Join example (#17122) 08 September 2021, 02:43:22 UTC
6b4abd5 [FLINK-24137][python][tests] Upload logs for PyFlink tests 08 September 2021, 01:42:37 UTC
2235dcb [FLINK-24137][python] Add more logs during starting up Python worker 08 September 2021, 01:42:33 UTC
6472d5b [FLINK-23611][yarn-tests] Disables INFO log messages coming from YARN's ResourceLocalizationService We observed regular INFO log messages being produced by ResourceLocalizationService after the test ran into a timeout: ``` 22:51:31,785 [AsyncDispatcher event handler] INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService [] - Cache Size Before Clean: 0, Total Deleted: 0, Public Deleted: 0, Private Deleted: 0 22:51:32,398 [AsyncDispatcher event handler] INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService [] - Cache Size Before Clean: 0, Total Deleted: 0, Public Deleted: 0, Private Deleted: 0 ``` These log messages appeared every 10 minutes which prevented the ci/tools/test_controller.sh's watchdog mechanism to kick in. The watchdog mechanism relies on no output being produced for a given amount of time. This way, the test_controller script was unable to archive the YARN Flink log. 07 September 2021, 14:06:24 UTC
de07689 [hotfix][yarn-tests] Replaces runtime by timestamp The intend is to improve local debugging matching log events in different log files (YARN, Flink) via the timestamp. 07 September 2021, 14:06:24 UTC
27843b3 [hotfix][yarn-tests] Fixes path definition for local YARN-test working directory artifact collection 07 September 2021, 14:06:24 UTC
3c20439 [FLINK-24162][tests] Check MAX_WATERMARK per attempt in FLIP-147 tests Adjust DrainingValidator to validate subtask events if: - finished normally, last attempt - finished normally, recovered, finished - both attempts must emit MAX_WATERMARK and end input Skip if task wasn't finished and the attempt failed (by any subtask). 07 September 2021, 13:46:40 UTC
ba749b7 [hotfix][tests] Parameterize PartiallyFinishedSourcesITCase with failover strategy 07 September 2021, 13:46:40 UTC
510c2fc [FLINK-24151][connectors/kafka] Add concurrent checkpoint test to KafkaSinkITCase. 07 September 2021, 12:07:24 UTC
869befd [FLINK-24131][connectors/kafka] Fix KafkaWriter currentSendTime metric. The metric can only be registered once and should simply take the currentProducer to calculate. 07 September 2021, 12:07:24 UTC
40da6fc [FLINK-24131][connectors/kafka] Improve threading model of KafkaWriter. Removed pending records as it doesn't add anything to KafkaWriter#flush (same post-condition as per JavaDoc) but introduces instabilities because of concurrency. 07 September 2021, 12:07:24 UTC
back to top