https://github.com/apache/flink

sort by:
Revision Author Date Message Commit Date
6044b7f Commit for release 0.10.0 30 October 2015, 11:56:20 UTC
698fbc3 [release][scripts] shade away curator correctly with different Scala versions 30 October 2015, 11:50:07 UTC
e1f30b0 [FLINK-2559] Clean up JavaDocs - Remove broken HTML tags like <br/>, <p/>, ... - close unclosed HTML tags - replaces special chars by HTML escaping, e.g., '<' by &lt; - wrap code examples by {@code} - fix incorrect @see and @link references - fix incorrect @throws - fix typos This closes #1298 29 October 2015, 12:25:27 UTC
44b03f2 [FLINK-2800] [kryo] Fix Kryo serialization to clear buffered data The Kryo serializer uses Kryo's Output class to buffer individual write operations before it is written to the underlying output stream. This Output class is flushed by Flink's KryoSerializer upon finishing its serialize call. However, in case of an exception when flushing the Output, the buffered data is kept in the buffer. Since Flink uses EOFExceptions to mark that an underlying buffer is full and has to be spilled, for example, it can happen that the record triggering the spilling is written twice after it is rewritten. The reason is that Kryo's Output buffer still contains the serialization data of the failed attempt which is also flushed to the emptied output stream. This duplication of records can lead to corrupted data which eventually let's the Flink program crash. The problem is solved by clearing Kryo's Output when the flush operation was not successful. This closes #1308 29 October 2015, 12:25:07 UTC
cdc0dfd [FLINK-2932] Examples in docs now download shell script using https instead of http This closes #1309 29 October 2015, 12:25:07 UTC
fcc1eed [FLINK-2902][web-dashboard] Sort finished jobs by their end time, running jobs by start time This closes #1296 29 October 2015, 12:25:07 UTC
6a13b9f [FLINK-2934] Remove placeholder pages for job.statistics, taskmanager.log and taskmanager.stdout This closes #1307 29 October 2015, 12:25:07 UTC
51ac46e [FLINK-1610][docs] fix javadoc building for aggregate-scaladoc profile 28 October 2015, 16:10:49 UTC
54375b9 [scala-shell][docs] add scala sources in earlier phase Otherwise the javadoc fails to generate.. 28 October 2015, 16:10:45 UTC
ae19d2b [FLINK-2927] [runtime] Provide default required configuration keys in flink-conf of binary distribution 27 October 2015, 17:27:13 UTC
874c500 Add org.apache.httpcomponents:(httpcore, httpclient) to dependency management 27 October 2015, 17:26:35 UTC
04e25e1 [docs] remove hard-coded version artifacts 27 October 2015, 14:15:59 UTC
b240a80 [FLINK-2878] [webmonitor] Fix unexpected leader address pattern The HandlerRedirectUtils.getRedirectAddress decides whether the retrieved leader address is equal to the local job manager address. The local job manager address is, however, in the form akka.tcp://flink@url/user/jobmanager whereas the leader address can be akka://flink/user/jobmanager if the local job manager is the current leader. Such a case produced a warning which is not correct. This PR checks for the local job manager address and signals that no redirection has to be done if it receives akka://flink/user/jobmanager. Add test for HandlerRedirectUtils This closes #1280. 27 October 2015, 13:03:33 UTC
16a9edc [hotfix] Fix issue with spaces in Path in start-*-streaming.sh 27 October 2015, 12:49:34 UTC
65fcd3a [FLINK-2891] [streaming] Set keys for key/value state in window evaluation of fast-path windows. 25 October 2015, 18:12:28 UTC
8e4cb0a [FLINK-2888] [streaming] State backends return copies of the default values 25 October 2015, 18:12:28 UTC
c2811ce [FLINK-2866] [runtime] Eagerly close FSDataInputStream in file state handle This closes #1282 25 October 2015, 18:12:28 UTC
ec1730b [docs] add information on how to use Kerberos 23 October 2015, 16:15:28 UTC
856b278 [hotfix] Improve handling of Window Trigger results This enhances the TriggerResult enum with methods isFire() and isPurge() that simplify the logic in WindowOperator.processTriggerResult(). Also, the operator now keeps track of the current watermark and fires immediately if a trigger registers an event-time callback for a timestamp that lies in the past. For this the TriggerResult now as method merge() that allows to merge to TriggerResultS. 23 October 2015, 15:38:24 UTC
bc5b852 [hotfix] Add Window Parameter in Trigger.onEventTime/onProcessingTime Before these trigger methods had no information about the window that they are responsible for. This information might be required for implementing more advanced trigger behaviour. 23 October 2015, 15:38:24 UTC
15d3f10 [FLINK-2895] Duplicate immutable object creation Operators defer object creation when object reuse is disabled. This closes #1288 23 October 2015, 14:05:26 UTC
8ec828c [FLINK-2893] [runtime] Consistent naming of recovery config parameters Rename config key prefix from 'ha.zookeeper' to 'recovery.zookeeper' Rename config key from 'state.backend.fs.dir.recovery' => 'state.backend.fs.recoverydir' Move ZooKeeper file system state backend configuration keys This closes #1286 23 October 2015, 14:05:26 UTC
c0d7073 [FLINK-1982] [record-api] Remove dependencies on Record API from flink-runtime tests Rename Match*Test to Join*Test and MapTaskTest to FlatMapTaskTest This closes #1294 23 October 2015, 14:05:26 UTC
712c868 [hotfix] Fix Mutable Object window aggregator/Disable Object Copy This fixes the aggregators to make copies of the objects so that it works with window operators that are not mutable-object safe. This also disables object copy in WindowOperator and NonKeyedWindowOperator. 23 October 2015, 13:46:28 UTC
45ab0eb [hotfix] Fix broken copy in OperatorChain Before, the StreamRecords was not copied, now it is. 23 October 2015, 13:46:20 UTC
c257abf Add copy() to Tuple base class. 23 October 2015, 13:46:11 UTC
85b73e0 [hotfix] Fix processing time triggering on Window Operator Before it would only trigger if expectedTime < time. Now it is expectedTime <= time. 23 October 2015, 09:15:21 UTC
c72eff4 [FLINK-2874] Fix recognition of Scala default setters 23 October 2015, 08:09:34 UTC
42b5ead [FLINK-2874] Fix Avro getter/setter recognition This closes #1252 23 October 2015, 08:09:34 UTC
5c3eb8b [FLINK-2668] [DataSet] [api-breaking] Chained Projections are no longer appended This closes #1279 23 October 2015, 08:09:21 UTC
dadb1a8 [FLINK-2206] [webui] Fix incorrect counts of finished, canceled, and failed jobs in new web dashboard This closes #1287 23 October 2015, 08:09:10 UTC
e340f83 [FLINK-2891] Set KV-State key upon Window Evaluation in General Windows Before, this was not set, leading to incorrect results if KV-State was used in the WindowFunction. This also adds a test. 22 October 2015, 14:02:52 UTC
db19973 [FLINK-2887] [gelly] make sendMessageToAllNeighbors respect the EdgeDirection if set in the configuration This closes #1281 22 October 2015, 11:55:15 UTC
4c1cffd [FLINK-2820] Configuration not passed to JobGraphGenerator This was previously reported as FLINK-2625 (commit 8a84937215ea575fa94a00d11c2517902d252756). The Client class was concurrently refactored with FLINK-2097 (commit 71bf2f570861daae53b24bfcf1d06aedb85311b9). This closes #1278 21 October 2015, 11:41:54 UTC
6666ea5 [FLINK-2876] Minutiae A collection of small documentation and grammar updates. This closes #1277 21 October 2015, 11:37:03 UTC
7a959bc [FLINK-2880] [streaming] Allow DeserializationSchema to forward exceptions. This closes #1275 21 October 2015, 09:52:39 UTC
dd3264e [FLINK-2834] Global round-robin for temporary directories Multiple TaskManager filesystems can be used by configuring multiple temporary directories. This patch changes the process of spilling files from a per-operator round-robin to a global round-robin such that each directory is written to in turn across all operators, reducing unbalanced I/O due to bunching. This closes #1272 21 October 2015, 09:46:06 UTC
e3cabe7 [docs] Fix documentation for building Flink with Scala 2.11 or 2.10 This closes #1260 21 October 2015, 09:45:01 UTC
5fb1c47 [FLINK-2743] Add XORShfitRandom and use it in RandomSamplers. This closes #1170 21 October 2015, 09:43:58 UTC
22510f0 [hotfix] Add shortcuts for creating Time objects This adds Time.milliseconds(), Time.seconds(), and so on. 21 October 2015, 09:37:44 UTC
06f6ac5 [FLINK-2877] Move Streaming API out of Staging package 21 October 2015, 09:01:43 UTC
42a3d8c [FLINK-2885][python] fix path building of Python resources 21 October 2015, 08:35:34 UTC
8b2d4f8 [FLINK-2872] [Documentation] Update the documentation for Scala part to add ExecutionEnvironment.readFileOfPrimitives. Add the missing Scala part for ExecutionEnvironment.readFileOfPrimitives API doc in the programming guide. Author: Henry Saputra <hsaputra@apache.org> Closes #1268 from hsaputra/add_readFileOfPrimitives_scala_guide and squashes the following commits: f543436 [Henry Saputra] Add variant of ExecutionEnvironment.readFileOfPrimitives in Java with additional delimiter. 1209019 [Henry Saputra] [FLINK-2872] [Documentation] Update the documentation for Scala part to add readFileOfPrimitives. 20 October 2015, 20:58:58 UTC
f760b61 Replace Trigger.onTime by Trigger.onProcessingTime/onEventTime This also renames WatermarkTrigger to EventTimeTrigger and ContinuousWatermarkTrigger to ContinuousEventTimeTrigger. 20 October 2015, 16:39:12 UTC
4442269 [FLINK-2864] Make State of General-Purpose Window Operators Fault-Tolerant This adds method state() on Trigger context that should be used to create an OperatorState to deal with fault-tolerant state. WindowAssigner now has a method getWindowSerializer() that is used to get a TypeSerializer for the Windows that it assigns. The Serializer for the Key is retrieved from the input KeyedStream and the serializer for the input elements is already available. During checkpointing all currently in-flight windows (per key, per window) are serialized using the TypeSerializers. The state that is accessible in Triggers using state() is kept in a HashMap<String, Serializable>, this is serialized using java serialization. 20 October 2015, 16:39:12 UTC
e711969 [FLINK-2873] detect & serve the job manager log files correctly 20 October 2015, 15:07:40 UTC
6c44d93 [FLINK-2865] remove upper direct memory size bound - set the upper bound to Long.MAX_VALUE For YARN, we set it to the calculated maximum container size (no need to fix). 20 October 2015, 08:10:42 UTC
a6890b2 [FLINK-2354] [runtime] Replace old StateHandleProvider by StateStorageHelper in ZooKeeperStateHandleStore The old StateHandleProvider used in ZooKeeperStateHandleStore had to be replaced because the state backend implementation has changed. Since the new state backend could not be used anymore, a new StateStorageHelper interface has been created. The default implementation FileSystemStateStorageHelper stores the given state onto the specified file system and returns a FileSerializableStateHandle. Various fixes due to rebasing. 20 October 2015, 07:57:34 UTC
630798d [FLINK-2852] [test-stability] Fix ScalaShellITSuite and ScalaShellLocalStartupITCase Changes test program to use an int accumulator which is checked at the end of the program. This avoids to look for the "Job status changed to FINISHED." string in the stdout output, which can sometimes not be printed if the JobExecutionResult arrives earlier than the JobStatusChanged message at the JobClientActor. 19 October 2015, 22:16:53 UTC
e3ad962 [FLINK-2793] [runtime-web] Rework JobManagerRetriever to avoid race conditions The JobManagerRetriever sets the new leaderGatewayPortFuture directly in the notifyLeaderAddress method instead of in one of the futures. This avoids race conditions between multiple futures which finish in a different order than they were started. Furthermore, this replaces promises by futures where a promise is not needed. Add logging statement Fix WebRuntimeMonitorITCase to use random port and proper state backend Add ChannelHandler.Sharable to RuntimeMonitorHandler Remove sanity check from WebInfoServer to let it work on Yarn 19 October 2015, 22:16:53 UTC
77fc0cc [FLINK-2793] [runtime-web] Redirect to leader in non-standalone mode Squashes: 5a88d5e [tests] Add HttpTestClient for testing HTTP responses 656d6d6 Split WebMonitor and LeaderRetrievalService start up a7e8da8 Move generated /web files to src/main/resources Add comment to webMonitorPort attribute and make line breaks more Scalaesque Don't block on leader retrieval and only resolve associated job manager once Make JobManagerRetriever independent of redirecting logic This closes #1202. 19 October 2015, 22:16:52 UTC
d18f580 [FLINK-2804] [runtime] Add blocking job submission support for HA The JobClientActor is now repsonsible for receiving the JobStatus updates from a newly elected leader. It uses the LeaderRetrievalService to be notified about new leaders. The actor can only be used to submit a single job to the JM. Once it received a job from the Client it tries to send it to the current leader. If no leader is available, a connection timeout is triggered. If the job could be sent to the JM, a submission timeout is triggered if the JobClientActor does not receive a JobSubmitSuccess message within the timeout interval. If the connection to the leader is lost after having submitted a job, a connection timeout is triggered if the JobClientActor cannot reconnect to another JM within the timeout interval. The JobClient simply awaits on the completion of the returned future to the SubmitJobAndWait message. Added test cases for JobClientActor exceptions This closes #1249. 19 October 2015, 22:16:52 UTC
c3a4d1d [FLINK-2805] [blobmanager] Write JARs to file state backend for recovery Move StateBackend enum to top level and org.apache.flink.runtime.state Abstract blob store in blob server for recovery This closes #1227. 19 October 2015, 22:16:52 UTC
c2989f2 [FLINK-2354] [runtime] Remove state changing futures in JobManager Internal actor states must only be modified within the actor thread. This avoids all the well-known issues coming with concurrency. Fix RemoveCachedJob by introducing RemoveJob Fix JobManagerITCase Add removeJob which maintains the job in the SubmittedJobGraphStore Make revokeLeadership not remove the jobs from the state backend Fix shading problem with curator by hiding CuratorFramework in ChaosMonkeyITCase 19 October 2015, 22:16:52 UTC
73c73e9 [FLINK-2354] [runtime] Add job graph and checkpoint recovery This closes #1153. 19 October 2015, 22:16:51 UTC
3aaee1e [FLINK-2792] [jobmanager, logging] Set actor message log level to TRACE 19 October 2015, 22:16:51 UTC
b233946 [FLINK-2652] [tests] Temporary ignore flakey PartitionRequestClientFactoryTest 19 October 2015, 22:16:51 UTC
5671c77 [FLINK-2107] Add hash-based strategies for left and right outer joins. This closes #1262 19 October 2015, 14:17:42 UTC
c900577 [FLINK-2842] [documentation] Remove Flink S3FileSystem, extend documentation to use Hadoop S3FileSystem. This closes #1245 19 October 2015, 14:17:37 UTC
640e63b [FLINK-2857] [gelly] Improve Gelly API and documentation. - Improve javadocs of Graph creation methods - Add fromTuple2 creation methods - Rename mapper parameters to vertexInitializer. - Improve javadocs and parameter names of joinWith* methods - Improve javadocs of neighborhood methods - Update docs to reflect api changes This closes #1263 19 October 2015, 13:39:37 UTC
da248b1 [FLINK-2725] Add Max/Min/Sum aggregation for mutable types. This closes #1191 19 October 2015, 13:39:28 UTC
6491559 [FLINK-2809] [scala-api] Added UnitTypeInfo and UnitSerializer. This closes #1217 19 October 2015, 13:38:31 UTC
8c1719c [FLINK-2844] remove old config entry from flink-conf.yaml 19 October 2015, 13:19:22 UTC
03579bb [FLINK-2731][web-dashboard] add access to JobManager stdout and logs This closes #1233. 19 October 2015, 12:32:09 UTC
da071bc [FLINK-2611][yarn] do not fail application in shutdown hook The application status is set to failed for jobs which did not complete successfully. If the client shuts down the YARN cluster, the applications should be reported as SUCCESSFUL. 19 October 2015, 10:06:34 UTC
d6567a0 [hotfix] Add debug output to (rarely) failing TimestampITCase I'm adding this so that we have a bit more context when it fails next time. 19 October 2015, 08:34:10 UTC
d047ddb [FLINK-2714] [gelly] add the algorithm description in the gelly docs; update test to get the directed graph as input This closes #1250 18 October 2015, 08:54:59 UTC
b443cb6 [FLINK-2714] [gelly] Copy triangle counting logic from EnumTrianglesOpt.java to Gelly library. Also reorganizing classes to use Gelly's Graph APIs. 18 October 2015, 08:51:58 UTC
580768c [FLINK-2841] [docs] Correcting ML roadmap link to point to confluence. This closes #1254 17 October 2015, 16:45:02 UTC
5ae0d12 [hotfix] Removed broken dependency to flink-spargel. This closes #1259 17 October 2015, 16:45:02 UTC
4593109 [FLINK-2844] [web frontend] Make web frontend URLs relative for YARN support This closes #1246 17 October 2015, 16:45:02 UTC
df44862 [FLINK-2844] [web frontend] Remove old web interface - make new web one the default - adapt tests - make web directory a resource to be included in the fat jar - serve static files of web interface dynamic through the class loader - run on YARN - remove Jetty dependencies from poms 17 October 2015, 16:45:02 UTC
a8eeb3b [FLINK-2863] [kafka connector] Kafka connector propagates async producer exceptions 17 October 2015, 16:45:02 UTC
728df39 [hotfix] Fix DataSet API programming guide 17 October 2015, 05:38:13 UTC
c78aded [FLINK-2855] [gelly] Add documentation for the Gelly library algorithms and improved javadocs for the library constructors. This closes #1258 16 October 2015, 16:35:46 UTC
34c232e [hotfix] [storm compatibility] Deactivate tests for split stream field grouping, which do not work in teh runtime and are now caught earlier 16 October 2015, 14:31:11 UTC
d68c8b1 [hotfix] [streaming] Remove obsolete internal state handle classes 16 October 2015, 13:26:11 UTC
7c20543 [FLINK-2550] [streaming] Allow multiple key/value states per operator on top of the new state backend 16 October 2015, 13:26:11 UTC
bb1f5fd [hotfix] [streaming scala] Expose key type information for key selectors on connected data streams 16 October 2015, 13:26:11 UTC
4ee5b4c [hotfix] [tests] Make StreamTaskTimerTest more robust 16 October 2015, 13:26:11 UTC
3b7fb63 [FLINK-2550] [tests] Add an end-to-end failure/recovery test for fast path processing time windows 16 October 2015, 13:26:11 UTC
da159ef [FLINK-2846] [streaming] Emit downstream checkpoint barriers at beginning of the checkpoint scope 16 October 2015, 13:26:11 UTC
ca8c73d [hotfix] Proper exception chaining in key/value state access of StreamingRuntimeContext 16 October 2015, 13:26:10 UTC
b81598e [hotfix] [core] TypeExtractor correctly handles non-public types as generic types (rather than failing with an exception) 16 October 2015, 13:26:10 UTC
d5a016c [hotfix] StreamTask and OperatorChain properly clean up partially initialized resources upon failures during initialization 16 October 2015, 13:26:10 UTC
f2d5038 [hotfix] [streaming] Initialize StreamingRuntimeContext to rich functions early 16 October 2015, 13:26:10 UTC
69dfc40 [FLINK-2550] [streaming] Rework JoinStreams and CoGroupStreams to properly implement operator builder syntax 16 October 2015, 13:26:10 UTC
c24dca5 [FLINK-2550] [streaming] Make fast-path processing time windows fault tolerant 16 October 2015, 13:26:10 UTC
479bec0 [FLINK-2808] [streaming] Refactor and extend state backend abstraction 16 October 2015, 13:26:10 UTC
5ac2872 [hotfix] Remove remaning classes for old window triggers 16 October 2015, 13:26:10 UTC
eec2d15 [hotfix] Correct name of HDFS tests from 'org.apache.flink.tachyon' to 'org.apache.flink.hdfstests' 16 October 2015, 13:26:09 UTC
6cb0fb5 [FLINK-2856] Introduce flink.version property into quickstart archetype 15 October 2015, 13:37:11 UTC
c82ebbf [FLINK-2774] [scala shell] Extended default imports for ScalaShell This closes #1247 -- PRs closed due to inactivity This closes #1077 15 October 2015, 09:34:01 UTC
fbc18b9 [FLINK-2479] Refactor runtime.operators.* tests This closes #1160 15 October 2015, 09:30:42 UTC
d9e32da [FLINK-2843] Add documentation for DataSet outer joins. This closes #1248 15 October 2015, 09:28:57 UTC
f7ab4f3 Change "it's" contraction to possession "its" This closes #1256 14 October 2015, 21:30:35 UTC
a7a1cbc [hotfix] Add default Trigger for GlobalWindows This also adds notes about possible non-parallelism for windowAll windows. 11 October 2015, 16:21:58 UTC
5dfc897 [FLINK-2817] [streaming] FileMonitoring function logs on empty location Instead of throwing NPE when location is empty Closes #1251 11 October 2015, 12:29:09 UTC
2475c82 [docs] fix typo in Scala functions type descriptors Closes #1253 11 October 2015, 12:26:45 UTC
86080bb [hotfix] Add countWindow and countWindowAll shortcut 11 October 2015, 10:12:41 UTC
back to top