I²: INTERACTIVE REAL-TIME VISUALIZATION FOR STREAMING DATA WITH APACHE FLINK AND APACHE ZEPPELIN
We present I², an interactive development environment for real-time analysis pipelines, which is based on Apache Flink and Apache Zeppelin. The sheer amount of available streaming data frequently makes it impossible to visualize all data points at the same time. I² coordinates running Flink jobs and corresponding visualizations such that only the currently depicted data points are processed in Flink and transferred towards the front end. We show how Flink jobs can adapt to changed visualization properties at runtime to allow interactive data exploration on high bandwidth data streams. Moreover, we present a data reduction technique which minimizes data transfer while providing loss free time-series plots. We show I² in a live demonstration in which we replay recorded sensor data from a football match (ca. 12k event/s). I² was first presented at EDBT’17 where it was awarded as best demonstration. The demonstration is available as open source at github.com/TU-Berlin-DIMA/i2.
The video from the talk and slides can be found here.