StreamingStreaming

The extension adds Operators to design and deploy streaming analytic process on Flink or Spark. This is an alpha version. Please use it carefully, as features are still heavily under development.

The extension adds Operators to design and deploy streaming analytic process on Flink or Spark.

This is an alpha version. Please use it carefully, as features are still heavily under development.

The streaming analytic process is designed inside the Streaming Nest operator. Processes are deployed by creating and providing a connection object to the corresponding cluster.

Processes designed are platform independent. They can be deployed on other platforms (changing also from Flink to Spark and vise-versa) by just changing the provided connection object.

Deployed streaming jobs can be monitored and managed by using the Streaming Dashboard (new in 0.2.0), which can be enabled over the View -> Show Panel menu.

The extension provides the following Operators:

  • Streaming

    • Financial Server

      • Get Quote Symbols

      • Get Quotes

      • Get Depths

      • Quote or Depth Stream

    • Kafka

      • Read Kafka Topic

      • Write Kafka Topic

    • Streaming Optimization

    • Streaming Nest

    • Aggregate

    • Duplicate Stream

    • Filter Stream

    • Join Streams

    • Connect Streams

    • Kafka Sink

    • Kafka Source

    • Map Stream

    • Union Stream (new in 0.3.0)

    • Select Stream

    • Parse Field Stream

    • Stringify Field Stream

    • Synopsis Data Engine (updated in 0.3.0)

    • Athena Online Machine Learning Engine

    • Spring Complex Event Forecasting Engine (new in 0.3.0)

    • Complex Event Forecasting Engine

    • Maritime Event Detection

Version 0.3.0 (2021-03-08)

  • Added Spring Complex Event Forecasting engine operator, to deploy the forecasting functionality provided by the SPRING research partner, in a streaming analytical workflow
  • Added Union Stream operator, to perform union operations on two or more input streams in a streaming analytical workflow
  • Updated the Synopsis Data Engine Operator, to include the latest developments of the corresponding engine.

Version 0.2.0 (2020-12-09)

  • Read Kafka Topic and Write Kafka Topic now uses connection objects, also bugfix for using remote clusters
  • Added streaming operator (only flink) to access the financial data server from SPRING in a streaming workflow ( Quote or Depth Stream )
  • Added new Streaming Dashboard as a Panel in RapidMiner Studio to monitor and manage deployed streaming workflows
  • Update Streaming Optimization Operator to use connection objects, to have more advanced parameter handling and splitted the subprocess into two subprocesses, one handling the logical workflow and one showing the optimized workflow

Version 0.1.0 (2020-10-26)

  • Initial alpha version with the main functionality implemented.

  • Main functionality for the deployment of streaming analytic processes implemented

    • Added the Streaming Nest operator for designing and deploying streaming analytic processes

  • Added streaming operators to pull and push from/to Kafka (Kafka Source, Kafka Sink)

  • Added operators to pull from Kafka and convert to ExampleSet (Read Kafka Topic) and to convert ExampleSet to data events and push to Kafka (Write Kafka Topic)

  • Added operators to access the financial data server from SPRING (Get Quote Symbols, Get Quotes , Get Depths)

  • Added streaming operators to perform several basic streaming functionalities (Aggregate Stream , Duplicate Stream, Join Stream, Connect Stream, Filter Stream, Map Stream, Select Stream, Parse Field Stream, Stringify Stream)

  • Added integration with the INFORE Optimizer (Streaming optimization operator)

  • Added integration with different INFORE Components (Synopsis Data Engine, Athena Online Machine Learning Engine, Complex Event Forecasting Engine, Maritime Event Detection)


Product Details

Version 0.3.0
File size 168 MB
Downloads 754 (2 Today)754 downloads
Vendor RapidMiner Labs
Category Operators
Released 3/8/21
Last Update 3/8/21 3:29 PM
(Changes)
License AGPL
Product web site
Rating 0.0 stars(0)