StreamingStreaming

The extension adds Operators to design and deploy streaming analytic process on Flink or Spark. This is an alpha version. Please use it carefully, as features are still heavily under development.

The extension adds Operators to design and deploy streaming analytic process on Flink or Spark.

This is an alpha version. Please use it carefully, as features are still heavily under development.

The streaming analytic process is designed inside the Streaming Nest operator. Processes are deployed by creating and providing a connection object to the corresponding cluster.

Processes designed are platform independent. They can be deployed on other platforms (changing also from Flink to Spark and vise-versa) by just changing the provided connection object.

Deployed streaming jobs can be monitored and managed by using the Streaming Dashboard (new in 0.2.0), which can be enabled over the View -> Show Panel menu.

The extension provides the following Operators:

  • Streaming

    • Financial Server

      • Get Quote Symbols

      • Get Quotes

      • Get Depths

      • Quote or Depth Stream (new in 0.2.0)

    • Kafka

      • Read Kafka Topic (improved and fixed in 0.2.0)

      • Write Kafka Topic (improved and fixed in 0.2.0)

    • Streaming Optimization (improved in 0.2.0)

    • Streaming Nest

    • Aggregate

    • Duplicate Stream

    • Filter Stream

    • Join Streams

    • Connect Streams

    • Kafka Sink

    • Kafka Source

    • Map Stream

    • Select Stream

    • Parse Field Stream

    • Stringify Field Stream

    • Synopsis Data Engine

    • Athena Online Machine Learning Engine

    • Complex Event Forecasting Engine

    • Maritime Event Detection

Version 0.2.0 (2020-12-09)

  • Read Kafka Topic and Write Kafka Topic now uses connection objects, also bugfix for using remote clusters
  • Added streaming operator (only flink) to access the financial data server from SPRING in a streaming workflow ( Quote or Depth Stream )
  • Added new Streaming Dashboard as a Panel in RapidMiner Studio to monitor and manage deployed streaming workflows
  • Update Streaming Optimization Operator to use connection objects, to have more advanced parameter handling and splitted the subprocess into two subprocesses, one handling the logical workflow and one showing the optimized workflow

Version 0.1.0 (2020-10-26)

  • Initial alpha version with the main functionality implemented.

  • Main functionality for the deployment of streaming analytic processes implemented

    • Added the Streaming Nest operator for designing and deploying streaming analytic processes

  • Added streaming operators to pull and push from/to Kafka (Kafka Source, Kafka Sink)

  • Added operators to pull from Kafka and convert to ExampleSet (Read Kafka Topic) and to convert ExampleSet to data events and push to Kafka (Write Kafka Topic)

  • Added operators to access the financial data server from SPRING (Get Quote Symbols, Get Quotes , Get Depths)

  • Added streaming operators to perform several basic streaming functionalities (Aggregate Stream , Duplicate Stream, Join Stream, Connect Stream, Filter Stream, Map Stream, Select Stream, Parse Field Stream, Stringify Stream)

  • Added integration with the INFORE Optimizer (Streaming optimization operator)

  • Added integration with different INFORE Components (Synopsis Data Engine, Athena Online Machine Learning Engine, Complex Event Forecasting Engine, Maritime Event Detection)


Product Details

Version 0.2.0
File size 168 MB
Downloads 439 (0 Today)439 downloads
Vendor RapidMiner Labs
Category Operators
Released 12/9/20
Last Update 12/9/20 10:09 AM
(Changes)
License AGPL
Product web site
Rating 0.0 stars(0)