The extension adds Operators to design and deploy streaming analytic process on Flink or Spark. This is an alpha version. Please use it carefully, as features are still heavily under development.

The extension adds Operators to design and deploy streaming analytic process on Flink or Spark.

This is an alpha version. Please use it carefully, as features are still heavily under development.

The streaming analytic process is designed inside the Streaming Nest operator. Processes are deployed by creating and providing a connection object to the corresponding cluster.

Processes designed are platform independent. They can be deployed on other platforms (changing also from Flink to Spark and vise-versa) by just changing the provided connection object.

Deployed streaming jobs can be monitored and managed by using the Streaming Dashboard (updated in 0.4.0), which can be enabled over the View -> Show Panel menu.

The extension provides the following Operators:

  • Streaming

    • Financial Server

      • Get Quote Symbols

      • Get Quotes

      • Get Depths

      • Quote or Depth Stream

    • Kafka

      • Read Kafka Topic

      • Write Kafka Topic

    • Streaming Optimization (updated in 0.5.0)

    • Streaming Nest (updated in 0.5.0)

    • Aggregate

    • Duplicate Stream

    • Filter Stream

    • Join Streams

    • Connect Streams

    • Kafka Sink

    • Kafka Source

    • Map Stream

    • Union Stream

    • Select Stream

    • Parse Field Stream

    • Stringify Field Stream

    • Timestamp Stream (new in 0.5.0)

    • Synopsis Data Engine

    • Athena Online Machine Learning Engine

    • Spring Complex Event Forecasting Engine

    • Complex Event Forecasting Engine (updated in 0.5.0)

    • Maritime Event Detection

    • Apply Model on Stream

Version 0.4.0 (2021-10-21)

  • Added the possibility to provide an INFORE Optimizer connection object to the connection input port of the Streaming Nest operator.
    • In this case, the StreamGraph defining the streaming workflow is serialized and uploaded to the FileServer of the INFORE Optimizer Service.
    • Added the host and port parameters for the file server of the INFORE Optimizer Service to the corresponding connection class.
  • Added new operator Timestamp Stream 
    • This operator puts a timestamp on the events of a stream (assigns it to the key provided as parameter).
  • Updated the Complex Event Forecasting Engine operator with three new parameters ("domain stream", "timestamp key", "config topic").
  • Updated Streaming Optimization Operator to comply with the latest developments of the INFORE Optimizer Service

Version 0.4.0 (2021-09-14)

  • Added new operator Apply model on Stream, which enables to deploy a RapidMiner model trained offline in a streaming process
    • The streaming cluster used for this, needs to have the RapidMiner Core available
  • Updated Complex Event Forecasting operator
    • Added the parameters k and offset reset
  • Updated Streaming dashboard
    • Improved visuals and UX
    • Added the process location of the RM process which was used to deploy the streaming job.
  • Update Streaming Optimization operator
    • New connection object to connect to the INFORE Optimizer Service
    • Available computing sites can be more easily defined by an updated port structure
    • Logical and optimized workflow are now separated in different subprocess
    • Adapted configuration parameters and the internal working to changed interface of the INFORE Optimizer Service
  • Fixed problem with extensive logging in the financial source operators

Version 0.3.0 (2021-03-08)

  • Added Spring Complex Event Forecasting engine operator, to deploy the forecasting functionality provided by the SPRING research partner, in a streaming analytical workflow
  • Added Union Stream operator, to perform union operations on two or more input streams in a streaming analytical workflow
  • Updated the Synopsis Data Engine Operator, to include the latest developments of the corresponding engine.

Version 0.2.0 (2020-12-09)

  • Read Kafka Topic and Write Kafka Topic now uses connection objects, also bugfix for using remote clusters
  • Added streaming operator (only flink) to access the financial data server from SPRING in a streaming workflow ( Quote or Depth Stream )
  • Added new Streaming Dashboard as a Panel in RapidMiner Studio to monitor and manage deployed streaming workflows
  • Update Streaming Optimization Operator to use connection objects, to have more advanced parameter handling and splitted the subprocess into two subprocesses, one handling the logical workflow and one showing the optimized workflow

Version 0.1.0 (2020-10-26)

  • Initial alpha version with the main functionality implemented.

  • Main functionality for the deployment of streaming analytic processes implemented

    • Added the Streaming Nest operator for designing and deploying streaming analytic processes

  • Added streaming operators to pull and push from/to Kafka (Kafka Source, Kafka Sink)

  • Added operators to pull from Kafka and convert to ExampleSet (Read Kafka Topic) and to convert ExampleSet to data events and push to Kafka (Write Kafka Topic)

  • Added operators to access the financial data server from SPRING (Get Quote Symbols, Get Quotes , Get Depths)

  • Added streaming operators to perform several basic streaming functionalities (Aggregate Stream , Duplicate Stream, Join Stream, Connect Stream, Filter Stream, Map Stream, Select Stream, Parse Field Stream, Stringify Stream)

  • Added integration with the INFORE Optimizer (Streaming optimization operator)

  • Added integration with different INFORE Components (Synopsis Data Engine, Athena Online Machine Learning Engine, Complex Event Forecasting Engine, Maritime Event Detection)

Product Details

Version 0.5.0
File size 178 MB
Downloads 1412 (0 Today)1412 downloads
Vendor RapidMiner Labs
Category Operators
Released 10/21/21
Last Update 10/21/21 3:02 PM
License AGPL
Product web site
Rating 0.0 stars(0)