FREE Training Courses — 100+ Hours of Akka and Reactive Architecture Expertise Get Started
Support
akka akka-streams spark kafka kafka-streams fast-data-platform

Akka, Spark or Kafka? Selecting The Right Streaming Engine For the Job

A Look At Latency, Volume, Integration, And Data Processing Needs

View on Slideshare

Many engineers we talk to on a daily basis come to us with the same issue: that the batch-oriented architecture of Big Data–where data is captured in large, scalable stores, then processed later–is simply too slow. If you’re looking for a competitive advantage like our clients PayPal, HPE, Starbucks and Capital One, then you’ve got to embrace streaming and “Fast Data” architectures, where data is processed as it arrives.

But for most people we’ve talked to, there is rarely a “one size fits all” technology that can handle all streaming use cases. With so many stream processing tools, which ones should you choose? There are several considerations when making the right selection for the specific needs of your application, such as:

  • Latency: How low is necessary? What types of tasks are you processing? 
  • Volume: How high is required? Is Complex Event Processing involved?
  • Integration with other tools: Which ones and how? We know that nothing lives in isolation in these systems
  • Data processing: What kinds? In bulk? As individual events?

In this talk by Dean Wampler, PhD., VP of Fast Data Engineering at Lightbend, we’ll look at the criteria you need to consider when selecting technologies, plus the context and background to make good decisions when it comes to adopting streaming frameworks.

Using our Fast Data Platform as an example, which supports a host of Reactive and streaming technologies like Akka Streams, Kafka Streams, Apache Flink, Apache Spark, Mesosphere DC/OS and our own Reactive Platform, we’ll look at how to serve particular needs and use cases in both Fast Data and microservices architectures.


Watch The Full Video (40 Min)

Watch on YouTube


Additional Resources To Enjoy

As always, Lightbend is here to make your streaming, Fast Data, and Machine Learning journey successful. To find out more about our platform subscription, getting-started engagement services, or anything else, feel free to contact us below and schedule a 20-min introduction.

GET IN TOUCH