Loading…
This event has ended. Visit the official site or create your own event on Sched.
View analytic
Wednesday, October 5 • 5:30pm - 6:20pm
Implementing an akka-streams materializer for big data

Sign up or log in to save this to your schedule and see who's attending!

Akka Streams provides a tremendously flexible architecture to build reactive pipelines that can be imported, exported and otherwise composed as partial DAGs. Its current, default implementation is ActorMaterializer which provides reactive streams across actors within a single JVM. 
Here we show how we implemented a GearpumpMaterializer which distributes reactive streams across a set of remote workers on the Apache Gearpump platform. We discuss how this was implemented and a number of challenges we faced with specific GraphStages and their semantics. Additional we cover how different materializer implementations can interoperate together to materialize different parts of the pipeline. We show that the changes we introduced internally within Akka Streams will enable other implementations of akka stream materializers and suggest a template based on our implementation. We will contribute the Gearpump materializer as open source to https://github.com/akka/akka-stream-contrib or make it available as part of an upcoming Apache Gearpump release.

Speakers
avatar for Kam Kasravi

Kam Kasravi

Senior Software Engineer, Intel Corp
Kam is a real time streaming architect developing analytic pipelines that integrate into Intel's Trusted Analytics Platform. He is an avid fan of akka, scala and platforms based on these architectures with past roles within eBay, Paypal and Yahoo.


Wednesday October 5, 2016 5:30pm - 6:20pm
Texas Ballroom

Attendees (29)