Reactive Summit 2016

Everyone is talking about microservices, but there is more confusion than ever about what the promise of microservices really means and how to deliver on it. In this talke we will explore microservices from first principles, distilling their essence and putting them in their true context: distributed systems.

We will start by examining individual microservices and explaining why it is important to adhere to the core traits of isolation, single responsibility, autonomy, exclusive state, asynchronous message-passing, and mobility. But what many people forget is that microservices are collaborative by nature and only make sense as systems. It is in between the microservices that the most interesting and rewarding, but also challenging, problems arise—here we are entering the world of distributed systems.

Distributed systems are inherently complex, and we enterprise developers have been spoiled by centralized servers for too long to easily understand what this really means. Slicing an existing system into various REST services and wiring them back together again with synchronous protocols and traditional enterprise tools—designed for monolithic architectures—will set you up for failure. What we need in order to build resilient, elastic, and responsive microservices-based systems is to embrace microservices as systems and re-architect them from the ground up using reactive principles.

In this hands on talk and demonstration I'll give a very short introduction to stream processing and then dive into writing code and demonstrating the features in Apache Flink that make truly robust stream processing possible.  We'll focus on correctness and robustness in stream processing.

During this live demo we'll be developing a realtime analytics application and modifying it on the fly based on the topics we're working though.  We'll exercise Flink's unique features, demonstrate fault-recovery, clearly explain and demonstrate why Event Time is such an important concept in robust stateful stream processing and talk about and demonstrate the features you need in a stream processor to do robust stateful stream processing in production.

We'll also use a realtime analytics dashboard to visualize the results we're computing in realtime.  This will allow us to easily see the effects of the code we're developing as we go along.

Some of the topics covered will be:

• Apache Flink
• Stateful Stream Processing
• Event Time vs. Processing Time
• Fault tolerance
• State management in the face of faults
• Savepoints
• Data re-processing

 

Vert.x is a framework to create reactive distributed and polyglot applications on the Java Virtual Machine.

Vert.x takes the JVM to new levels of performance yet having a small API. It lets you build scalable microservice-based applications transparently distributed and packaged as a single jar file.

Due to this simplicity, deploying and managing Vert.x applications on OpenShift is a breeze, upload your jar and Vert.x internal cluster manager will connect all your pods in single distributed network. Several examples are shown during the talk and demonstrate how Vert.x can simplify DevOps daily job when working together with OpenShift such as deployment, rolling updates, monitoring, metrics...

Enterprises hear about the promise of application containers, but realizing meaningful business results from containers requires more than abandoning virtual machines. In order to implement containers correctly, businesses must consider the operational implications, as well as the new types of applications they want to build using microservices. In this session, Ed Hsu, Vice President of Enterprise DC/OS at Mesosphere, discusses how to capitalize on new opportunities that can accelerate your IT modernization initiatives.

Groundhog Day, Die Hard, and The Dark Knight aren't just great movies. They're systems that withstand onslaughts of stress and surprise. Can studying great stories help architects and coders of reactive systems think about their work in new ways? What do superpowers tell us about scalability? What does the structure of subplots teach us about resilience? What do genre-busting movies like Rushmore, Fight Club, and Deadpool say about the future of elasticity? There are strange rules for how stories work. Those rules offer clues to building better systems.

Apache Kafka is now widely adopted among modern distributed systems. In this presentation we will see how Akka Streams can add even more power to Kafka by exposing it as a reactive stream. We will see some examples of how the Reactive Kafka library can be leveraged to build a well-fitting and elegant match of partitioned, pull-based distributed log and asynchronous, backpressured streaming model.

Kafka is core to Twitter's historical data products, offering access to every tweet since 2006. Learn how we reliably consume and process thousands of messages per second, serving Twitter's realtime and historical data APIs without skipping a beat.

To learn how we use Kafka to serve varying business needs, we'll trace the path of a tweet through our entire architecture, focusing on the reliability of that path. We'll discuss how our architecture has evolved over the years to respond to business needs and technological changes as well as lessons learned keeping this system flowing.

At Cake Solutions, we build highly distributed and scalable systems using Kafka as our core data pipeline.

Kafka has become the de facto platform for reliable and scalable distribution of high-volumes of data. However, as a developer, it can be challenging to figure out the best architecture and consumption patterns for interacting with Kafka while delivering quality of service such as high availability and delivery guarantees. It can also be difficult to understand the various streaming patterns and messaging topologies available in Kafka.

In this talk, we present the patterns we've successfully employed in production and provide the tools and guidelines for other developers to choose the most appropriate fit for given data processing problem. The key points for the presentation are: patterns for building reactive data pipelines, high availability and message delivery guarantees, clustering of application consumers, topic partition topology, offset commit patterns, performance benchmarks, and custom reactive, asynchronous, non-blocking Kafka driver.

Large-scale distributed systems must be built to anticipate and mitigate a variety of hardware and software failures. In order to build confidence that fault-tolerant systems are correctly implemented, an increasing number of large-scale sites practice Chaos Engineering, running regular failure drills in which faults are deliberately injected in their production system.  While fault injection infrastructures are becoming relatively mature, existing approaches either explore the space of potential failures randomly or exploit the “hunches” of domain experts to guide the search—the combinatorial space of failure scenarios is too large to search exhaustively. Random strategies waste resources testing “uninteresting” faults, while programmer-guided approaches are only as good as the intuition of a programmer and only scale with human effort. 
In this talk, I will present intuition, experience and research directions related to lineage-driven fault injection (LDFI), a novel approach to automating failure testing.  LDFI utilizes existing tracing or logging infrastructures to work backwards from good outcomes, identifying redundant computations that allow it to aggressively prune the space of faults that must be explored via fault injection.  I will describe LDFI’s theoretical roots in the database research notion of provenance, present results from the lab as well as the field, and present a call to arms for the reliability community to improve our understanding of when and how our fault-tolerant systems actually tolerate faults.

A code-free presentation about the core principles and philosophies of
Erlang, explaining how the whole is greater than the sum of its parts.

This presentation takes a perspective rooted in showing the reasoning
between each of the building blocks (message-passing, isolated
processes, links and monitors, and so on), and shows how they can be
assembled together to create larger systems where interactions between
subsystems take a front seat in defining the fault tolerance of these
systems.

This session describes our experience adopting a functional programming architecture approach for large-scale, complex enterprise application delivery. We describe a software architecture approach and patterns which provide: 

• More direct correspondence between business logic and code  
• Decoupling of business components from each other and from architecture plumbing  
• Testability  
• Painless support for non-blocking execution, for high-performance 

We address delivery challenges encountered with the adoption of Scala and the Lightbend stack: 

• Technology stack complexity  
• Shortage of skilled Scala architects and developers with enterprise software delivery background  
• Development, testing, and operations are much more challenging with non-blocking code than with traditional blocking code 

A modern business operates 24/7 and generates data continuously. Shouldn’t we process it continuously too?

A rich ecosystem of real-time data-processing frameworks, tools and systems has been forming around Apache Kafka that allows data to be processed continuously as it occurs. Jay Kreps will introduce Kafka and explain why it has become the de facto standard for streaming data. He will draw on practical experience building stream-processing applications to discuss the difference between architectures and the challenges each presents. Jay will then outline Kafka Streams, which offers new stream processing functionality in Kafka, and explain how it helps tame some of the complexity in real-time architectures.

Since its stable release earlier this year, Akka Streams is quickly becoming the de facto standard integration layer between various Streaming systems and products.

This comes from the Reactive Streams initiative in part, which has been long led by Lightbend and others, allowing multiple streaming libraries to inter-operate between each other in a performant and resilient fashion, providing back-pressure all the way. But perhaps even more so thanks to the various integration drivers that have sprung up in the community and the Akka team—including drivers for Kafka, Cassandra, Streaming HTTP, Websockets and much more.

In this talk we'll explore what and why Reactive integration for streaming matters, how Akka Streams makes it trivial, and what the future holds in this landscape.

There are plenty of reactive technologies out there, but these are only the building blocks for building reactive systems, using these technologies to build a system does not necessarily make the system reactive. A reactive system will have a fundamentally different architecture to the traditional monolith found in the enterprise.

In this presentation we take a hands on look at how the architecture of a system, including the flow of data, the types of communication used, and the way the system is broken down into components, will need to change as you decompose a monolith into a reactive microservice based system.

In this presentation the ConductR team will demonstrate a number of real-world scenarios that can occur with distributed systems. The demonstrations will be live and include a split brain scenario along with auto scaling based on demand. Things will no doubt go wrong so come and enjoy four guys making fools of themselves!

Netty is one of the best known and most widely used (if not the most widely used) asynchronous network application frameworks for the JVM.
This talk will show you how Netty itself works and explain why some  design choices were made. Beside this it will include war stories about
the many JVM-related challenges the Netty community has faced during Netty development and explain what action were taken to workaround these.