Reactive Summit 2016

Everyone is talking about microservices, but there is more confusion than ever about what the promise of microservices really means and how to deliver on it. In this talke we will explore microservices from first principles, distilling their essence and putting them in their true context: distributed systems.

We will start by examining individual microservices and explaining why it is important to adhere to the core traits of isolation, single responsibility, autonomy, exclusive state, asynchronous message-passing, and mobility. But what many people forget is that microservices are collaborative by nature and only make sense as systems. It is in between the microservices that the most interesting and rewarding, but also challenging, problems arise—here we are entering the world of distributed systems.

Distributed systems are inherently complex, and we enterprise developers have been spoiled by centralized servers for too long to easily understand what this really means. Slicing an existing system into various REST services and wiring them back together again with synchronous protocols and traditional enterprise tools—designed for monolithic architectures—will set you up for failure. What we need in order to build resilient, elastic, and responsive microservices-based systems is to embrace microservices as systems and re-architect them from the ground up using reactive principles.

Enterprises hear about the promise of application containers, but realizing meaningful business results from containers requires more than abandoning virtual machines. In order to implement containers correctly, businesses must consider the operational implications, as well as the new types of applications they want to build using microservices. In this session, Ed Hsu, Vice President of Enterprise DC/OS at Mesosphere, discusses how to capitalize on new opportunities that can accelerate your IT modernization initiatives.

Large-scale distributed systems must be built to anticipate and mitigate a variety of hardware and software failures. In order to build confidence that fault-tolerant systems are correctly implemented, an increasing number of large-scale sites practice Chaos Engineering, running regular failure drills in which faults are deliberately injected in their production system.  While fault injection infrastructures are becoming relatively mature, existing approaches either explore the space of potential failures randomly or exploit the “hunches” of domain experts to guide the search—the combinatorial space of failure scenarios is too large to search exhaustively. Random strategies waste resources testing “uninteresting” faults, while programmer-guided approaches are only as good as the intuition of a programmer and only scale with human effort. 
In this talk, I will present intuition, experience and research directions related to lineage-driven fault injection (LDFI), a novel approach to automating failure testing.  LDFI utilizes existing tracing or logging infrastructures to work backwards from good outcomes, identifying redundant computations that allow it to aggressively prune the space of faults that must be explored via fault injection.  I will describe LDFI’s theoretical roots in the database research notion of provenance, present results from the lab as well as the field, and present a call to arms for the reliability community to improve our understanding of when and how our fault-tolerant systems actually tolerate faults.

A modern business operates 24/7 and generates data continuously. Shouldn’t we process it continuously too?

A rich ecosystem of real-time data-processing frameworks, tools and systems has been forming around Apache Kafka that allows data to be processed continuously as it occurs. Jay Kreps will introduce Kafka and explain why it has become the de facto standard for streaming data. He will draw on practical experience building stream-processing applications to discuss the difference between architectures and the challenges each presents. Jay will then outline Kafka Streams, which offers new stream processing functionality in Kafka, and explain how it helps tame some of the complexity in real-time architectures.