Patterns for Writing Structured Streaming Applications

Abstract

In the last chapter, we built up a streaming frame of reference for working with data in-flight. By introducing common problems and pitfalls that tend to occlude the path to streaming success, you discovered the nuances between local and global coordinated time, aka the trouble with time. You also learned about the complex problems that arise when corrupt or invalid data makes its way into your data network, in data problems in flight. In a perfect world, each event or data point would magically appear in the correct time-aligned (synchronous) order, but in a world devoid of perfection, there are tools you can use to lend a helping hand.