This comprehensive video presentation explores the concepts of streaming and real-time data processing, distinguishing between low latency and high latency and elucidating their impact on data processing. The discussion encompasses the differences between real-time and near real-time processing, emphasizing the necessity of clear communication with stakeholders. Challenges and considerations in working with streaming data, such as out-of-order events and late-arriving data, are addressed. Additionally, the importance of fault tolerance and its relation to data recovery is touched upon. The video serves as a valuable resource for data engineers and anyone interested in gaining insights into the intricacies of streaming and real-time data processing. [Recorded on Nov 28th]
60 mins