Spark – Streaming Data, Capturing and Querying
Today we will look at how to capture streaming data and perform some simple queries as the data is streamed. We will use the regular expressions library and the PySpak library. The streaming data comes from a weather station that transmits different weather at different intervals. We will need to find the correct data out of the stream and output the results.
Let’s get started.