Spark – SQLContext
Today we will look at the SQLContext object from the PySpark library and how you can use it to connect to a local database. In the example below we will:
Connect to a local PostgreSQL database and read the contents into a dataframe.
Run some simple SQL queries
And join two data frames together
Let’s get started.