Coding Musings

Trying to share some code ideas


Spark – SQLContext

Spark – SQLContext

Today we will look at the SQLContext object from the PySpark library and how you can use it to connect to a local database.  In the example below we will:
Connect to a local PostgreSQL database and read the contents into a dataframe.
Run some simple SQL queries
And join two data frames together

Let’s get started.

Hadoop Spark – Hello World, Word Count

Hadoop Spark – Word Count

One of the first things to do in most programming languages is create a “Hello World!” program.  The equivalent in Spark is to create a program that will read the contents of a file and count the number of occurrences of each word.

Below I will show a basic example, so let’s start counting.

Read CSV files using Python Panda Dataframes

The great thing about the Panda’s library for Python is how easily it can manipulate data sources.  We will look at one of the first things you will want to do, read a .csv file.

WebGL – Rotating Cube

In this posting I will again be using WebGL.  However as opposed to just drawing a triangle and doing a gradient fill, which you can see here.  Gradient Triangle.

The code listed below will use WebGL to draw a cube and make it rotate.

WebGL Rotating a Cube

WebGL – Hello World

WebGL is really a fascinating topic to me. Being able to utilize the power of a local machine’s GPU via the web browser is a very novel concept.

If one was to ask back in 1996 would you be able to render Open GL code in a web browser via  GPU, even the most tech savvy may have replied, “Open GL? GPU? what are those things, you speak about?”

WebGL Hello World

Class for Parsing NGrams

Train an NGram Model

Will take the ngrams, reading and writing from the SQLite database and train the model utilizing Katz-Backoff methodology on the likelihood of the next word.

See code below

File Handler for RDS Formatted Files

File Handler for RDS Formatted Files

Using a list, load RDS formatted files into standard data frame. Allows for grouping multiple files into one data frame for processing.

See code below

SQLite Interface Class for R

SQLite Interface Class for R

I wrote an interface class to help with working with the SQLite database.

Methods include, selectQuery, updateQuery, readTable, writeTable


see code below

Binary Search Tree – Is BST?

Binary Search Tree

Traverse the tree in natural order and test if each node is between the previous and the next node. Properly detect duplicate values. If a wrong sub-tree is encountered immediately stop the script and indicate the error.

See code below

Merging Tables

Simulates a sequence of merge operations with tables in a database.

See code below

Page 3 of 4

Powered by WordPress & Theme by Anders Norén