Vik's Blog (Page 2)

Predicting season records for NFL teams - part 2

This is the second, technical, part of this series. See the first part [https://www.vikparuchuri.com/predicting-nfl-season-records-with-percept] for the overview. Introduction This post will introduce the technical details behind the nfl season record prediction that was introduced in part one [https://www.vikparuchuri.com/predicting-nfl-season-records-with-percept]. After selecting the error

machine-learning

Predicting season records for NFL teams - overview

This is the first, non-technical, part of this series. See the second part [https://www.vikparuchuri.com/predicting-season-records-for-nfl-teams-part-2] for more detail. Introduction I was recently looking for a good machine learning task to try out, and I thought that doing something NFL-related would be interesting, because the NFL season is

nlp

Natural Language Processing Tutorial

Introduction This will serve as an introduction to natural language processing. I adapted it from slides for a recent talk at Boston Python. We will go from tokenization to feature extraction to creating a model using a machine learning algorithm. The goal is to provide a reasonable baseline on top

nlp

My Talk at Boston Python

I just gave a talk at Boston Python [http://www.meetup.com/bostonpython/events/107649802/] about natural language processing in general, and edX ease [https://github.com/edx/ease] and discern [https://github.com/edx/discern] in specific. You can find the presentation source here [https://github.com/VikParuchuri/boston-python-ml]

ansible

Creating a Wordpress Single or Multisite Install Using Cloudformation and Ansible

Intro I recently had to create some sites quickly. After evaluating a few options, setting up a wordpress multisite seemed like a good option. In order to make this change, I setup a wordpress multisite installation with domain mapping. A multisite installation is when one wordpress install lets you run

ggplot2

How Many Data Scientists Are There?

I’ve seen a lot of articles lately about “Big Data” and the looming “talent gap.” This article [http://online.wsj.com/article/SB10001424052702304723304577365700368073674.html] from the Wall Street Journal is a good example. It cites a McKinsey estimate that states that we will need 1.5 million more managers

text-mining

Tracking US Sentiments Over Time In Wikileaks

Introduction I recently posted [http://viksalgorithms.blogspot.com/2012/06/finding-word- use-patterns-in-wikileaks.html] about using the Wikileaks cable corpus to find word use patterns, both over time, and in secret cables vs unclassified cables. I received a lot of good suggestions for further topics to pursue with the corpus, and

basketball

NBA Predictions -- Finals

Now we are on to the finals! The algorithm enters the finals with a 6-4 record so far. Here is what we have for tonight: So, let’s see if OKC wins this one.

text-mining

Finding Word Use Patterns in Wikileaks Cables

6/18: A follow-up to this post is now available here [http://viksalgorithms.blogspot.com/2012/06/tracking-us-sentiments-over- time-in.html]. Recent Discoveries When I was a diplomat, I was always interested in the Wikileaks cables and what could be done with them. Unfortunately, I never got a chance to look

machine-learning

NBA Playoffs Update 5 (5-4)

This is the sixth post in my series on predicting the NBA playoffs [http://viksalgorithms.blogspot.com/2012/05/predicting-nba-finals- with-r.html] with an algorithm. After the Boston loss in their last game, the algorithm is now 5-4 in the playoffs. Hopefully it is correct tonight! Open Sourcing the Code

machine-learning

NBA Playoff Predictions Update 4 (5-3)

This is update 4 to my original post about predicting the NBA playoffs with R [http://viksalgorithms.blogspot.com/2012/05/predicting-nba-finals- with-r.html]. With the Thunder beating the Spurs and the Heat losing to the Celtics, the algorithm went 1-1 on predictions, making it 5-3 so far. Making some

machine-learning

NBA Playoff Predictions Update 3 (4-2)

This is my third update to my original post on predicting the NBA playoffs with an algorithm. [http://viksalgorithms.blogspot.com/2012/05/predicting-nba- finals-with-r.html] Here are updates 1 [http://viksalgorithms.blogspot.com/2012/06/predicting-nba-playoff-games- results.html] and 2 [http://viksalgorithms.blogspot.com/2012/06/nba-playoff- predictions-update-2-and.html]

machine-learning

NBA Playoff Predictions Update 2 and Results (3-1)

This is my second follow-up to my previous two posts which were about predicting NBA games with an algorithm [http://viksalgorithms.blogspot.com/2012/05/predicting-nba-finals- with-r.html], and my first update to the algorithm [http://viksalgorithms.blogspot.com/2012/06/predicting-nba-playoff- games-results.html]. The algorithm’s record is now

machine-learning

Predicting NBA Playoff Games - Results and Update 1

Game Results I recently made a post about developing an algorithm [http://viksalgorithms.blogspot.com/2012/05/predicting-nba-finals- with-r.html] to predict the NBA playoffs, and I concluded with 2 predictions. Although Miami beat the Celtics to make my algorithm 1-0 in terms of predictions, it fell to 1-1 when

predict

Predicting the NBA Finals with R

This is the initial post about the algorithm. See updates 1 [http://viksalgorithms.blogspot.com/2012/06/predicting-nba-playoff-games- results.html], 2 [http://viksalgorithms.blogspot.com/2012/06/nba-playoff- predictions-update-2-and.html], and 3 [http://viksalgorithms.blogspot.com/2012/06/nba-playoff-predictions- update-3-4-2.html] for more. The algorithm is currently 4-2 in the

zips

Mapping US Radiation Levels in R

I have posted previously about the open data available on Socrata (https://opendata.socrata.com/), and I was looking at the site again today when I stumbled upon a listing of levels of various radioactive isotopes by US city and state. The data is available at https://opendata.socrata.com/

packages

Loading and/or Installing Packages Programmatically

In R, the traditional way to load packages can sometimes lead to situations where several lines of code need to be written just to load packages. These lines can cause errors if the packages are not installed, and can also be hard to maintain, particularly during deployment. Fortunately, there is

foreach

Monitoring Progress Inside a Foreach Loop

The foreach package for R is excellent, and allows for code to easily be run in parallel. One problem with foreach is that it creates new RScript instances for each iteration of the loop, which prevents status messages from being logged to the console output. This is particularly frustrating during

windows

Using LaTeX, R, and Sweave to Create Reports in Windows

LaTeX [http://www.latex-project.org/] is a typesetting system that can easily be used to create reports and scientific articles, and has excellent formatting options for displaying code and mathematical formulas. Sweave [http://www.statistik.lmu.de/%7Eleisch/Sweave/] is a package in base R that can execute R code

foreach

Parallel R Model Prediction Building and Analytics

Modifying R code to run in parallel can lead to huge performance gains. Although a significant amount of code can easily be run in parallel, there are some learning techniques, such as the Support Vector Machine, that cannot be easily parallelized. However, there is an often overlooked way to speed

data-analysis

Analyzing US Government Contract Awards in R

As I was exploring open data sources, I came across USA spending [http://usaspending.gov/]. This site contains information on US government contract awards and other disbursements, such as grants and loans. In this post, we will look at data on contracts awarded in the state of Maryland in the

regression

R Regression Diagnostics Part 1

Linear regression can be a fast and powerful tool to model complex phenomena. However, it makes several assumptions about your data, and quickly breaks down when these assumptions, such as the assumption that a linear relationship exists between the predictors and the dependent variable, break down. In this post, I

finance

Analyzing Federal Bailout Recipients in R

I was searching for open data recently, and stumbled on Socrata [http://opendata.socrata.com/]. Socrata has a lot of interesting data sets, and while I was browsing around, I found a data set on federal bailout recipients. Here [http://opendata.socrata.com/Government/Bailout-Recipients/gbdy-vjgr] is the data set.

machine-learning

Intro to Ensemble Learning in R

Introduction This post incorporates parts of yesterday’s post [http://viksalgorithms.blogspot.com/2012/01/build-your-own-bagging- function-in-r.html] about bagging. If you are unfamiliar with bagging, I suggest that you read it before continuing with this article. I would like to give a basic overview of ensemble learning. Ensemble learning

r

Improve Predictive Performance in R with Bagging

Bagging, aka bootstrap aggregation, is a relatively simple way to increase the power of a predictive statistical model by taking multiple random samples(with replacement) from your training data set, and using each of these samples to construct a separate model and separate predictions for your test set. These predictions