Skip to content

scraping broken SSL pages with node.js

To collect the data for courseoff.com I scrape university websites on demand. This means that I have to perform http queries to the sites and then parse the resulting pages. Usually, this is not a problem as node.js has very simple APIs for performing both http and https requests. However, some schools have old webservers which make this task trickier.

(Continued)

headphones for snowboarding

I recently bought a snowboarding helmet to get home and realize that it has cutouts in the earmuffs for headphones. Lucky for me I had a pair of headphones laying around that would be a perfect fit. Below are some pictures of the the headphones before and after my minor modifications.

(Continued)

course/off visualization – gatech Spring 2012

Since I have all this data from course/off, I figured I would make some interesting visualizations with some of it. One of the first things that comes to mind is: how popular is a given course? Without further delay, I present the glorious bubble diagram of course popularity at Georgia Tech for Spring 2012.

(Continued)

The tail of MongoDB

I recently gave a talk at MongoNYC 2011 about using MongoDB as a messaging layer. The rationale behind doing something like this would be to maintain a log of all the messages passed between two clients. While there are many different IPC schemes available, I thought it would be interesting to build something simple on top of MongoDB using the tailable cursor feature. After giving the talk, I wanted to explore the performance characteristics of such a setup.

(Continued)

re-tipping the cab

After having a brief discussion with some friends about my views on tipping I have come to give the idea of tipping some more thought. The gist of the discussion led me to realize that I had been looking at tipping from a somewhat wrong point of view and as such want to just make a few more comments on the matter and share a few random tidbits I ran across while looking into taxi cabs in NYC.

(Continued)

tipping the cab

Recently (as it happens every once in a while) I decided to re-evaluate my views on tipping and see if I needed to add or remove anything from the “will tip” pool. Sure enough, after some [not so] deep thought I decided that tipping a cab driver for a ride is one of those things I can safely remove from the required tip pool.

(Continued)

the internet, open 24/7

So just when you thought webpages were available 24/7 (or at least not when under maintenance), along comes the government to prove you wrong!

www.socialsecurity.gov/onlineservices/

Check out the left side. The “Field Office Locator” is only available during certain hours. Note, this isn’t even a chat feature or anything like that… its a simple lookup service. On the right side, there are even more hours posted for what appears to be the site in general. I’m not sure why this is. Maybe they run out of RAM on a known schedule. Anyhow… no longer will I “assume” the internet is open.

new kdm theme

For the longest time I found the default Kubuntu KDM theme … well… not very pretty. Besides having lots of “dead” space I felt that it didn’t work well at all with the default splash screen. The transition from one to the other was not very clean. It didn’t help that for Karmic the wallpaper/ksplash were updated but the kdm was not. This caused it to look even worse; so I decided to come up with a new one.

(Continued)

live assistant for ubuntu

Live assistant is an idea that I proposed at Ubuntu Developer Summit Lucid. The basic premise is to provide new Ubuntu users the same “live chat” experience that many large websites are starting to offer.

(Continued)

schedule creator – spring 2010

The Spring 2010 schedule is now available. Head over to http://schedule.shtylman.com/ to begin setting up your course schedule.

As always, if you notice any error in the information please let me know.

Enjoy :)