Hulk, Bash!

Bash Profiles One of the cool things we've had an opportunity to do at Metis was to play with our .bash_profile files. I've edited mine before but never really knew exactly what it did. This time around, I wanted to make sure that I try some more complex things…

Scraped Knees, Logging Helps

Scraping Problems BeautifulSoup for Python is a great tool for scraping but that doesn't necessarily mean everything is easy. Scraping is all about the unknown. And one of the golden rules is that you can't trust the page you're trying to scrape. You can't trust that it's telling you the…

In a Pickle

Getting Out of a Pickle with Pickle The Problem One of the problems with writing functions for scraping data is that in the process while testing you may have to do multiple calls of the scraping program. And when you are sometimes making hundreds, if not thousands, or even far…