correl.phoenixinquis.net

projects and coding adventures


Project maintained by correl Hosted on GitHub Pages — Theme by mattgraham

Adventuring Through SICP


01 January 2015

Back in May, a coworker and I got the idea to start up a little seminar after work every couple of weeks with the plan to set aside some time to learn and discuss new ideas together, along with anyone else who cared to join us.

Learning Together

Over the past several months, we've read our way through the first three chapters of the book, watched the related video lectures, and did (most of) the exercises.

Aside from being a great excuse to unwind with friends after work (which it is!), it's proved to be a great way to get through the material. Doing a section of a chapter every couple of weeks is an easy goal to meet, and meeting up to discuss it becomes something to look forward to. We all get to enjoy a sense of accomplishment in learning stuff that can be daunting or difficult to set aside time for alone.

The best part, by far, is getting different perspectives on the material. Most of my learning tends to be solitary, so it's refreshing to do it with a group. By reviewing the different concepts together, we're able to gain insights and clarity we'd never manage on our own. Even the simplest topics can spur interesting conversations.

SICP

Our first adventure together so far has been the venerable Structure and Interpretation of Computer Programs. This book had been on my todo list for a long time, but never quite bubbled to the top. I'm glad to have the opportunity to go through it in this format, since there's plenty of time to let really get into the excercises and let the lessons sink in.

SICP was originally an introductory textbook for MIT computer programming courses. What sets it apart from most, though, is that it doesn't focus so much on learning a particular programming language (while the book does use and cover MIT Scheme) as it does on identifying and abstracting out patterns common to most programming problems. Because of that, the book is every bit as useful and illuminating as ever, especially now that functional paradigms are re-entering the spotlight and means of abstracting and composing systems are as important as ever.

What's next?

We've still got plenty of SICP left to get through. We've only just gotten through Chapter 4, section 1, which has us building a scheme interpreter in scheme, so there's plenty of fun left to be had there.

We're also staring to do some smaller, lunchtime review meetings following the evening discussions to catch up the folks that can't make it. I may also try sneaking in some smaller material, like interesting blog posts, to keep things lively.


If anyone's interested, I have the exercise work along with some notes taken during the meetings hosted online. I apologize for the lack of notes early on, I've been trying to get better at capturing memorable excerpts and conversation topics recently. I may have to put some more posts together later on summarizing what we discussed for each chapter; if and when I do, they'll be posted on the seminar website.

Getting Organized with Org Mode


25 November 2014 emacs · org-mode · themes

Org mode logo

I've been using Emacs Org mode for nearly a year now. For a while I mostly just used it to take and organize notes, but over time I've discovered it's an incredibly useful tool for managing projects and tasks, writing and publishing documents, keeping track of time and todo lists, and maintaining a journal.

Project Management

Most of what I've been using Org mode for has been breaking down large projects at work into tasks and subtasks. It's really easy to enter projects in as a hierarchy of tasks and task groupings. Using Column View, I was able to dive right into scoping them individually and reporting total estimates for each major segment of work.

Example projects org file

Because Org Mode makes building and modifying an outline structure like this so quick and easy, I usually build and modify the project org document while planning it out with my team. Once done, I then manually load that information into our issue tracker and get underway. Occasionally I'll also update tags and progress status in the org document as well as the project progresses, so I can use the same document to plan subsequent development iterations.

Organizing Notes and Code Exercises

More recently, I've been looking into various ways to get more things organized with Org mode. I've been stepping through Structure and Interpretation of Computer Programs with some other folks from work, and discovered that Org mode was an ideal fit for keeping my notes and exercise work together. The latter is neatly managed by Babel, which let me embed and edit source examples and my excercise solutions right in the org document itself, and even export them to one or more scheme files to load into my interpreter.

Exporting and Publishing Documents

Publishing my notes with org is also a breeze. I've published project plans and proposals to PDF to share with colleagues, and exported my SICP notes to html and dropped them into a site built with Jekyll. Embedding graphs and diagrams into exported documents using Graphviz, Mscgen, and PlantUML has also really helped with putting together some great project plans and documentation. A lot of great examples using those tools (and more!) can be found here.

Emacs Configuration

While learning all the cool things I could do with Org mode and Babel, it was only natural I'd end up using it to reorganize my Emacs configuration. Up until that point, I'd been managing my configuration in a single init.el file, plus a directory full of mode or purpose-specific elisp files that I'd loop through and load. Inspired primarily by the blog post, "Making Emacs Work For Me", and later by others such as Sacha Chua's Emacs configuration, I got all my configs neatly organized into a single org file that gets loaded on startup. I've found it makes it far easier to keep track of what I've got configured, and gives me a reason to document and organize things neatly now that it's living a double life as a published document on GitHub. I've still got a directory lying around with autoloaded scripts, but now it's simply reserved for tinkering and sensitive configuration.

Tracking Habits

Another great feature of Org mode that I've been taking advantage of a lot more lately is the Agenda. By defining some org files as being agenda files, Org mode can examine these files for TODO entries, scheduled tasks, deadlines and more to build out useful agenda views to get a quick handle on what needs to be done and when. While at first I started by simply syncing down my google calendars as org-files (using ical2org.awk), I've started managing TODO lists in a dedicated org file. By adding tasks to this file, scheduling them, and setting deadlines, I've been doing a much better job of keeping track of things I need to get done and (even more importantly) when I need to get them done.

Agenda view snippet

This works not only for one-shot tasks, but also habits and other repetitive tasks. It's possible to schedule a task that should be done every day, every few days, or maybe every first sunday of a month. For example, I've set up repeating tasks to write a blog post at least once a month, practice guitar every two to three days, and to do the dishes every one or two days. The agenda view can even show a small, colorized graph next to each repeating task that paints a picture of how well (or not!) I've been getting those tasks done on time.

Keeping a Journal and Tracking Work

The last thing I've been using (which I'm still getting a handle on) is using Capture to take and store notes, keep a journal, and even track time on tasks at work.

(setq org-capture-templates
      '(("j" "Journal Entry" plain
         (file+datetree "~/org/journal.org")
         "%U\n\n%?" :empty-lines-before 1)
        ("w" "Log Work Task" entry
         (file+datetree "~/org/worklog.org")
         "* TODO %^{Description}  %^g\n%?\n\nAdded: %U"
         :clock-in t
         :clock-keep t)))

(global-set-key (kbd "C-c c") 'org-capture)

(setq org-clock-persist 'history)
(org-clock-persistence-insinuate)

For my journal, I've configured a capture template that I can use to write down a new entry that will be stored with a time stamp appended into its own org file, organized under headlines by year, month and date.

For work tasks, I have another capture template configured that will log and tag a task into another org file, also organized by date, which will automatically start tracking time for that task. Once done, I can simply clock out and check the time I've spent, and can easily find it later to clock in again, add notes, or update its status. This helps me keep track of what I've gotten done during the day, keep notes on what I was doing at any point in time, and get a better idea of how long it takes me to do different types of tasks.

Conclusion

There's a lot that can be done with Org mode, and I've only just scratched the surface. The simple outline format provided by Org mode lends itself to doing all sorts of things, be it organizing notes, keeping a private or work journal, or writing a book or technical document. I've even written this blog post in Org mode! There's tons of functionality that can be built on top of it, yet the underlying format itself remains simple and easy to work with. I've never been great at keeping myself organized, but Org mode is such a delight to use that I can't help trying anyway. If it can work for me, maybe it can work for you, too!

There's tons of resources for finding new ways for using Org mode, and I'm still discovering cool things I can track and integrate with it. I definitely recommend reading through Sacha Chua's Blog, as well as posts from John Wiegley. I'm always looking for more stuff to try out. Feel free to drop me a line if you find or are using something you think is cool or useful!

Learning Functional Programming, Part One


09 April 2012 functional · python

Part One: Lambdas? In my Python?

Over the past few months, I’ve decided to take a stab at learning some functional programming. I’d been doing python for a few years (and completely falling in love with it), and so I’d been exposed to a few functional concepts it offers - primarily higher-order functions and list comprehensions, both of which allow for very clear, concise and powerful code. Since that’s where I started my journey, that’s where my post will begin as well.

Functions are objects, too

Having graduated to python from PHP and C/C++, perhaps the biggest new thing to wrap my head around (besides readable code, whitespace-as-syntax, programming being fun again, and all that), is that in python, functions (and classes!) are objects, just like anything else. They can still be defined in the usual way, but they can also be assigned, passed as arguments, even modified and replaced like any other value or object in your program.

def do_a():
    print "Doing something"

do_b = do_a

do_b()

# Prints "Doing something"

Functions themselves no longer require formal definitions, either, they can be created anonymously:

my_send = lambda person, thing: send(person.upper(), thing, subject="Check this out!")
ucase_people = map(lambda name: name.upper(), ["Joe", "Mary", "Zach"])

Abstracting behaviour

You’ll find you can now start abstracting away common idioms. For example, you probably very often find yourself looping over some list of items, performing some set of actions on them, or passing them to some other function or method:

people = ["Joe", "Chris", "Matt", "Jennifer"]
for person in people:
    u_person = person.upper()
    send(person, super_fun_thing)

Instead of that, you could have a function that takes a list as one argument, and a function to apply to each item in it as another:

def dostuff(action, things):
    result = []
    for thing in things:
        result.append(action(thing))
    return result

dostuff(send, people)

The above example is actually just a simple definition of one of the most common higher-order functions, map, which python already provides for you. Another particularly useful higher-order function is filter which, given a function that returns true of false if its criteria are met by the passed item, will return the subset of the passed list that satisfy the filtering function:

stuff = ["My notes.txt", "Matt's notes.txt",  "My music.pls"]
my_stuff = filter(lambda s: s.startswith("My "), stuff)

# my_stuff = ["My notes.txt", "My music.pls"]

List comprehensions provide a cleaner, easier to read way to perform mapping and/or filtering on a list:

stuff = ["My notes.txt", "Matt's notes.txt",  "My music.pls"]

my_stuff = [file for file in stuff if file.startswith("My ")]
# ["My notes.txt", "My music.pls"]

upper_stuff = [file.upper() for file in stuff]
# ["MY NOTES.TXT", "MATT'S NOTES.TXT", "MY MUSIC.PLS"]

music = [file.upper() for file in stuff if file.endswith(".pls")]
# ["MY MUSIC.PLS"]

Tip of the iceberg

This is just a very small taste of functional programming concepts. Later, I’ll introduce a couple of functional languages, and explain what sets them apart from object-oriented and imperative programming languages.

Transmission, RSS and XBMC


27 April 2011 bittorrent · htpc · linux · python · transmission · xbmc

I’m a huge fan of XBMC. My pc (currently running Ubuntu 10.04) has taken root in my living room, piping all my movies and tv shows straight to my HDTV.

While my pc is set up as a DVR using MythTV to record shows off my FIOS box, it tends to be a little unreliable, which can suck when it’s time to catch up on Daily Show and Colbert episodes. I’ve had Transmission set up for a while for all my torrenting needs, and I’ve even written an XBMC script to manage torrents, so I got to looking for tools to track tv show torrent rss feeds.

My first stop was TED. TED worked well enough, but would occasionally hang. Since it’s a GUI java app running in the taskbar, it would require me to dig out my mouse and break out of full screen XBMC to fiddle with it. I eventually got tired of dealing with TED and went back to prodding Myth.

Recently I’ve been itching to reliably watch my shows again, so I checked around for a simple command-line utility to track rss feeds and download torrents. Finding none, I loaded up vim and threw together a python script to handle it all for me.

I also have another, simple script from when I was using TED (or just manually downloading shows) which looks at completed torrents, compares their names with the folders in my TV directory, and moves the shows into them for XBMC to see.

A couple cron jobs and a few rss feeds later, and I’ve got all my shows automatically delivered straight to XBMC for my lazy evening viewing pleasure.

trss.py

Download

Usage:
    trss.py add <rss-url> [<recent-items>]
        Adds an RSS feed to follow
        rss-url:        Full URL to the RSS feed
        recent-items:   (Optional) number of recent items to queue
                        for downloading
    trss.py remove <index>
        Remove an RSS feed
        index:          Numeric index of the feed to remove as
                        reported by the list command
    trss.py list
        Displays a list of followed feeds

    trss.py download
        Fetch all feeds and download new items

    trss.py set [<setting> [<value>]]
        Set or view configuration settings
        Call without any arguments to list all settings and their values
        Call with a setting and no value to see the current value for that setting

        Currently, the only used setting is 'download_dir', which allows you to set
        a directory to store all retrieved torrents, such as a directory your
        torrent application watches for new downloads. If 'download_dir' is not set,
        the current directory will be used.

transmission-tv.py

#!/usr/bin/python
import os
import re

import transmissionrpc

TV_PATH = '/media/Gaia/Video/TV/'

class TVShowCollection:
	def __init__(self, path):
		self.path = path
		self.shows = os.listdir(path)
		self.patterns = [[s.lower().replace(' ', '.?'), s] for s in sorted(self.shows, key=len, reverse=True)]
	def match(self, filename):
		for pattern, show in self.patterns:
			if re.findall(pattern, filename.lower()):
				return show
		return None

def move(self, ids, location):
	"""Move torrent data to the new location."""
	self._rpc_version_warning(6)
	args = {'location': location, 'move': True}
	self._request('torrent-set-location', args, ids, True)

if float(transmissionrpc.__version__) < 0.4:
	# The move function is not present in versions 0.3 and older
	transmissionrpc.Client.move = move

collection = TVShowCollection(TV_PATH)
client = transmissionrpc.Client()

torrents = client.info()
for i, torrent in torrents.iteritems():
	status = torrent.status
	if status not in ['seeding', 'stopped']:
		continue
	show = collection.match(torrent.name)
	if show is None:
		continue
	path = '{0}{1}/'.format(TV_PATH, show)
	if torrent.downloadDir.startswith(path):
		continue
	print 'Found {0} torrent \'{1}\' in show \'{2}\', moving...'.format(status, torrent.name, show)
	result = client.move(i, path)
	if status == 'seeding':
		print 'Re-starting torrent to continue seeding'
		client.start(i)