Forgive me, Gnu, for the sins I am about to commit.

I have a plain old text file, whose contents are valid xml. It is my task to convert it into some sort of PowerPoint monstrosity.

Forgive me for embarking on this path. Forgive also those who have asked for said monstrosity, as they may know no better. Give me the strength to show them the light you have given the world. Indeed, thank you for showing me the way, that I decided initially to put my data into plain text, knowing that it could later be converted to anything.

I do not know if it is a larger sin, or some sort of compensation, that my current plan of attack is to first convert my file into an odp file. Or that I will be doing as much of the conversion as I can on a Linux system.

In the name of Stallman, Raymond, and Torvalds (and the many eyes I know not), amen.

This post is slightly more tech-heavy than most of my recent posts. If that's not your thing, feel free to move on now.

[Update 20091223: NEVERMIND. I got it sorted. The url you re-direct a user to with OAuth doesn't need extra OAuth headers. So you really could just use webbrowser.open(). My bad.]

Suppose I want to interact with the twitter API via some python code running on my personal computer. Suppose, for grins, that instead of using the Basic Authentication I'd rather try OAuth (even though it's all running locally...). Part of the flow of OAuth authentication is that my script is supposed to direct the user to an address at twitter (oauth/authorize), with some OAuth specification headers in the HTTP request for that address (I hope I'm saying things within a few shades of correctly). Well, python provides the webbrowser module which should open up a url in the users browser of choice. And it does, pretty easily (based on my 1-test sample). The problem is that for the OAuth dealings, I'm supposed to pass additional HTTP headers, and I can't figure out how to do that with the webbrowser module. I tried creating a Request object, from the urllib2 module. If I were just making a url request using this library, I could make the Request object, with the extra headers, and things would go fine. But the webbrowser.open() method seems to want its url parameter to be a string, not a Request object.

So... how am I supposed to do this? Or am I not supposed to do this?

Am I supposed to use some other existing python based browser? How is the user supposed to feel like I'm not still in the middle of the authentication process? I mean, if my script displays a webpage using some graphical widget, and waits for the user's input, then I could just be grabbing their username and password while they log in to twitter, no? The point of having the user go to twitter and get a pin is that the user then tells me that pin. I don't put myself in the middle and grab the pin (or their username/password) somehow.

Does any of this make sense? Can somebody point me at a solution? Existing code that solves this problem?

Leonhard Euler's Flying Circus!

That will be the home of the little Python/Sage learning group I mentioned recently.

Now, let's see if this thing gets off the ground...

I recently ran across projecteuler.net, a collection of intriguing programming puzzles with a mathematical bent. Before long, I had decided that it would be fun to get a group of my fellow UVA math grad students together to work through these problems. I've wanted to learn python for a while now, and thought perhaps others might as well, and that using the problems from projecteuler.net would be fun. So I sent out an email and have gotten several others who would like to join me, which is pretty encouraging. We're going to start after the semester ends, which is just a few weeks away.

When I was thinking about the group initially, I thought perhaps we'd organize some meeting time and talk about our code. But then do we print out our code and pass it around? Write it on the chalkboard? Perhaps bring in thumb drives and a laptop and projector and present out code to each other? And then I thought maybe just putting all of the code up on a group-run blog would be the best idea. We don't have to worry about organizing meetings, people can look at anything on their own time, and, for what it's worth, our work would be out in the wild for anybody to see.

My question is... can you think of a clever name for such a blog (one that you'd happily let us use)? My first thought, something like "Let's Learn Python", or so, isn't hugely fun. I feel like there's potential using π instead of "py" in "python"...

Once we get going, you can expect to see a link.

Update 20090513: Here it is: Leonhard Euler's Flying Circus!

I don't spend a lot of time visiting the actual pages for many of the blogs I follow, since I get all (or at least, most, and the main portion) of the content from their rss/atom feeds. Recently though, one of the feeds I had indicated that the author was quitting. For whatever reason (perhaps because it mentioned having the world's best math blogroll), I was inspired to visit the actual page, instead of just removing the feed from my list (or doing nothing).

The feed was from Vlorbik on Math Ed. Upon visiting the page, I found that Vlorbik kept a pretty substantial blogroll of math blogs. Liking to be in the know, I figured I might subscribe to some. Of course, I already probably do subscribe to some (and author some :)), but there are surely plenty there that I don't subscribe to. And perhaps some of them are ones I would like to follow. But I didn't want to click each link, load each page, find it's feed, and add it to Google Reader. I'm pretty lazy, and my computer would slow down a bit and frustrate me.

This evening, though, I decided to see if I could write a script to grab the rss/atom feeds for any or all of the linked pages in the blogroll. I had a great time doing so. Remembering some fun pattern matching variables in perl(like \$' and \$& (the \ there only because of how I'm doing LaTeX in Blogger)), and using curl to grab the pages... good times. Then some reformatting of appropriate strings, and out pops an OPML file. Handy, because that's what Google Reader expects if you want to import a bunch of feeds. I've played with similar things before.

Anyway, the long and short of it is, I thought perhaps other people might find this OPML file helpful. Blogger won't let me upload anything besides pictures (where's my damn GDrive?), so the file is currently (as of this writing) on my UVA personal page, here. If you'd like to blindly add these feeds to your feed reader, and then trim them down individually based on content or whatever, I encourage you to do so. The only reader I've used is Google's, so I'll give some instructions for that.

The first step is to download my OPML file, and save it somewhere convenient (you only need it temporarily on your computer). In Reader, at the bottom of the left-hand pane is the 'Manage Subscriptions' link. Click on that, and then the 'Import/Export' link at the top of the settings page that pops up. In the file upload form where it says 'Select an OPML file to upload', pick the file out from wherever you saved it, and then click 'Upload'. Wait patiently as Google imports the new feeds (it really doesn't take that long, though it might take longer for news items to start flowing in). It'll send you back to the main settings page, so click 'Back to Google Reader' to start reading. You'll notice that the feeds all show up in a folder in your subscriptions panel, called 'vlorbik' (if you already have such a folder, you might modify my OPML file before upload... I should have told you that earlier). If you already subscribed to one of the feeds, it won't mess anything up, and they won't show up as duplicates in your news stream. Of course, when making this file I grabbed the atom files, where available, so if you are subscribed to the rss feed (as I am, in many cases), then you will have duplicates. But whatever, I'll let you sort out your own subscription list.

So, with this success, I feel like perhaps I should visit actual pages (instead of just watching the news stream go by in reader) more often. Perhaps find some other blogrolls?

Anyway, enjoy. Sorry, Vlorbik, that I only started getting to know you on your way out.

Recently I have been trying to get OpenID working on a website. I want to use the JanRain OpenID PHP library, and it looked like the EasyOpenID extension to it would make things easier, so I wanted to use that as well.

My first few attempts with them seemed to have various setup issues (the blame for which rests solely with me, I concede), so I decided to remove everything and start fresh. I remembered at some point one of the issues being about getting the proper include path, so the first thing I did was put all of the .php files in the same directory. I also re-did the 'include' and 'require_once' lines to accommodate this, and removed the initial bits of the example OpenID consumer code that set up the php include path. I figured this would at least eliminate one of the problems, and if I got it all working from here, it should be feasible to undo this step.

So, the index.php loaded correctly and asked for my OpenID url. I entered one, and was sent to try_auth.php, where after a few seconds I was presented with a "500 Internal Server Error". Hmm. This is code other people have written and published. People who I expect blow me out of the water, programming-wise, so something strange is going on. Nevertheless, I tried tracking down the error. The only way I knew to do this was to put lots of error_log lines into the program to trace execution, and update my php.ini on the server to write errors to a log file. Reloading the try_auth.php file (with my url specified as a GET parameter) I'd still get the 500 error, and now I could track what was going on by also reloading the error log file. Slow and tedious, but it seemed to work.

When I finally tracked down the last line of the code that was executing, I was fairly surprised where it was. It was in JanRain's BigMath.php, and the actual line that was failing was a call to bcpowmod, a built-in PHP function (part of the encryption/decryption process, if I understand correctly). Since bcpowmod is only available in newer versions of PHP, BigMath.php has it's own powmod function, which gets called if bcpowmod is not available. So I changed the code to skip bcpowmod, and use the powmod function in BigMath. Still died. The powmod function is basically a single while loop, so I stuck a counter in to see how much it was looping. Somewhere among 500-600 iterations of the loop, the code would fail. The precise number of times was different each time, but it was always somewhere in that range. In the process I found the bcpowmod was also getting called at an earlier stage, where it finished without issue.

Ok, take a step back. Maybe it was something screwy with the OpenID provider I had picked. So I tried another one. No dice. When I finally decided to have some friends try, they also had the same error, and looking at the error log, it was in the same place. Ah, maybe wrap the offending code in a try/catch block. No dice. No exception thrown.

Well, maybe there was actually something strange with bcpowmod. So I had the code print out the values it was calling bcpowmod with (nice long integers) in the error log. I copied those into a little local script whose only job was to call bcpowmod. Worked fine. Ok, upload that to the server. Worked fine there. Hmm.

Perhaps the server was cutting the script off at a certain amount of time. I seem to recall, in reading Google App Engine documentation (not what I'm using, but still), that code had only a few seconds to return before it was interrupted, for performance reasons. Perhaps that's how the server I'm on is set up. So I found that php.ini contains those sort of limiting parameters, both execution time and memory usage. Comparing with my local php.ini I noticed the memory usage allowance was a little lower on the server, so I tried temporarily up-ing it. Didn't work. The execution time allowance was set at the default 300 (seconds), and the script was dying after about 6, so I figured that wasn't it. All the same, to test it, I changed the allowed time to about 5 seconds, to see what would happen when the script was forced to quit early. For grins, I also told the script to pause for a few seconds using a usleep() call. This gave me a different error message, so I decided that wasn't the problem.

So, now I'm out of ideas. I contacted the hosting company by submitting a support request online. Tracking the work log, it looks like they've taken a look (at something, my notice at least, hopefully the issue itself), but it's now been 3 days (2 since they last checked it out). Makes me feel like this is at least a little bit of a justifiable puzzler.

Anybody have any ideas? If this gets resolved sometime, I'll let you know how. Perhaps it's just time to start over fresh? Maybe without the EasyOpenID bit? Just get something, anything, working?

I'm not terribly impressed with the trend reporting in Google Reader. It doesn't seem to match well with how I use Reader. I'd like to have a somewhat better idea which of my feeds I actually read the most. Even more so, I'd like to know which feeds I don't really care for, so I can remove them, or at least reorganize my lists.

With that in mind, I sat down this morning (and into the afternoon) with a couple existing scripts, dive into greasemonkey, and firebug (awesome), and cranked out 'Reader Rater'. Clever, huh? It's a greasemonkey script, and it seems to be an ok first go at something. The script is here, and accompanying (brief) documentation here.

Let me know if you try it, and how it goes. If you find any bugs, or have any suggestions (easy to implement ones, please :)), please leave a comment below.

[While I was looking up references for this post, I found this script, and at first glance it looks pretty nice.]