Monday, January 21, 2008
Case Study: Using Haskell and HAppS for Openomy API v2.0
At Openomy we are continuously trying out new technologies and, in particular, programming languages. We've been pretty successful thus far (with some perl background scripts, a couple Ruby on Rails services, a C#/mod_mono frontend, a Java service, etc) and continue to innovate.
The latest release of our API is written in Haskell using the new HAppS framework. This turned out to be a really interesting project and one that fundamentally changed how I think about programming. I wanted to highlight some of the reasons we chose Haskell, what worked, and what didn't.
Our backend is a collection of REST-based internal services. For example, we have a "files service" which deals with the uploading/downloading/etc of actual binary file data. We have a "metadata service" which deals with the metadata about a file (name, size, tags, etc). We have a "user service" which manages user accounts, sessions, etc. Each of these is fully independent and interact with each other through normal REST service calls. The front-ends (the web site or the API) tie them together as necessary.
Because of this, the API is essentially just a transformer: from the XML our internal services produce to XML that our developers can use.
But there were many problems along the way. When we first started, we were working with HAppS v0.8.x. Quite frankly, there was a lot not to like about it at the time. There were many quirks and bugs and in general, things we just simply couldn't figure out. HAppS comes packed with a lot of features (such as an S3 library, a Facebook library, a stellar state management system, etc), most of which were totally unnecessary for us. This caused a bit of clunkiness as we figured out what we needed and how we could avoid using the rest. A lot of this comes from the fact that the HAppS documentation was (and is) absolutely NOT up to par. In my opinion, this problem is two-fold: (1) Haddock (the Haskell documentation engine) is just an odd interface and one I don't particularly like, and (2) HAppS' Haddock and other docs are almost non-existent.
However, by the time we were nearing our launch, HAppS 0.9.1 was launched and we decided to upgrade with it. There were some substantial changes, including basic things such as how to define routes for URLs. However, in the end, we were much happier spending some time getting up to date and using the new, nicer HAppS API. It ended up being much cleaner and much simpler to understand an maintain.
One thing about Haskell in general that really tripped us up initially was finding a non-code littering, easy to understand method of handling errors. After many iterations, we finally decided -- since we spent much of our time within the IO monad already -- upon using Haskell's Control.Exception. This ended up working out very well for us, and keeps our code relatively clean and concise.
Once we had everything implemented and tested, we moved towards figuring out how to deploy the project in production. We decided to do a very similar process as we did with our Java service, using Capistrano to build and deploy our service to multiple boxes. We then simply run round-robin DNS to load-balance these machines.
Since being in production, we've so far found very few issues and everything runs quickly and smoothly. Haskell and HAppS have turned out to be a very nice addition to our stable of tools and services.
The latest release of our API is written in Haskell using the new HAppS framework. This turned out to be a really interesting project and one that fundamentally changed how I think about programming. I wanted to highlight some of the reasons we chose Haskell, what worked, and what didn't.
Why Haskell/HAppS
We initially chose Haskell/HAppS for a few reasons. First, we were getting a little tired of the same old OOP languages, and wanted an entirely different way of thinking. We're geeks and we like to try new things, especially those which make us think. Haskell offers that with functional programming. But, more than just being a "cool" language, it did have advantages that made it the right tool for the job.Our backend is a collection of REST-based internal services. For example, we have a "files service" which deals with the uploading/downloading/etc of actual binary file data. We have a "metadata service" which deals with the metadata about a file (name, size, tags, etc). We have a "user service" which manages user accounts, sessions, etc. Each of these is fully independent and interact with each other through normal REST service calls. The front-ends (the web site or the API) tie them together as necessary.
Because of this, the API is essentially just a transformer: from the XML our internal services produce to XML that our developers can use.
Thoughts
Haskell's type system (and HaXml) work very nicely for our transformer, making it a great language choice for this project. It was simple to define our transformers and connect with HAppS to serve back to the client.But there were many problems along the way. When we first started, we were working with HAppS v0.8.x. Quite frankly, there was a lot not to like about it at the time. There were many quirks and bugs and in general, things we just simply couldn't figure out. HAppS comes packed with a lot of features (such as an S3 library, a Facebook library, a stellar state management system, etc), most of which were totally unnecessary for us. This caused a bit of clunkiness as we figured out what we needed and how we could avoid using the rest. A lot of this comes from the fact that the HAppS documentation was (and is) absolutely NOT up to par. In my opinion, this problem is two-fold: (1) Haddock (the Haskell documentation engine) is just an odd interface and one I don't particularly like, and (2) HAppS' Haddock and other docs are almost non-existent.
However, by the time we were nearing our launch, HAppS 0.9.1 was launched and we decided to upgrade with it. There were some substantial changes, including basic things such as how to define routes for URLs. However, in the end, we were much happier spending some time getting up to date and using the new, nicer HAppS API. It ended up being much cleaner and much simpler to understand an maintain.
One thing about Haskell in general that really tripped us up initially was finding a non-code littering, easy to understand method of handling errors. After many iterations, we finally decided -- since we spent much of our time within the IO monad already -- upon using Haskell's Control.Exception. This ended up working out very well for us, and keeps our code relatively clean and concise.
Once we had everything implemented and tested, we moved towards figuring out how to deploy the project in production. We decided to do a very similar process as we did with our Java service, using Capistrano to build and deploy our service to multiple boxes. We then simply run round-robin DNS to load-balance these machines.
Since being in production, we've so far found very few issues and everything runs quickly and smoothly. Haskell and HAppS have turned out to be a very nice addition to our stable of tools and services.
posted by Ian Sefferman at 2:34 AM
1 Comments:
Nice Stuff!
I will definitely be monitoring how state specification changes and upgrades to newer Happs will be accommodated on the field.
Post a Comment
<< Home