Generate unique ID for URL

Tue Nov 13 22:38:50 EST 2012

In article <1ce88f36-bfc7-4a55-89f8-70d1645d27ad at googlegroups.com>,
 Richard <richardbp at gmail.com> wrote:
> So the use case - I'm storing webpages on disk and want a quick retrieval 
> system based on URL. 
> I can't store the files in a single directory because of OS limitations so 
> have been using a sub folder structure.
> For example to store data at URL "abc": a/b/c/index.html
> This data is also viewed locally through a web app.
>> If you can suggest a better approach I would welcome it. 

Ah, so basically, you're reinventing Varnish?
Maybe do what Varnish (and MongoDB, and a few other things) do? Bypass 
the file system entirely. Juar mmap() a chunk of memory large enough to 
hold everything and let the OS figure out how to page things to disk.