Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

cantas/DFS

Folders and files

NameName
Last commit message
Last commit date

Latest commit

History

27 Commits

Repository files navigation

DFS: a simple distributed file system

DFS is a distributed file system writtent in Python for which main parts are:

  • a nameserver (which hold mappings betweens directories names & file servers)
  • a lockserver (which hold locks about files!)
  • fileservers (which are responsibles of distributing the files)
  • a client API (which work more or less like the standard API for files)

Features

  • each part are independant:
    • you can develop each part in whatever language you want
    • you can modify a part without breaking other parts
  • the nameserver automagically discover fileservers (they just have to contact the nameserver at startup)
  • use a REST api
  • automatic locking of files when they're open in write mode
  • easily extendable (adding more servers, using servers for replication, ...)
  • dead simple configuration files (five lines of JSON at most)
  • resistant to failure (you can kill -9 a {file,lock,name}server, it will restart in the same state as when it was killed)
  • use an upload/download model (so it's more designed for big files, which are just read or just wrote, like in MapReduce for instance, but it doesn't support the append operation of GFS)
  • (not yet but soon) file caching on client side (the directories/servers pairs are already cached)

Requirements

  • web.py

Design

I'll make a report to explain the design and my choices as soon as possible.

How to use?

There's no documentation for now except the docstrings, also take a look at tests/ to get an idea of how to run the dfs (you'll need to tweak the fileservers config).

It doesn't aim to be used in production (lack of authentification/encryption/file permissions are the main concern obviously). It's a project for my course of distributed systems at the Trinity College of Dublin.

About

A distributed file system, in Python

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

Contributors

Languages

  • Python 72.1%
  • TeX 26.6%
  • Shell 1.3%

AltStyle によって変換されたページ (->オリジナル) /