How to effectively (but minimally) *simulate* a distributed computing environment?

Question

I am doing some personal research on Byzantine Faults. I am trying out algorithms from various research papers which present algorithms for consensus in distributed systems like this one. For this i need a small scale simulation environment of a distributed system. How do i approach this problem ?

Answer 1

We use this: http://lxc.sourceforge.net/

Linux Containers are extremely light weight virtual machines (basically zero overhead). We are using only 250 machines per physical machine, but there shouldn't be any issue going for thousands per machine.

We successfully used both LVM partitions and overlay filesystems as the system root.

• Chani: hi can you please tell me if the documentation is solid enough ?
• Šimon Tóth: @Scrooge It's very simple to use and there are a bunch of tutorials.
• Chani: can you pls tell me if i can run lxc on Damn Small Linux or Lubuntu or some similar light weight linux ?
• Šimon Tóth: @Scrooge Just use debian.
• Chani: so are there issues if i use ubuntu ?

Answer 2

Virtual machines may be the way to go, the problem is that if you setup 5 VM's on one server the response time between them will be effectively 0, which may not be true in your real environment where there could be a delay between servers.

If that is an issue (or could be) I would suggest setting up some servers on AWS or the like to test. Chances are that this could be done for less than 10ドル or 20ドル.

• vcsjones: Some VM software, like VMWare ESX allows you to simulate network latency as well. (Just something to consider).
• Zachary K: Didn't actually know that, very cool

Answer 3

I just skipped through the paper that you mention, but here I go. We often use a virtualised environment to create experimental setups that involve multiple computers. Using Hyper-V on Windows Server (or similar software) you can easily create a private, isolated network of machines that you can configure on the fly and reuse as needed.

You seem to be working on fault detection and the associated consensus reaching, so I wouldn't think that the low latency, high bandwidth that a virtualised environment would yield would be a problem. In any case, if it is a problem, you can always use a WAN simulator such as WAN Connection Emulator to simulate an arbitrarily high-latency, low-bandwidth connection between your nodes.

Answer 4

Short of virtual machines -- which are probably the way to go -- most client server software lets the same machine behave as client and server. So you might need a distributed pardagim on a single PC.