Create and manage computing tasks
|
Bas van der Vlies
2b99e0a474
Merge branch 'readme-update' into 'master'
Updated README to include dependicies See merge request surfsara/stopos!2 |
||
|---|---|---|
| doc | Initial commit | |
| aantekeningen.txt | Initial commit | |
| cleanit | Initial commit | |
| clocks.cpp | Initial commit | |
| clocks.h | Initial commit | |
| credits | Initial commit | |
| files_pool.cpp | Initial commit | |
| files_pool.h | Initial commit | |
| flatfile_pool.cpp | Initial commit | |
| flatfile_pool.h | Initial commit | |
| gdbm_pool.cpp | gdbm locking interfere with our locking so disable it | |
| gdbm_pool.h | Initial commit | |
| index.html | Initial commit | |
| LICENSE | Initial commit | |
| Make.inc | Initial commit | |
| Makefile | Initial commit | |
| makefunction | Initial commit | |
| makegdbmh | Initial commit | |
| makeit | Initial commit | |
| modulefile | Initial commit | |
| mysql_pool.cpp | Initial commit | |
| mysql_pool.h | Initial commit | |
| prenv.cpp | Initial commit | |
| protocol | Initial commit | |
| README.md | Updated README to include dependicies | |
| runtiming | Initial commit | |
| sara-get-mem-size | Initial commit | |
| sara-get-mem-size.1 | Initial commit | |
| sara-get-num-cores | Initial commit | |
| sara-get-num-cores.1 | Initial commit | |
| shellenv.cpp | Initial commit | |
| shellenv.h | Initial commit | |
| showflatfiledb.c | Initial commit | |
| split.cpp | Initial commit | |
| startthttpd | Initial commit | |
| stopos.h | Initial commit | |
| stopos.man | Initial commit | |
| stopos_key.cpp | Initial commit | |
| stopos_key.h | Initial commit | |
| stopos_pool.cpp | Initial commit | |
| stopos_pool.h | Initial commit | |
| stoposclient.cpp | Initial commit | |
| stoposdump | Initial commit | |
| stopospools | Initial commit | |
| stoposserver.cpp | Initial commit | |
| stoposzztostr.cpp | Initial commit | |
| test1 | Initial commit | |
| test2 | Initial commit | |
| test_pool.cpp | Initial commit | |
| todo | Initial commit | |
| watchit | Initial commit | |
| wrequest.cpp | Initial commit | |
| wrequest.h | Initial commit | |
| wtd.h | Initial commit | |
| wutils.cpp | Initial commit | |
| wutils.h | Initial commit | |
| zzdo | Initial commit | |
Welcome to stopos
What is stopos
Stopos is a tool to manage large number of tasks, mostly started in jobs.
First, you define a 'pool' and fill it with lines (command lines, parameters, ...). Then, tasks (usually in jobs) ask stopos for a line. Stopos takes care that lines that are committed, will not be committed again, unless the user specifically requires to re-commit lines.
Typical usage
- The user fills a pool with lines, consisting of parameters that are to be used by a program.
- In a job: as many tasks as there are cores are started in parallel.
- Each task asks stopos a line: stopos will give each task another line.
- The task executes the required command using the line as parameter.
- The task tells stopos that this line is now invalidated, it must not be committed ever again.
Stopos is server-based
Stopos is based on a client-server model. The clients runs on the compute cluster, the server runs on a server node.
Licence
Apache 2.0 2004
Download and install
Requirements
Debian:
- zlib1g-dev
- libcurl4-openssl-dev
- g++
- make
- git
- libgdbm-dev
SLES/RHEL:
- zlib-devel
- libcurl-devel
- git-core
- gcc-c++
- make
- gdbm-devel
Download
git clone https://gitlab.com/surfsara/stopos.git
cd stopos
Compile
# edit the make-script Make.inc to your taste
./makeit
After succesfull compilation, there are:
stoposclient, the client programstoposserver, the server programstoposzztostr, a program that takes on stdin a hex-encoded string (with pre-and postfix) and outputs the decoded string. This program can be useful when looking at the files on the server side.stoposdump, a script that demonstrates the use of 'stopos dump'.- a script
makefunctionthat creates a shell function calledstopos. This shell function is normally used when invoking the program. It takes care of setting environment variables. - a man page:
stopos.1which describes the usage, also containing an example. modulefile: an example of a modulefile to activate the stopos command. (Only useful if you use modules, but otherwise, it illustrates how to create a suitable alias (calledstopos)) for the C-shell- a comprehensive description of the program in
doc/description.pdf
More documentation
We have more user documentation on SURF's website.