I have an easily parallelizable workload that I would like to run on multiple computers on the same network.
Ideally I'd like it to work something like this:
* A manager process at a known location listens for incoming connections from workers.
* The manager handles the details of work assignment to online workers, storing results, networking, etc. It merely asks a loaded module for details of how to partition the workload.
* A worker process connects to the server and spins up threads according to the computer's capabilities, calling into a module in each thread to do the computation and then sending back completed results to the manager.
* Again, the worker takes care of the low lever details such as networking and threading.
The code in the loaded module (my code) should be something not much more complex than
1 2 3 4 5 6 7 8 9 10 11 12 13 14
|
//Called from manager
Workload get_workload(){
...
}
//Called from worker
Result process_workload(Workload &){
...
}
//Called from manager
void results_complete(const std::vector<Result> &){
...
}
| |
Does anyone know of a system/library/whatever that does something like this, or how I could search for it?