Questions tagged [distributed-computing]
The distributed-computing tag has no usage guidance.
6
questions
0
votes
0answers
32 views
Control Strategy for Multi-Agent Systems
I am working on a smart grid system which can be modeled with multiple agents interacting with each other.
The agents are physically coupled with non-linearities, the action of one agent has a direct ...
1
vote
1answer
223 views
Why can't we train neural networks in a peer-to-peer manner?
I have recently been exposed to the concept of decentralized applications,
I know that neural networks require a lot of parallel computing infra for training.
What are the technical difficulties one ...
1
vote
0answers
26 views
Do I need to maintain a separate population in each distributed environment when implementing PBT in a MARL context?
I have questions regarding on how to implement PBT as described in Algorithm 1 (on page 5) in the paper, Population Based Training of Neural Networks to train agents in a MARL (multi-agent ...
5
votes
1answer
192 views
Why do we average gradients and not loss in distributed training?
I'm running some distributed trainings in Tensorflow with Horovod. It runs training separately on multiple workers, each of which uses the same weights and does forward pass on unique data. Computed ...
2
votes
1answer
64 views
In how few updates can a multi layer neural net be trained?
A single iteration of gradient descent can be parallelised across many worker nodes. We simple split the training set across the worker nodes, pass the parameters to each worker, each worker computes ...
4
votes
2answers
695 views
How can we use crowdsourcing for deep learning?
Most companies dealing with deep learning (automotive - Comma.ai, Mobileye, various automakers, etc.) do collect large amounts of data to learn from and then use lots of computational power to train a ...
