How would you explain Federated Learning in simple layman terms for a non-STEM person? What are the main ideas behind Federated Learning?
$\begingroup$
$\endgroup$
2
asked Feb 17, 2021 at 14:06
-
$\begingroup$ is it just similar to distributed learning? $\endgroup$– Dan D.Feb 18, 2021 at 9:19
-
1$\begingroup$ @datdinhquoc, it would be a subtype of distributed learning. $\endgroup$– CorruptedHeapScapeGoatFeb 18, 2021 at 15:12
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
The analogy is to a federal system of government. In a federation, smaller pieces follow the direction of a higher piece. In federated machine learning, you give your data for processing to the higher machine. The federation in this analogy is a collection of smaller computers. The central computer breaks up your data and gives portions of it to each smaller computer. When those computers are done they return the results and the central computer reassembles them into a single model.
The main idea is distributed processing. Some benefits are:
- Cost: it may be cheaper to operate multiple inexpensive computers rather than fewer more expensive computers.
- Privacy: if this is sensitive data like healthcare records then perhaps you don't want all of one person's data in a single place where the wrong person can grab it.
answered Feb 17, 2021 at 14:58
$\begingroup$
$\endgroup$
I think the answer given my @CorruptedHeadScapeGoat is very good.
If I can, I would like to offer an example I like to use, and it's that of a mobile phone.
If you imagine a phone with a next word predictor model running on it, and it has been training on all the words you have been typing into the phone, getting better as it receives and training on more of your data. The model on your device (the local model) sends updated back to the global model which is retrained not only on your device's information, but on the information of all the devices out there associated with this system, to refine the global model, and then the refined global model is sent back to the local device.
Generally, typically, a system would send the data back to the global model for the global model to retrain on the new data.
Of course, you won't want all the data you have been typing into your phone to be sent to some central server; that is your private information. So, when it is time to send updates back to the global model, the system will just send the updated local model back, and not the data.
This means that the local devices will keep it's data safe, and private. And this is of course one of the main benefits of the FL.
This idea of sending local models back to the global models and the updates being sent back to the local model is very much an iterative process, and will happen at either periodic or aperiodic intervals - depending on whether the device is able to update or not (which could be due to device failures, communication issues etc.).
