Federated Learning: How It Works, Why It Matters, and Real-World Applications

Key takeaways

• Federated learning is a distributed machine learning approach in which decentralized devices, servers, or organizations collaboratively train a shared artificial intelligence (AI) model without transferring sensitive data.

• Federated learning applications range from mobile apps and health care to industrial Internet of Things (IoT), finance, and smart cities.

• The Flower federated learning framework is an open-source way of training ML models that unifies federated learning with analytics and evaluation.

Federated learning works by repeatedly sharing a global AI model with participating devices or organizations to train on their local data, combining the model updates they return, and iteratively improving the model. Explore how federated learning works, the benefits it offers over traditional machine learning, and real-world applications.

If you’re ready to start building expertise in machine learning, consider enrolling in the IBM Machine Learning Professional Certificate. You’ll have the opportunity to learn how to compare and contrast different machine learning algorithms to build machine learning models in as little as three months. Upon completion, you’ll earn a career certificate to add to your resume.

What is federated learning?

Federated learning is a distributed approach to machine learning (ML), where several decentralized devices, servers, or organizations collectively train a shared AI model with local data that never leaves its original location. Each participating entity trains the AI model with its own data set and shares only the updated model with a central server, which aggregates updates from all entities to improve the global model. This allows you to train AI models on large, diverse data sets from different devices while ensuring sensitive information doesn’t leave the device.

For example, through federated learning, a group of hospitals can collaborate to create a medical handbook without revealing patients’ personal information. Each hospital collects, analyzes, and learns from its own data and shares only what it has learned. This way, you update the handbook solely based on the findings, without revealing patients’ sensitive information [1].

What is the difference between ML and federated learning?

In traditional machine learning, you’ll store raw data collected from various sources in a single, central repository, and train the ML model on the aggregated data. However, this may lead to the collection and storage of personally identifiable information (PII) and create vulnerabilities due to single-point failures, increasing data privacy and security risks.

Federated learning, with a decentralized infrastructure that brings model training to the data source instead of the other way around, maintains data privacy and security by only sharing model updates with the central server. This allows federated learning to preserve sensitive information on the original device or server while learning from diverse data sets distributed across multiple devices [2].

How does federated learning work?

Federated learning is a cyclic process in which a central server coordinates model distribution and aggregation as multiple participating entities contribute to model training with their own data. You can break down the workflow as follows [3]:

1. Initialization

The central server initializes a global ML model using a general data set and distributes a copy of this model, including relevant details such as model parameters, to specific client nodes (smartphones or organizational servers).

2. Local model training

Upon receiving the global model and parameters, each participating node trains the model using its local data set. During training, incremental updates to model performance, known as gradients, take place as the ML model completes the specified number of epochs. Upon completion, the client node encrypts the updated model parameters or gradients and sends them back to the central server, transmitting only what the model learned rather than the raw data.

3. Global model aggregation and update

The central server aggregates all updates from different client nodes using a method such as federated averaging, which computes a weighted average of all updates. It then combines the updates to create a new version of the global model based on the cumulative learning of all participating nodes.

4. Iteration

The server then sends the new model to the client nodes for further training, and the entire process repeats until the global model achieves the desired level of performance and accuracy across all client nodes and diverse data sets.

What are types of federated learning?

The federated learning type you employ will depend on the distribution of your data, the type of client nodes, and how your participants collaborate. Explore the different types of federated learning [4].

Horizontal vs. vertical

In horizontal federated learning, the data set contains the same features but different user samples. For example, multiple hospitals might collaborate to build a diagnostic model that collects the same types of patient data, such as age, symptoms, and lab results, but their patient populations differ.

In vertical federated learning, client nodes share data for the same user base but for different features. For example, a bank and an e-commerce retailer might have data on the same user, but the data types differ (financial information versus purchase history), which they can use to deliver more targeted recommendations or even detect fraud.

Cross-device vs. cross-silo

Cross-device federated learning involves millions of edge devices, such as smartphones, with unreliable network connectivity and low computing power, contributing small amounts of data. Consider, for example, a smart keyboard that collects typing data across a large number of users to improve word suggestions or autocorrect features.

In cross-silo federated learning, a limited number of participants, such as banks, hospitals, or enterprises, with more stable connectivity and significant computational resources, contribute large amounts of data. For example, several hospitals can collaborate to build an AI model for detecting cancer, with each hospital training the model locally with its own data while retaining patients’ PII.

Federated transfer learning

When both the user sample and features differ, you can use federated transfer learning to adapt a model trained on one data set to another related but different data set. Federated transfer learning is beneficial when participants with non-overlapping data sets want to apply knowledge from one domain to another domain. For example, a small clinic may adapt a large hospital’s disease-detection model and train it on its own data to diagnose a specific disease.

Why does federated learning matter?

Federated learning is important because of the benefits it offers in terms of data privacy, regulatory compliance, and cost-effectiveness [2].

• Improved data privacy and security: By keeping raw data localized on source devices, federated learning can drastically minimize the risk of sensitive information exposure during transfer, helping prevent data privacy breaches and helping organizations comply with data protection regulations. By encrypting model updates, federated learning also helps ensure data security.

• Collaborative improvement: By enabling organizations to contribute without transferring or centralizing proprietary or sensitive data, federated learning helps ensure that institutions with stringent data regulations, such as hospitals and banks, can still collaborate to build accurate AI models while retaining ownership of their data.

• Enhanced compliance with data regulations: Since federated learning minimizes data movement by keeping data local, it can help organizations comply with data residency requirements set by regulations such as the General Data Protection Regulation (GDPR) and the Health Insurance Portability and Accountability Act (HIPAA), which are crucial in industries such as finance and health care, that handle sensitive data.

• Increased efficiency and reduced costs: By transferring only model updates rather than large amounts of raw data, federated learning can reduce latency and bandwidth costs associated with training AI models, especially across edge devices.

• Scalability: Since federated learning can train ML models across many devices, it can enhance the scalability of edge computing devices that require distributed training across limited resources.

What are real-world federated learning applications?

While traditional machine learning models often involve simpler implementations, federated learning is increasingly adopted by industries from health care to urban management to address data privacy concerns, collaborate across highly regulated industries, and reduce bandwidth and resource costs. Consider these real-world applications of federated learning [1].

Mobile applications

Federated learning helps mobile AI applications learn from user behavior data generated across devices like smartphones and tablets without transmitting sensitive information. Federated learning can improve the performance and functionality of mobile apps using keyboard text prediction, voice recognition, and personalized services while maintaining data privacy by aggregating model updates across millions of devices to gradually improve the global model.

Health care

The health care industry often heavily regulates patient data and silos it across hospitals and institutions, making it difficult for hospitals to develop AI prediction models based solely on their patients' individual medical records. Using federated learning, multiple hospitals can collectively train a predictive model without compromising their patients’ sensitive information. This helps ensure compliance with regulations while improving disease prediction and diagnosis, personalized treatment strategies, biomedical image analysis, and drug discovery.

Read more: What Is Machine Learning in Health Care?

Industrial IoT

In industries like energy and manufacturing, data generated by IoT devices, such as sensors, cameras, and smart machines, can help develop and train AI models on the edge with localized data. By keeping proprietary information within factory grounds, federated learning from edge devices can help improve the performance of predictive maintenance, anomaly detection, and quality control models, enabling real-time analytics while reducing latency, communication costs, and bandwidth usage.

Finance

Banking and financial services institutions often silo sensitive transaction data due to stringent data protection regulations such as GDPR. By leveraging federated learning, financial institutions can collaborate across multiple branches and organizations to train AI models with encrypted financial records to improve fraud detection, credit risk assessment, and personalized investment advice. This helps ensure confidential customer data remains secure while enhancing the accuracy of predictions by diversifying data sets.

Smart cities

IoT sensors and other urban infrastructure devices, such as traffic management systems, fitted across cities can generate large amounts of decentralized data. Combining this data with federated learning can help collect insights for efficient traffic route optimization, resource allocation, environmental monitoring, and emergency response while keeping citizens’ data private.

What is the Flower federated learning framework?

The Flower framework for federated learning is an open-source collaborative approach that simplifies ML model training by consolidating federated learning, analytics, and evaluation. The framework is highly scalable across numerous clients and compatible with most ML workflows, programming languages, and operating systems [5].

What are challenges and limitations of federated learning?

Implementing federated learning requires balancing trade-offs between data security, communication overhead, and data heterogeneity with model accuracy and performance. Learn more about the challenges of federated learning and how to address them [1].

• Security risks: Despite the benefits of federated learning in securing private data, it’s not immune to all adversarial attacks. For example, data poisoning attacks can corrupt ML models by injecting malicious data during training, and inference attacks can expose sensitive information if cybercriminals intercept models when sharing updates. Mitigating these risks requires robust security measures such as anomaly detection, differential privacy, and secure aggregation, though this may increase computational costs.

• Communication bottlenecks: Although federated learning reduces raw data transfer, frequent exchange of model updates between the clients and server can result in bandwidth bottlenecks, especially when dealing with a large number of client nodes, resource constraints, and unstable networks. Strategies such as data or model compression and selective update sharing can help with these issues.

• Heterogeneity: Data for federated learning is collected across diverse decentralized devices and environments, so data distribution and computational capabilities vary across clients. This non-independent and identically distributed (non-IID) data can introduce bias and variability into the model, reduce the global model’s accuracy across all clients, and slow model convergence. You need to implement adaptive training, algorithms such as FedProx, and weighted aggregation techniques to overcome these challenges.

Empower your growth with curated learning resources

If you’re interested in expert guidance and no-fluff tips to help you build your skills, subscribe to our YouTube channel. You can also explore free learning resources, emerging topics, and career insights related to machine learning and AI:

• Read our Career Chat issue: 3 Machine Learning Projects to Build Your Skills and Portfolio

• Plan your learning journey: Machine Learning Roadmap: Beginner to Expert

• Hear from an expert: 6 Questions With a Google AI Research Director

Accelerate your career growth with a Coursera Plus subscription. When you enroll in either the monthly or annual option, you’ll get access to over 10,000 courses.

Build job-ready skills with Coursera Plus

Start 7-day free trial

• 
• 
• 
• 

Start 7-day free trial