Federated Learning

Federated learning allows models to complete training updates locally on the data, uploading only gradients rather than raw data

Federated learning allows multiple agencies to jointly train a model without sharing raw data, with each agency's data remaining local and only model gradients being exchanged.

Federated learning was proposed by Google in 2016, keeping data local and uploading only model gradients

DP-SGD is a mechanism for achieving differential privacy in stochastic gradient descent, implemented by clipping per-sample gradients and then adding Gaussian noise.

Privacy-preserving technologies such as federated learning and on-device inference allow models to process data on the user's device, avoiding the upload of raw personal data to the cloud.

Federated learning was proposed by Google in 2016, originally to address privacy issues in model training on mobile devices.

Federated learning faces challenges including low communication efficiency, convergence difficulties caused by data heterogeneity, and gradient leakage attacks.

Differential privacy makes it impossible for attackers to reverse-engineer any individual user's data from the output by adding mathematically calibrated random noise to training data or model outputs.

Google has applied Federated Learning technology to Gboard keyboard prediction.

The degradation of explanation quality was identified as a threshold effect closely related to the strength of client heterogeneity, with a critical point beyond which explanations are no longer trustworthy.

Heterogeneity in federated learning refers to the degree to which each client's local data distribution deviates from the global distribution, commonly quantified with Dirichlet distribution parameters, label distribution divergence (such as KL divergence), or feature distances between clients.

The higher the heterogeneity, the harder it is for the global model obtained through federated aggregation to behave consistently across clients, and the more inconsistent the gradient directions, which is a potential mechanism by which explanation maps may become distorted.

In extremely heterogeneous environments, one should be cautious about relying on the explanation outputs of federated models, because the explanations may mislead analysts in judging the basis for detection.

The relationship between federated training and explanation quality is non-monotonic: when the data distribution is close to IID or moderately heterogeneous, the explanation faithfulness of federated models can match or even exceed the centralized baseline; but under extreme heterogeneity, explanations collapse entirely.

The study provides practical guidance: assess the level of client heterogeneity, verify explanation faithfulness before deployment, and establish continuous monitoring mechanisms.

Federated learning itself does not necessarily harm explanation quality, but differences in data distribution are a core risk factor.

In federated learning, secure aggregation allows the server to compute the sum of the local model updates of N clients without leaking any individual client's update information (beyond the sum).

In federated learning and asynchronous distributed optimization, gradient information may be sparse or delayed, in which case, beyond focusing on bias, the impact of feedback frequency on stability also needs to be rigorously assessed.

If Proton wants to embed AI in its products, it can often only take paths such as local inference, client-side processing or privacy-enhancing technologies (such as federated learning and homomorphic encryption), and these approaches still lag cloud-based large models in cost and effectiveness.

Addressing the representativeness of training data requires techniques such as federated learning, data augmentation and domain adaptation, as well as external validation on diverse population cohorts.