Abstract:
Fine-tuning is crucial for adapting pretrained foundation models (FMs) to specific downstream tasks. When datasets are distributed across multiple clients due to privacy concerns, federated learning (FL) enables collaborative fine-tuning of FMs without requiring data sharing. In this talk, I will present our ongoing work addressing two key challenges in federated fine-tuning of FMs: (1) task heterogeneity, where clients focus on different tasks, and (2) model heterogeneity, where clients use different local model architectures. By aligning output distributions across clients through knowledge distillation, our approach enables effective collaboration among heterogeneous clients and improves performance on their local tasks.

Committee:
Prof. Artur Dubrawski (Chair)
Prof. Srinivasa Narasimhan
Prof. Barnabás Póczos
Mononito Goswami