First, this thesis presents FACTR, a force-aware imitation learning framework that combines a low-cost bilateral teleoperation system with a force-attending curriculum, improving generalization to unseen objects in contact-rich tasks. Second, it presents DexWild, a scalable framework that uses co-training on human and robot demonstrations to improve generalization to unseen environments while reducing robot-specific data requirements and supporting transfer across embodiments. Third, it presents Deep Reactive Policy (DRP), a simulation-trained framework for reactive motion generation that transfers zero-shot to the real world and achieves robust performance in complex dynamic environments. Together, these results show leveraging various sources and modalities of data leads to policies exhibiting different modes of generalization.