Slum segmentation from satellite imagery holds significant promise in generating consistent global estimates of urban poverty. However, the morphological heterogeneity of informal settlements presents a major challenge, limiting the generalization of models trained on specific regions to unseen locations. To address this, we introduce a large-scale high-resolution dataset and propose GRAM (Generalized Region-Aware Mixture-of-Experts), a two-phase test-time adaptation framework that enables robust slum segmentation without labeled data from target regions. We compile a million-scale dataset of preprocessed satellite imagery from 12 cities across four continents for source training. Using this data set, GRAM employs a mixture-of-experts architecture to capture region-specific slum characteristics while learning universal features through a shared backbone. During adaptation, prediction consistency across experts filters unreliable pseudo-labels, allowing the model to generalize effectively to previously unseen regions. When tested on three African cities, GRAM outperforms state-of-the-art baselines in low-resource settings, offering a scalable and label-efficient solution for global slum mapping and data-driven urban planning.