Improving the diversity of generated results while maintaining high visual quality remains a significant challenge in image generation tasks. Fractal Generative Models (FGMs) are efficient in generating high-quality images, but their inherent self-similarity limits the diversity of output images. To address this issue, we propose a novel approach based on the Hausdorff Dimension (HD), a widely recognized concept in fractal geometry to quantify structural complexity, which aids in enhancing the diversity of generated outputs. To incorporate HD into FGM, we propose a learnable HD estimation method that predicts HD directly from image embeddings, addressing computational cost concerns and enabling efficient integration into generative modeling. Moreover, simply introducing HD as an auxiliary loss is insufficient to enhance diversity in FGMs. To this end, during training, we adopt an HD-based loss with a momentum-driven weighting strategy to progressively optimize hyperparameters to gain best diversity without sacrificing visual quality. Besides, during inference, we employ HD-guided rejection sampling to select geometrically richer outputs. Extensive experiments on the ImageNet dataset demonstrate that our FGM-HD framework yields a 39\% improvement in output diversity, compared to the baseline fractal model, while preserving comparable image quality. To our knowledge, this is the very first work introducing the Hausdorff Dimension into FGM. Our method effectively enhances the diversity of generated outputs while offering a principled theoretical contribution to the development of fractal-based generative models.