Acoustophoresis uses sound waves to manipulate small objects in mid-air and has broad potential in various applications. However, stable multi-particle levitation remains challenging due to complex acoustic dynamics and limitations of existing models. We introduce AcoustoReinforce, a reinforcement learning-based path planner that autonomously controls the motion of multiple levitated particles. Leveraging a decentralized architecture, it learns local neural policies that generate particle trajectories independently, enabling scalable, communication-free control even in densely populated acoustic fields. To ensure physical feasibility, acoustic trapping strength is incorporated as a constraint during both training and inference, producing trajectories that are collision-free, acoustically stable, and physically realizable within real-world system constraints. Experiments on a real-world levitation platform show that AcoustoReinforce outperforms state-of-the-art planners, improving task success rates by up to 130% across diverse configurations. These results demonstrate the effectiveness of learning-based decentralized control for complex multi-object acoustophoresis in real environments.