We propose Curriculum-Guided Bayesian Reinforcement Learning for ROI-Constrained Bidding, which obtains superior in-distribution and out-of-distribution generalization, learning efficiency and stability.