How Many Neutrons Can You Stack Before Reality Breaks?