How AI feedback replaces human labels: Constitutional AI's critique-and-revision and RL phases, what goes in a constitution, AI judges, and RLAIF vs RLHF.