Who Should Be Generated? Justifying Demographic Targets in Open-Ended Generation

Fairness evaluation concerns not only what a model produces, but also what its outputs ought to be compared against.

Fairness evaluation concerns not only what a model produces, but also what its outputs ought to be compared against. When a model generates "a CEO in the United States," the prompt leaves demographic realization to the model. Existing group fairness definitions assume that…