Ai2 tests demographic decision stability with FairStress

What changed

Ai2 reports unequal resistance to persuasion even when initial choices agree.

Source: Ai2 FairStress research announcement

Our analysis

Builder’s view: have the product owner specify legitimate decision criteria before evaluating a model. Review decision reversals and their consequences separately from an aggregate accuracy score.

Keep in mind

Its equality-centered definition is limited, not a general fairness certification.

Source: Ai2 FairStress research announcement