Convergence

The search for counterfactuals can be seen as an optimization problem, where the goal is to find a point in the input space. One questions that has received surprisingly little attention is how to determine when the search has converged. In a recent paper, we have briefly discussed why it is important to consider convergence (Altmeyer et al. 2024):

One intuitive way to specify convergence is in terms of threshold probabilities: once the predicted probability $p(y^{+}|x^{\prime})$ exceeds some user-defined threshold γ such that the counterfactual is valid, we could consider the search to have converged. In the binary case, for example, convergence could be defined as $p(y^{+}|x^{\prime}) > 0.5$ in this sense. Note, however, how this can be expected to yield counterfactuals in the proximity of the decision boundary, a region characterized by high aleatoric uncertainty. In other words, counterfactuals generated in this way would generally not be plausible. To avoid this from happening, we specify convergence in terms of gradients approaching zero for all our experiments and all of our generators. This is allows us to get a cleaner read on how the different counterfactual search objectives affect counterfactual outcomes.

In the paper, we were primarily interested in benchmarking counterfactuals generated by different search objectives. In other contexts, however, it may be more appropriate to specify convergence in terms of threshold probabilities. Our package allows you to specify convergence in terms of gradients, threshold probabilities or simply in terms of the total number of iterations. In this section, we will show you how to do this.

using CounterfactualExplanations.Convergence
generator = GenericGenerator(λ=0.01)

GradientBasedGenerator(nothing, CounterfactualExplanations.Objectives.distance_l1, 0.01, false, false, Descent(0.1), NamedTuple())

Convergence in terms of gradients

As gradients approach zero, the conditions defined by the search objective and hence the generator are satisfied. We therefore refere to this type of convergece criterium as GeneratorConditionsConvergence

conv = GeneratorConditionsConvergence(gradient_tol=0.01, max_iter=1000)
ce_gen = generate_counterfactual(x, target, counterfactual_data, M, generator; convergence = conv)

CounterfactualExplanation
Convergence: ✅ after 179 steps.

Convergence in terms of threshold probabilities

In this case, the search is considered to have converged once the predicted probability $p(y^{+}|x^{\prime})$ exceeds some user-defined threshold γ such that the counterfactual is valid. We refer to this type of convergence criterium as DecisionThresholdConvergence.

conv = DecisionThresholdConvergence(decision_threshold=0.75)
ce_dec = generate_counterfactual(x, target, counterfactual_data, M, generator; convergence = conv)

CounterfactualExplanation
Convergence: ✅ after 9 steps.

Convergence in terms of the total number of iterations

In this case, the search is considered to have converged once the total number of iterations exceeds some user-defined threshold max_iter. We refer to this type of convergence criterium as MaxIterConvergence.

conv = MaxIterConvergence(max_iter=25)
ce_max = generate_counterfactual(x, target, counterfactual_data, M, generator; convergence = conv)

CounterfactualExplanation
Convergence: ✅ after 25 steps.

Comparison

plts = []
for (ce, titl) in zip([ce_gen, ce_dec, ce_max], ["Gradient Convergence", "Decision Threshold Convergence", "Max Iterations Convergence"])
    push!(plts, plot(ce; title=titl, cbar=false))
end
plot(plts..., layout=(1,3), size=(1200, 380))

References

Altmeyer, Patrick, Mojtaba Farmanbar, Arie van Deursen, and Cynthia CS Liem. 2024. “Faithful Model Explanations Through Energy-Constrained Conformal Counterfactuals.” In Proceedings of the AAAI Conference on Artificial Intelligence, 38:10829–37. 10.