AnalyticExpectedUtilityOfBestOption for multiple comparisons

[changkun]

According to https://botorch.org/api/acquisition.html, currently, AnalyticExpectedUtilityOfBestOption only allows specifying either q=2 or q=1 with a given previous winner. This only permits pairwise comparison at a time. How to extend this to parallel evaluations, such as evaluating multiple paired comparisons at a time?

Is it possible to extend this to 4 candidates' comparisons, and how? For example, if x1, x2, x3, and x4 are estimated as four optimize_acqf candidates. 4 candidates would be able to provide C(4, 2)=6 pairwise comparison results.

[saitcakmak]

cc @ItsMrLin

[ItsMrLin]

Hi @changkun, currently analytical EUBO (i.e., AnalyticExpectedUtilityOfBestOption) is specifically designed for the q=2 use case. If you need to have q > 2, qEI (https://botorch.org/api/acquisition.html#botorch.acquisition.monte_carlo.qExpectedImprovement) is usually a useful acquisition function choice in practice.

[ItsMrLin]

Depending on the use case. For example, in the BOPE tutorial (https://botorch.org/tutorials/bope), if you want to generate multiple pairs of candidates, you can simply rerun optimize_acqf with a newly instantiated FixedSingleSampleModel, and it will produce different candidates even for the same pref_model.

[changkun]

Thanks again for the elaboration.

simply rerun optimize_acqf with a newly instantiated FixedSingleSampleModel

Does this mean the different candidates are just a random perturbation to the old paired candidates?

[ItsMrLin]

Not quite. Under the BOPE context, there is some uncertainty in what parameters X will result in what outcome Y, over which we try to learn the preference function (i.e., U = g(Y)). In this case, we are perturbing what the possible Y might be given a X, then EUBO is going to find the corresponding X for a particular deterministic outcome function Y = f(X). In that case, re-running optimize_acqf with a newly instantiated FixedSingleSampleModel is a sensible thing to do.

On the other hand, in the context of preferential BO where there is no intermediate outcome Y, and we simply want to find the maximizer of f(X), then running EUBO sequentially and update the model after every iteration is the sensible thing to do here. Or if we really wanna do batch (q > 2 and we compare all C(q, 2) pairs), doing qEI like I mentioned above is also reasonable, since another way to view this is that by making pairwise comparisons about some points, we are (indirectly) "observing" the value of those points.

[changkun]

I see. Thanks a lot for the explanation! Perhaps one more question regarding the use of PairwiseGP. How should we handle observed ties here? According to the documentation:

    In the example below, the user/decision maker has stated that they prefer
    the first item over the second item and the third item over the first item,
    generating comparisons [0, 1] and [2, 1].

However, it does not yet say anything about the case of indifference. How should we indicate indifference in y? For instance, for candidates a and b, let X = torch.tensor([a, b]). If a is indifference to b, then which Y should be the input: 1) Y=torch.tensor([[0, 1], [1, 0]]) 2) Y=torch.tensor([[0, 1]]), or 3) Y=torch.tensor([[1, 0]])?

[ItsMrLin]

The model does not support indifference/ties and the decision maker (DM) has to make a choice. Since the model assumes the user may input with errors, so for an actual indifferent choice pair, the model would expect the user just to give a random response. While very inefficient, if we have [[0, 1], [1, 0]] as the data (or even repeat for a few times), this may help create "indifference" response in the data. That being said, I'd recommend to avoid such questions and try to elicit the DM preference even for the smallest difference between a and b.