Issues with Acquistion Function Optimizations (optimize_acqf and LogEI)

[ohassn]

Hello,

I had wanted to discuss two topics regarding acquisition function optimizations.

1. Going from version 0.11.3 to 0.13 addresses an issue where LogEI exhibits overly exploratory behaviour. For instance, if one were to run optimize_acqf 100 times (without updating the model) and plot the candidates, there is a clear issue on version 0.11.3:

The generated candidates are entirely different, quite often at the bounds. This issue doesn't persist in version 0.13:

However, there is still the occasional outlier (which is not seen with UCB). Is there a possibility of fixing this?

2. Why does optimize_acqf perform the optimization over unnormalized bounds? I tested two cases using a SingleTaskGP:

• Defined the GP with an input_transform using Normalize, with the unnormalized bounds for optimize_acqf.

• Defined the GP without an input_transform but instead a manual min-max transformation on the inputs. Using the normalized inputs as the bounds for optimize_acqf.

The latter method is much quicker, my tests show a 5x speed boost. Is there a reason this isn't done by default?

Thank you in advance! Please let me know if you need any additional information.

[Balandat]

Can you share a bit more about your setup and ideally a repro with the code you used to generate these plots?

Going from version 0.11.3 to 0.13 addresses an issue where LogEI exhibits overly exploratory behaviour.

Great to hear. Do you happen to know what changes in particular fixed this? Mostly so that we can be on the lookout for similar issues in the future. My suspicion is that this may be a change @SebastianAment made to avoid using Boltzmann sampling on the initial candidates to avoid "overexploitative" behaviors in the random restarts.

However, there is still the occasional outlier (which is not seen with UCB). Is there a possibility of fixing this?

I would think so, but we'd first have to understand the mechanism that causes this. One obvious guess would be that as we're optimizing a nonconvex function (over a potentially high-dimensional domain) our random restart initializations miss the basin of attraction of the global optimum (or are otherwise degenerate) and so the optimization only finds a local optimum. However, this could also be some other failure of the optimization. If you provide a reproducible example (ideally with a fixed seed or set of initial conditions) we could take a look at what's going on when this failure occurs.

Why does optimize_acqf perform the optimization over unnormalized bounds?

The main reason this is done by default is that when that was not the case we've observed many users in the past struggle b/c they were not normalizing their inputs. So we opted to do this automatically for them by default.

The latter method is much quicker, my tests show a 5x speed boost. Is there a reason this isn't done by default?

Seems like the difference in speed comes from getting rid of the overhead of the transform during the optimization, but it would be good to profile this. I'd need to know more about the problem here to say something more intelligible - what dimension, how many observations, what exactly are you optimizing, etc. That said, a 5x boost seems quite substantial. @saitcakmak you have observed some slowdowns from the transforms, but IIRC not nearly as large - any thoughts on this?

[ohassn]

Do you happen to know what changes in particular fixed this?

Not really, I tried going through the release notes but I didn't find anything regarding this.

Can you share a bit more about your setup and ideally a repro with the code you used to generate these plots?

training_set.csv
acq_func.txt
acq_func2.txt

I have a 6D input space and I'm using a SingleTaskGP to optimize on an observed quantity. The first file is some data I collected while running the code. The next two files are recreating the GP state at the end of optimization based on this data and the model parameters at the time, I then evaluate optimize_acqf over 100 trials while timing it.

The second file is the regular method: unnormalized bounds, input transform for normalization), and the third file is using normalized bounds everywhere.

Context behind this work can be found in this paper. My PyTorch version is 2.6.0+cpu and BoTorch is 0.13.0.

Thank you for the responses, do let me know if anything else is needed!

[ohassn]

Hello all, just checking in to see if you managed to look into this issue!

[Balandat]

I did some profiling. The reason the version where the transforms are applied as part of the model is much slower is b/c it's doing a lot more gradient steps as part of the acquisition function optimization (around 4 times as many, so that explains the runtime difference). Now I don't know why that is, but it's definitely something worth digging into more deeply - cc @saitcakmak

acq_func

ncalls	tottime	percall	cumtime	percall	filename:lineno(function)
6002	3.094	0.0005154	3.094	0.0005154	~:0(built-in method torch.cdist)
5802	2.582	0.0004449	2.582	0.0004449	~:0(method 'run_backward' of 'torch._C._EngineBase' objects)
6003	0.6571	0.0001095	4.676	0.000779	matern_kernel.py:86(forward)
12006	0.2948	2.455e-05	0.2948	2.455e-05	~:0(method 'div' of 'torch._C._TensorBase' objects)
96058	0.2893	3.012e-06	0.4166	4.337e-06	functional.py:76(broadcast_shapes)

acq_func2

ncalls	tottime	percall	cumtime	percall	filename:lineno(function)
26923	12.29	0.0004564	12.29	0.0004564	~:0(method 'run_backward' of 'torch._C._EngineBase' objects)
27123	5.519	0.0002035	5.519	0.0002035	~:0(built-in method torch.cdist)
27124	1.582	5.834e-05	9.949	0.0003668	matern_kernel.py:86(forward)
461117	1.449	3.141e-06	2.075	4.5e-06	functional.py:76(broadcast_shapes)
461091/27123	1.225	4.517e-05	4.572	0.0001686	module.py:807(_apply)

[saitcakmak]

@Balandat Yeah, in this case there are 4 times as many evaluations (of f_and_grad) within the optimizer. The number of evaluations can be quite variable, so it's hard to say what exactly causes this. It could be that it ends up being harder to satisfy the convergence conditions for L-BFGS-B with Normalize.

[esantorella]

Interesting. I hadn't realized this, but with an affine transform, the steps chosen by L-BFGS (after untransforming) do generally depend on the scale of the transform. Additionally, at initialization, some transforms will work better or worse with L-BFGS's initial approximation of the Hessian as the identity function.

Here's an example of a simple 2d minimization problem where optimizing with a transform works better than optimizing directly in the transformed space: We have a transform that multiplies the first element of x by 0.5, and we want to minimize f(transform(x)), where f(x) = (2 x[0]) ** 2 + x[1] **2. So f(transform(x)) = x[0] ** 2 + x[1] ** 2.

In this case, with the transform, the L-BFGS step direction is perfect because the Hessian approximation is perfect, and it takes just 2 steps to converge. If we optimize directly in the transformed space, the step is tilted towards x0.

I could have just as easily rigged an example where optimizing in the transformed space works better. So, I don't think either normalizing or not is generically better.

I'm not sure why exactly this is happening with ohassn's data, but in that data, there is one value in train_y that is vastly higher than all the others, so the GP might have trouble modeling this.