Use pretrained Gpytorch model for SingleGP

[skiyohara]

Dear botorch developers and users,

I am a beginner of botorch and wondering how to use the pretrained GpyTorch model.
I constructed a new ExactGP model (MyExactGP) which combined with a neural network-based feature transformer by GpyTorch pacakge and trained it.

Then, I want to use the model to find an optimized objective value through ExpectedImprovement, ProbabilityOfImprovement, and UpperConfidenceBound.

Probably, I should custom SingleTaskGP-like model (MySingleTaskGP) which is a subclass of MyExactGP and GPyTorchModel), but the forward function (transformer) of my GpyTorch model receives multiple inputs for feature transformation.

How should I realize to receive multiple inputs?

Here is a pseudocode of MyExactGP.

class MyExactGP(gpytorch.models.ExactGP):
        def __init__(self, train_x, train_y, likelihood):
            super(MyExactGP, self).__init__(train_x, train_y, likelihood)
            self.mean_module = gpytorch.means.ConstantMean()
            self.covar_module = gpytorch.kernels.GridInterpolationKernel(
                gpytorch.kernels.ScaleKernel(gpytorch.kernels.RBFKernel(ard_num_dims=num_dims)),
                num_dims=num_dims
            )
            self.feature_extractor = feature_extractorNN() # feature_extractorNN is a deep neural network that transforms multiple arguments into high dimensions (defined elsewhere)

            self.scale_to_bounds = gpytorch.utils.grid.ScaleToBounds(-1., 1.)

        def forward(self, x, y, z):  # receive multiple arguments
            # I'm first putting my data through a deep net (feature transformer)
            projected_x = self.feature_extractorNN(x, y, z)
            projected_x = self.scale_to_bounds(projected_x) 

            mean_x = self.mean_module(projected_x)
            covar_x = self.covar_module(projected_x)

            return gpytorch.distributions.MultivariateNormal(mean_x, covar_x)

Any suggestion would be helpful.
Thanks in advance.

Best regards,
Shin

[Balandat]

What size / dimension are the three inputs x, y, z in your case?

[skiyohara]

x is Torch Tensor ((q,m)-shape), and y and z are nested Lists (their length are q). Here, I don't use batch estimation.

[saitcakmak]

At a high level, BoTorch doesn't care about the internals of your model. The main interface the acquisition functions utilize to access model predictions is the posterior model and the Posterior object it returns. To use your custom model in BoTorch, you'll need to define a posterior method that passes down the necessary inputs down to model and returns the Posterior object with the expected output shapes.

A recent discussion that may be relevant: #2020
The posterior implementation for GPyTorch based models: https://github.com/pytorch/botorch/blob/main/botorch/models/gpytorch.py#L338
The GPyTorchPosterior object that works with GPyTorch MVNs: https://github.com/pytorch/botorch/blob/main/botorch/posteriors/gpytorch.py#L37

[skiyohara]

Thank you for your suggestion. I will modify the posterior function to receive multiple inputs.