model fitting errors for SingleTaskMultiFidelityGP & MultiTaskGP

[ToennisStef]

Hi
During optimization runs using the SingleTaskMultiFidelityGP and the MultiTaskGP i often encounter model fitting errors using the fit_gpytorch_mll function.

I was wondering where these model fitting problems came from. I was also wondering if both models require a strict ordering of the fidelity values? and how do these models handle non-linear inter-task/inter-fidelity dependencies?
For example, I got this test problem with 4 fidelity levels:

Here the SingleTaskMultiFidelityGP had some model fitting issues.
I know that these issues can also arise when there are identical or many close data points, which then leads to an ill-conditioned covariance matrix. But could some problems also arise from the fact that there are non-linear inter-fidelity/inter-task dependencies and/or that the model requires a strict ordering of the fidelity functions, which is not given (meaning target fidelity has lowest function values on the whole domain, second highest fidelity has 2nd lowest function values on the whole domain, etc.)?

To be honest I also set the noise covariance of the models close to zero, which I know can also be troublesome, but I cannot pinpoint the exact problem since the error is not very informative.

Best regards,
Stefan Tönnis