Initialising MLPRegressor with precomputed matrices of weights and biases

[tamoghnabh]

Is it possible to initiate an MLPRegressor with pre-computed weights and biases?

I am working on a problem where I am using an online MLPRegressor. Initially, I have some data with which I can pre-train the model. I am using learn_many for this. Since River has no GPU support, the process exclusively runs on the CPU and takes a good amount of time when the data set is large enough. This can be avoided if the regressor is initiated with weights and biases computed using a GPU-enabled library.

[MaxHalford]

Hello 👋

This is not a use case we support as of now. In theory you could do this by accessing the w (weights) and b (biases) attributes. But that sounds a bit hacky, especially if you're using a different library than River to train the model. Indeed, cross-library weight sounds super finicky, especially because the implementations are likely to be different, so you'll have a hard time to reproduce outputs.

Maybe the way to go is to add some GPU support to River's MLPRegressor. For instance, I know Numba is able to transpile numpy code to GPU instructions. There's also CuPy.

I hope that helps! The neural_net module is pretty basic, and needs some TLC. Feel welcome to contribute.

[dberardo-com]

related question to MLPRegressors:

is it possible to have a multi-output neural network (basically where the Y vector has more than one column )?

and more in general: what are the current multi-output regressors supported by river?

Regards

---

UPDATE:

i have had a look at the multioutput.RegressorChain class, and it seems not to work for the MLPRegressor. My question above refers to whether there is any Model currently supported that can handle multiple output natively.

In case this needs always to be achieved with RegressorChains: is it possible to prevent the output from the first regressor to become a feature in the second one ? or would this make no sense at all, as doing so would practically result in having N completely independent regressors, one for every N target variable ?

[MaxHalford]

is it possible to have a multi-output neural network (basically where the Y vector has more than one column )?

Yes, it's possible, but not very well tested. Some work is needed here.

and more in general: what are the current multi-output regressors supported by river?

For now, there's only tree.iSOUPTreeRegressor and the wrappers in the multioutput module.

In case this needs always to be achieved with RegressorChains: is it possible to prevent the output from the first regressor to become a feature in the second one ? or would this make no sense at all, as doing so would practically result in having N completely independent regressors, one for every N target variable ?

That would indeed result in independent regressors. We could add a parameter to RegressorChain that allows this behavior though. But why would you want it? Because you have a model with a fixed number of inputs?

[dberardo-com]

thanks for the reply. my thought was to use RegressorChains as a workaround for this issue, but indeed it makes no sense to make them independent (although sometimes it might be handy to have only "one big model" to be persisted on disk, rather than a variety of models, but this is not the matter here).

Thanks for pointing me to the iSOUPTreeRegressor, i might have a look at it. I am still developing a use case to be honest. I have some issues figuring out the actual use case for regressors in a streaming application. For now i am working on some sort of simple digitalTwin, which could be used to make simple simulations of a system and "what-if" analysis.

Also given a multi-output linear regression model, whose weights are represented by a matrix "A", and given a multi-output vector "y" and input vector "x", with the form Ax=y , then one might use the inverse of the equation to "give suggestions" to the users for how they could achieve a desired system state x* by changing y*, so basically: x*=A^-1 y*.

This would some sort of very low level control theory, that i am trying to experiment with ... again, i am just exploring some use cases here.