emcee solver: invalid value encountered in double scalars

[dmrowan]

Hi,

I am trying to use the EMCEE solver to fit a light curve and am running into a runtime issue that halts progression without raising any exceptions.

I am fitting a single band light curve, and have used the estimators + nelder-mead optimization to get an initial solution. Following the emcee tutorial, I checked that model from the initial distributions represents the observations well before starting the MCMC run.

I am trying to run the solver for 5000 iterations. After a few hundred steps, this error is printed:

Mon, 22 Aug 2022 10:18 BUNDLE ERROR Constraint 'requiv_max@primary@star@constraint' raised the following error while attempting to solve for 'requiv_max@primary@star@component'. Consider flipping the constraint or changing the value of one of ['q@binary@orbit@component', 'syncpar@primary@star@component', 'ecc@binary@orbit@component', 'sma@binary@orbit@component', 'incl@primary@star@component', 'long_an@primary@star@component', 'incl@binary@orbit@component', 'long_an@binary@orbit@component', 'requiv_max@primary@star@component'] until the constraint succeeds. Original error: name 'nan' is not defined
Mon, 22 Aug 2022 10:18 BUNDLE ERROR Constraint 'requiv_max@secondary@star@constraint' raised the following error while attempting to solve for 'requiv_max@secondary@star@component'. Consider flipping the constraint or changing the value of one of ['q@binary@orbit@component', 'syncpar@secondary@star@component', 'ecc@binary@orbit@component', 'sma@binary@orbit@component', 'incl@secondary@star@component', 'long_an@secondary@star@component', 'incl@binary@orbit@component', 'long_an@binary@orbit@component', 'requiv_max@secondary@star@component'] until the constraint succeeds. Original error: name 'nan' is not defined
:1: RuntimeWarning: invalid value encountered in double_scalars
Mon, 22 Aug 2022 10:18 BUNDLE ERROR Constraint 'asini@binary@orbit@constraint' raised the following error while attempting to solve for 'asini@binary@orbit@component'. Consider flipping the constraint or changing the value of one of ['sma@binary@orbit@component', 'incl@binary@orbit@component', 'asini@binary@orbit@component'] until the constraint succeeds. Original error: name 'nan' is not defined
Mon, 22 Aug 2022 10:18 BUNDLE ERROR Constraint 'asini@primary@star@constraint' raised the following error while attempting to solve for 'asini@primary@star@component'. Consider flipping the constraint or changing the value of one of ['sma@binary@orbit@component', 'incl@binary@orbit@component', 'q@binary@orbit@component', 'asini@primary@star@component'] until the constraint succeeds. Original error: name 'nan' is not defined
Mon, 22 Aug 2022 10:18 BUNDLE ERROR Constraint 'asini@secondary@star@constraint' raised the following error while attempting to solve for 'asini@secondary@star@component'. Consider flipping the constraint or changing the value of one of ['sma@binary@orbit@component', 'incl@binary@orbit@component', 'q@binary@orbit@component', 'asini@secondary@star@component'] until the constraint succeeds. Original error: name 'nan' is not defined
Mon, 22 Aug 2022 10:18 BUNDLE ERROR Constraint 'logg@secondary@star@constraint' raised the following error while attempting to solve for 'logg@secondary@star@component'. Consider flipping the constraint or changing the value of one of ['mass@secondary@star@component', 'requiv@secondary@star@component', 'logg@secondary@star@component'] until the constraint succeeds. Original error: name 'nan' is not defined

Then, a runtime warning is printed repeatedly for much longer than the length of a single iterations.

RuntimeWarning: invalid value encountered in double_scalars

On my machine a single iteration takes ~3 seconds, but this warning will print repeatedly for more than an hour. The main iteration progressbar increases by 1, and then this series of errors repeats for the next iteration step.

I've also looked at the 'top' output while the MCMC is running. The number of processes running drops to 1 when this error starts. Maybe one walker is getting stuck?

I have tried running the MCMC in chunks of 50 iterations using the 'continue_from' qualifier until this error pops up. It looks like all of the walkers are moving in the right direction, and nothing jumped out as clearly wrong when I plotted 'lnprobability' or 'trace'.

I have tried running this on different machines, and also tried upgrading from version 2.3 to 2.4. I have been able to run MCMC successfully with a similar setup for different binaries, and I'm not quite sure what is different here.

Do you have any suggestions for addressing this issue? I am happy to email the script from 'export_solver' if that would be helpful.

Cheers,
Dom

[kecnry]

Hi Dom - what version of PHOEBE are you using? This sounds like a bug in the constraints, and 2.4.4 recently fixed a similar bug for the requiv constraints that might be related 🤞. If you are on 2.4.4 already and happen to have a version of the bundle that shows this error (it would probably either show in the logger or when calling b.run_checks()), that would be easiest, otherwise I can try to reproduce with the export solver script if you want to send that my way.

Thanks!
Kyle

[dmrowan]

Hi Kyle,

I am using version 2.4.4. I've emailed you the bundle. The issue occurs when I use
b.run_solver('emcee_solver', solution='emcee_sol')

Using b.run_checks(solver='emcee_solver') returns 3 failed items. One is a warming about pblum_method, and the other two are:

ERROR: requivfrac@primary@star@component is currently constrained but cannot automatically temporarily flip as solve_for has several options (requiv@primary@star@component, sma@binary@orbit@component). Flip the constraint manually first, set adopt_values=False, or remove requivfrac@primary@star@component from adopt_parameters. (4 affected parameters, affecting adopt_solution)

ERROR: requivfrac@secondary@star@component is currently constrained but cannot automatically temporarily flip as solve_for has several options (requiv@secondary@star@component, sma@binary@orbit@component). Flip the constraint manually first, set adopt_values=False, or remove requivfrac@secondary@star@component from adopt_parameters. (4 affected parameters, affecting adopt_solution)

This seems odd to me since these parameters are not in the init_from distribution (I'm solving for requiv@primary and requiv@secondary, though). If this was an issue, it also seems odd to me that it would cause the problem after ~100 iterations of the MCMC sampler.

Thanks for the help!
Dom

[kecnry]

I think I've figured out what's going on. You've flipped the constraints to provide the mass as an input parameter and a sampling distribution which allows drawing negative masses, but the limits on the mass parameter are not properly rejecting negative values (this is a bug).

I filed an issue (#631) and try to get a bugfix release out when I get the chance, but for now, you should be able to get around this by either setting a boxcar prior on the mass, using AND logic with a boxcar initializing distribution on top of your existing gaussian distribution, or setting the limits on the parameter yourself (this generally isn't officially supported to be changed manually, but is essentially what the bugfix will do):

for param in b.filter(qualifier='mass', context='component').to_list():
    param._limits = [1e-6, None]

[dmrowan]

Hi Kyle,

Wow, thank you! I narrowed the problem down the mass but did not realize it was going to negative values. I used the second option, setting param._limits, and was able to run the full chain.

Instead of using param._limits = [1e-6, None], I had to also give the unit: param._limits = [1e-6*u.Msun, None]

Thanks for your help! I had been stuck on this for a while now.

Cheers,
Dom