[Feature Request] Select the best-fitting GHOST model as in MixtureFinder

[HS6986]

Dear IQ-TREE Developers,

Sorry for posting this twice. I thought this post would be more appropriate in Discussion, so I moved it from Issue.

Recently, Ren et al. (2024) proposed the “MixtureFinder” approach for selecting the best-fitting nucleotide mixture model. Although I am not knowledgeable in phylogenetics, I believe that this approach could potentially be extended for selecting the best-fitting GHOST model (Crotty et al., 2020). This method might be able to be called, for example, GHOSTFinder. The algorithm is as follows:

1. Determine the best-fitting single (non-mixture) model without a rate heterogeneity across sites (RHAS) parameter for the dataset.
2. Add classes incrementally, as long as it increases model fit.
3. Repeat steps 1 and 2 for the following variants of the GHOST model, each of which has a completely independent model condition (see Crotty et al., 2020 and Complex models documentation).

• Linked GHOST models with optimized base frequencies (MIX{AA+FO,...,ZZ+FO}+Hx)
• Unlinked GHOST models with empirical base frequencies (MIX{AA+F,...,ZZ+F}*Hx)
• Unlinked GHOST models with optimized base frequencies (MIX{AA+FO,...,ZZ+FO}*Hx)

4. Present the best-fitting GHOST model.

This feature would enable users to efficiently select the best-fitting GHOST model. Also, it would allow the use of a GHOST model composed of classes with different Q-matrices, potentially improving model fit compared to a GHOST model composed solely of classes with a specific Q-matrix (e.g., GTR). This method can be extended to amino acid, binary, multistate, and morphological data.

In actual use, it would be desirable to compare the model fit of the best-fitting GHOST model with that of other mixture models and unpartitioned non-mixture models (model fit between partitioned non-mixture models and unpartitioned mixture models cannot be compared using information criteria such as AIC and BIC; Crotty & Holland, 2022; Liu et al., 2023). For morphological data, only MK+FQ+Hx is applicable (though unpartitioned morphological phylogenetic analysis cause observer bias; see replies in #395), and the comparison can only be made with unpartitioned non-mixture models with or without a RHAS parameter.

A potential challenge with this approach is that, for models with a high number of parameters, including the GHOST model, AIC tends to overfit data, while BIC tends to underfit data (Zhou et al., 2007; Crotty et al., 2020), but this limitation may also apply to MixtureFinder.

I would greatly appreciate it if you could consider adding this feature in a future IQ-TREE release (if my understanding is correct).

[bqminh]

This is an interesting idea! I like GHOSTFinder name :-) @HuaiyanRen what do you think?