Hi all,

I think I may have inadvertently stumbled upon a bug after using the runmlwin command for the Stata MLwiN add-on (from the Centre for Multilevel Modelling) and outputting the results via the esttout package.

In short the (perceived) bug is that after using the runmlwin command to conduct Bayesian analyses on a dataset of > 400,000 with a binary DV, I get the results of these analyses in my Stata console window. When I output these results as odds ratios (OR) to a word document using the esttab command (part of the estout package) the significance levels of a handful of variables changes wildly for, at least as far as I can tell, no logical reason. In other words, a few key variables have completely different significance levels in the output of the results in the Stata console window when compared to the output of the results in the document that is produced using the esttab command. Again, as far as I can see this happens for no logical reason unless I am being ignorant and missing something stupendously obvious. This occurs across a number of models that I run and is not restricted to a single use case.

Interestingly however, when I simply output my results as their raw coefficients that haven't been exponentiated to odds ratio to an external document via the esttab command, the significance levels of the variables are 'correct' in that they concur with the p-values that the Stata console window outputs. Unfortunately my main laptop which I run my State estimations on is currently away being repaired so I can't accompany my post with screenshots and copies of these outputs currently.

It should be relatively straightforward to determine whether this is a bug or not at the moment however - although where the bug occurs/what add-in is causing it is a different problem to diagnose entirely. Quite simply, if there is no theoretical reason for the p-values of variables to change when a mean for a variable (in line with Bayesian MCMC modelling) is outputted as an odds-ratio as opposed to a raw coefficient, then this can only be a bug and, if this is the case, then I can get on why trying to diagnose where it lies (whether that be within Stata itself, the esttout add-on, or the runmlwin package) - any guidance that anybody could offer regarding this would be greatly appreciated.

Many thanks,