Hi Statalist members,

I was wondering whether there is any explicit test that can tell me if I need a multi-level analysis?

Is it correct to check for the percentage of the variance in the outcome variable that is attributable to “membership” in the group in which the observations are nested? For example: The observations in my study are business units (over ~15 years); and these business units are nested in firms. I study the influence of a certain strategy of a business unit (independent variable of interest) on the financial performance of the business unit (dependent variable).

That is, I checked for the % of the variance in the financial performance of business units (DV) that is attributable to firm membership. If that number turns out to be very low, a multi-level model may not necessarily be needed?

More specifically, I did the following:
  • First, I ran the following Stata code. BU_performance is the financial performance of the business unit; and Firm_ID is the identifier of the firm to which the business unit belongs:
Code:
mixed BU_performance || Firm_ID:, var
  • The result of this is:
Code:
Mixed-effects ML regression                     Number of obs     =      4,496
Group variable: Firm_ID                          Number of groups  =        280

                                                Obs per group:
                                                              min =          1
                                                              avg =       16.1
                                                              max =         65

                                                Wald chi2(0)      =          .
Log likelihood =  1484.6113                     Prob > chi2       =          .

--------------------------------------------------------------------------------
BU_performance |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
---------------+----------------------------------------------------------------
         _cons |  -.0047854   .0025938    -1.84   0.065    -.0098692    .0002985
--------------------------------------------------------------------------------

------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
Firm_ID: Identity             |
                  var(_cons) |   4.34e-22   1.28e-21      1.33e-24    1.42e-19
-----------------------------+------------------------------------------------
               var(Residual) |    .030249    .000638      .0290241    .0315257
------------------------------------------------------------------------------
LR test vs. linear model: chibar2(01) = 0.00          Prob >= chibar2 = 1.0000
  • Then, I computed the following:
Code:
var(_cons)   /   (var(_cons) + var(Residual)) =
= 4.34e-22 / (4.34e-22 + 0.030249) =
=~ 0.0000
  • From the above, it seems that less than 1% of the variance in the financial performance of business units is attributable to firm membership? This seems like a very low value. I have heard that anything above 5% definitely requires a multi-level model, but values lower than 1% may not.
Thanks so much for any advice on whether the above approach is alright.

Franz