I am working on a research project that predicts that country-level variables will predict individual-level perceptions. I’ve never used a multi-level model and I could really use some advice to make sure I’m doing things correctly!

I have collected survey data from respondents across 18 countries, with each respondent reporting their perceptions (4 items that I average to 1 factor called “Perception”) across a range of different scenarios.
At level 1, the DV (Perception) and four predictor variables (Medium, Large, (Small is omitted), Self, and QUI are measured at the Respondent-Scenario level.
At level 2, I have a factor variable for each Respondent (Because of different response rates, each respondent provided perceptions on between 5 and 18 scenarios—I’ve dropped any observations where they reported on fewer than 5 scenarios.)
At level 3, I have 5 country-level variables (each Respondent is embedded in a single country), which are measured using archival data. These are IN, PD, UA, MA, and Tight. (There’s a sixth Level-3 variable I’d like to add, but I’m having trouble getting the model to converge so have omitted it for now.)
Ideally, I would like to test 4 of the country-level variables as an interaction with one of the level-1 variables (QUI), with the prediction that QUI predicts Perception as a main effect, and that each of the country-level variables (IN, PD, UA, MA) affect the nature of that relationship. Moreover, theory would suggest that the fifth country-level variable (Tight) might influence the magnitude of each of those four relationships (i.e., three-way interactions).
I’m willing to collect more data, but it is expensive and so I’d like to make sure I understand the empirical model before I do so—to make sure I’m collecting the right data. In that regard, I have several questions.
  1. I’m new to mixed level modeling. Is the syntax of the model (copy/pasted below) correct for my objective? I don’t understand why Stata notes that there are 345 Level-3 groups (exactly the same as the Level 2 Groups). When I -tab- the variable, I get a list of 30 countries.
  2. I was thinking that a different approach would be to consider only the data for which QUI==1 and forget about the interaction effects, but I wasn’t sure I could put country-level variables into Level-1 of the model without the interaction effect with a Level-1 variable. Is this possible, and what would the syntax look like? (Hypotheses could make sense either way, and this would eliminate the need for the three-way interaction.)
  3. When interpreting the data for hypothesis tests, do I simply focus on the main regression table? Or do the data for the Random Effects Parameters matter in terms of assessing hypotheses (assuming standard errors are fully reported by Stata?)
  4. My biggest problem right now is that often times models don’t converge. Or they converge but do not report Level-3 standard errors. In terms of additional data collection, my aim would be to collect data from respondents from more countries to come close to 50 countries represented at Level 3. Am I correct in assuming that should reduce this problem?
  5. I am posting code below using the [CODE] tag… hopefully I am doing it correctly! I copied the code from the model without the country-level Tight variable or 3-way interaction(s). (When I add Tight to the model (as an additional interaction with one of the Level-1 interactions), the model converges but does not display standard errors.) Any advice would be greatly appreciated!




[code]
. mixed Perception Medium Large Self QUI##c.UA QUI##c.MAS QUI##c.PD QUI##c.IN || ParticipantIDEncoded: || EncodedCountryLiving : c.MA c.UA c.PD c.IN c.Tight

Performing EM optimization:

Performing gradient-based optimization:

Iteration 0: log likelihood = -4917.4528
Iteration 1: log likelihood = -4913.7166 (not concave)
Iteration 2: log likelihood = -4911.1487
Iteration 3: log likelihood = -4904.5822 (not concave)
Iteration 4: log likelihood = -4901.1288 (not concave)
Iteration 5: log likelihood = -4900.5816
Iteration 6: log likelihood = -4899.9071
Iteration 7: log likelihood = -4898.9582
Iteration 8: log likelihood = -4898.5918
Iteration 9: log likelihood = -4898.569
Iteration 10: log likelihood = -4898.5633
Iteration 11: log likelihood = -4898.5629
Iteration 12: log likelihood = -4898.5628

Computing standard errors:

Mixed-effects ML regression Number of obs = 4,176

-------------------------------------------------------------
| No. of Observations per Group
Group Variable | Groups Minimum Average Maximum
----------------+--------------------------------------------
Participan~d | 345 5 12.1 18
EncodedCou~g | 345 5 12.1 18
-------------------------------------------------------------

Wald chi2(12) = 65.63
Log likelihood = -4898.5628 Prob > chi2 = 0.0000

------------------------------------------------------------------------------
Perception | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
Medium | -.0386188 .030414 -1.27 0.204 -.0982292 .0209916
Large | -.0299678 .0376828 -0.80 0.426 -.1038247 .0438891
Self | -.0109311 .0300319 -0.36 0.716 -.0697925 .0479303
1.QUI | -.3943429 .3550538 -1.11 0.267 -1.090236 .3015498
UA | .0013038 .0022572 0.58 0.564 -.0031203 .0057279
|
QUI#c.UA |
1 | -.0025959 .0017444 -1.49 0.137 -.0060148 .000823
|
MAS | -.0031465 .0030996 -1.02 0.310 -.0092215 .0029286
|
QUI#c.MAS |
1 | -.00458 .0024491 -1.87 0.061 -.0093801 .0002202
|
PD | .0099201 .0045963 2.16 0.031 .0009116 .0189286
|
QUI#c.PD |
1 | .0121409 .0037061 3.28 0.001 .004877 .0194048
|
IN | .008452 .0039308 2.15 0.032 .0007478 .0161562
|
QUI#c.IN |
1 | .0015094 .003118 0.48 0.628 -.0046016 .0076205
|
_cons | -.9229564 .4384154 -2.11 0.035 -1.782235 -.063678
------------------------------------------------------------------------------

------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
Participan~d: Identity |
var(_cons) | 4.82e-08 1.19e-07 3.78e-10 6.15e-06
-----------------------------+------------------------------------------------
EncodedCou~g: Independent |
var(MAS) | 2.51e-15 2.69e-12 0 .
var(UA) | 2.40e-07 6.67e-06 4.74e-31 1.21e+17
var(PD) | .0000457 .0000124 .0000268 .0000778
var(IN) | .0000108 5.12e-06 4.28e-06 .0000273
var(Tight) | 4.51e-14 9.84e-14 6.29e-16 3.24e-12
var(_cons) | 4.81e-08 .0000239 0 .
-----------------------------+------------------------------------------------
var(Residual) | .532151 .0123901 .5084127 .5569977
------------------------------------------------------------------------------
LR test vs. linear model: chi2(7) = 773.81 Prob > chi2 = 0.0000

Note: LR test is conservative and provided only for reference. [code]