Contrast coding, LMMS and interactions

[Thomas]

Dear Henrik,

Thanks for developing the afex package and for writing the piece on mixed models in EP. The latter piece made me reconsider the contrast coding in LMMs and I was hoping to get your thoughts on one such model. In particular, Im interested in testing whether a continuous variable is related to a continuous outcome AND whether this relationships is moderated by two categorical (binary) variables, such as e.g. sex and socioeconomic status, controlling for a set of nuisance variables (categorical and continuous vars). From your paper it seems that, with simple treatment coding contrasts, one can readily interpret the 3-way interaction, but not so much the 2-way interactions or “main”/simple effects of the continuous predictor by itself. I have been wondering how to proceed if the 3-way interaction is not significant:
1. Does one use sum-to-zero contrasts to be able to accurately interpret the two-way interactions and, if both are non-signifiant, the “main” effect of the continuous predictor, as is done automatically in AFEX, and/or
2. Would one refit the model without the 3-way interaction and interpret the 2-way interaction; a slightly more model selection procedure.

Thanks!

[henrik]

If you have interactions involving categorical variables, I do not see why you would use treatment contrasts. In my eyes sum-to-zero just makes more sense. So I would not spend too much time thinking about what treatment contrasts would mean in that case.

The answer to you question only depends on the balance of the two categorical variables and is essentially the same as the Type III versus Type II sums of squares question. In the balanced case it does not matter, both models would lead to the same outcome. In the case of imbalance, removing the three-way interaction is the same as using Type II tests.

In case of imbalance, refitting without the three-way interaction does not adjust (or “control”) the two-way interactions for the three-way interaction. In case of observational data that is probably preferred. In case of experimental data, I would use the full model with the three-way interaction.

[Thomas]

Dear Henrik,

This is great, many thanks.

I assume that with balanced you refer to the balance of the categories across subjects (e.g. an equal number of males and females). If that the case, yes, the two categorical variables are unbalanced. You suggest that removing the 3-way interaction, in the case of observational data, may be preferred – why do you believe this is the case?

In terms of interrogating the effects: I have indeed some observational data and wish to examine the relationship between two continuous variables, and whether this relationship is moderated by two categorical (binary) variables. Note, the categorical variables, as well as nuisance categorical (binary and with 3 levels) variables that are part of the model, are strongly unbalanced. Am I right in saying that:
1.one would first inspect the 3-way interaction in the ANOVA table (SS type 3). In case of non-significance, one would remove this interaction and refit the model with the 2 2-way interactions and again inspect the ANOVA table. In case one of them is significant, the 2-way interaction is to be interpreted / investigated. To this end, the interaction is to be visualised and one could take the regression estimate of the summary() table to interpret the strength/direction of this interaction?

2. In case none of the 2-way interactions are “significant” either, one would remove those and refit the model again. One would then use summary() to be able to interpret the regression coefficient of the variable of interest.

Finally, I have fit the same 3-way interaction model with afex::mixed and lmerTest::lmer (with set_sum_contrasts()), and noticed that the effects (beta’s, t-values, df’s) are slightly different, so is the ANOVA (SS3, method = “S”) output. Do you know why this is the case?

Thanks in advance.

[Thomas]

EDIT to my previous post: please ignore my final query about differences in output. This was caused by REML vs ML fitting.

[henrik]

If you do want the sequential type tests in which each test does not the effects of a higher level, you can simply use type = 2, this is exactly what it does. In mixed the implementation of Type II depends on the specific method, but the general approach is the same across methods.

So in this case I would run Type II tests (i.e., type = 2) and then look at the highest order effect that is significant. Then, refit the model that only includes those effects and use this model to inspect the effects. However, if there are interactions with continus variables, I would suggest using emmeans::emtrends for this instead of looking at the parameter estimates directly (i.e., emmeans::emtrends for looking at conditional slopes of interactions of continuous with categorical/continuous variables).

[Author]

Posts