Correct Split-plot Mixed Model Specification

[Dom42]

Hi there,

first of all: thank you for creating such a helpful and well-crafted package!

Now to my question:
I’m currently in the process of analysing the data of a split-plot experiment and am struggling to specify the correct model term and I was hoping you could help with that.

The design:
The participants (participant id = VP) were pseudo-randomly assigned to 3 different between subjects conditions (condition). Within these conditions they all completed 6 experimental blocks (blockID) coded as 0-5. Half of these blocks were of a higher difficulty than the other half (SM.blockDiff) with a randomly assigned order (so 3 measurements per difficulty with a varying order of block difficulties per participant). The trials within the blocks were assigned randomly, so the blocks would vary between persons but the general manipulation of SM.blockDiff was the same for all participants (i.e. it can be assumed that the blocks were basically equal for all participants). We measured a questionnaire score as the continuous DV (quest).

The research question:
We are expecting a fixed effect of the between subjects condition and the within subjects SM.blockDiff. This basic model was modeled using this term:

m1 <- mixed(quest ~ condition * SM.blockDiff + (SM.blockDiff|VP), data=df)
This model returned a significant effect for both fixed main effects but not for the interaction – which is what we expected.

We now wanted to control for possible exhaustion effects over the course of the experiment, introducing blockID as another fixed effect. We expect that there might be an interaction with condition but mainly we want to generally control for the timing aspect.
First of all, we are unsure, if such a representation is correct. Second of all, we struggled to implement the model into the afex framework as we are unsure about the correct error term). We tried this model:

m2 <- mixed(quest ~ blockID * condition * SM.blockDiff + (blockID + SM.blockDiff|VP), data=df)
Which returned the error message, that blockID is not centered (which is true, as this is a timing variable where centering would not be conclusive to us).

We were now wondering, if the second model is correctly specified in your eyes or if e.g. a SEM would be more appropriate although there is no real time lag between the blocks. Also we are unsure if this is the maximum model given the design above.

We know that model specifications are maybe more linked to the lme4 package but as we then would have to specify the model using effect/contrast coding, we figured we would try to do it using mixed from your package.

Your thoughts on the topic would be highly appreciated.

Thanks and regards,
Dom

[henrik]

mixed should only emit a warning if you enter a continuous covariate, not an error. This warning tells you that all model terms blockID interacts with are now evaluated when blockID is zero. In your case this refers to all terms of your m1 (i.e., main effects of condition and SM.blockDiff as well as their interaction), they are now all simple effects when blockID = 0. That is why centering makes sense as they are then evaluated at the mean of blockID (if centered at the mean). For more on this issue see:
Dalal, D. K., & Zickar, M. J. (2012). Some Common Myths About Centering Predictor Variables in Moderated Multiple Regression and Polynomial Regression. Organizational Research Methods, 15(3), 339–362. https://doi.org/10.1177/1094428111430540

You can avoid the issue of centering by only adding blockID as a main effect: quest ~ blockID + condition * SM.blockDiff + (blockID + SM.blockDiff|VP). However, I am generally not a big fan of “conrolling for” factors in this sense. It is often unclear to me what kind of substantial question are answered by that if indded shared variance is removed then.

Note that in any case, adding blockID only accounts for linear effects of time. If you want to take the idea of exhaustion or attention seriously, you need to consider more complicated models that allow for non-linear effects. See:
Baayen, H., Vasishth, S., Kliegl, R., & Bates, D. (2017). The cave of shadows: Addressing the human factor with generalized additive mixed models. Journal of Memory and Language, 94, 206–234. https://doi.org/10.1016/j.jml.2016.11.006

[Dom42]

Hi Henrik,

thank you for your quick response and excellent reads!

You are right, this is only a warning and not an error. I see the point of centering and the change in interpretation caused by this. We were just a little bit unsure whether centering would be applicable for a time variable such as this one (as the interpretation of a “mean” block felt rather unnatural).

Concerning the “controlling for” of this factor: when inspecting the data visually, we concluded that probably only a linear effect would be appropriate (if at all an effect of blockID would play a role). In order to show, that this linear exhaustion effect plays no role in the interpretation of the other effects, we wanted to include it in the model.
Judging from your model proposal, it would not be possible for us, to uncover interaction effects on condition if we wanted to avoid the centering issue, correct?

If I understand your answer correctly, you would propose (if including it at all) either using blockID as a main effect only or if including it with its interaction term interpreting the other two factors as simple main effects, correct?

What worried us as well is the question if the error term (when including blockID) was correctly specified as (blockID + SM.blockDiff|VP) judging from the experimental design. Is that one correct in your eyes?

Thank you again and with regards,
Dom

[henrik]

As far as I understand your design, the SM.blockDiff and blockID interaction only varies across participants. If this is indeed the case, it cannot receive random slopes. In this case, (blockID + SM.blockDiff|VP) is indeed the maximal model justified by the design for the by-participant random intercept. For more on this I recommend our chapter:

I also do not see how centering blockID would do any harm. That is usually a good idea for conintuous covariates. Again, I recommend reading:
Dalal, D. K., & Zickar, M. J. (2012). Some Common Myths About Centering Predictor Variables in Moderated Multiple Regression and Polynomial Regression. Organizational Research Methods, 15(3), 339–362. https://doi.org/10.1177/1094428111430540

[Dom42]

Thank you again for your swift reply.
I was not aware of your chapter, so thank you very much for the read. I will definitely have a look.

Yes, your assumption is correct. Then I’m glad, that the model specification is correct.

Concerning the centering: I now tried to rerun the model using a centered blockID – generated using blockID.c <- scale(blockID, scale = F).
Fitting the model this way now generates a warning message stating: Missing cells for: blockID.c. Interpret type III hypotheses with care..
The variable does not contain any NAs and the centering was successful (with distinct values of blockID.c ranging from -2.5 to 2.5 as expected with blockIDs ranging from 0-5). I was also not expecting any empty cells in the covariance matrix as every participant was assigned a condition, went trough the 6 blocks and experienced both difficulties.

I’m a bit unsure where this warning comes from and how it should be treated.

Sorry for bothering and thanks again for your help!

[henrik]

scale() does not return a vector. It is safer to calculate directly. One of the following should work:

df$blockID.c <- df$blockID - 2.5
df$blockID.c <- df$blockID - mean(df$blockID)

[Dom42]

Thanks for the advice.

Unfortunately the warning message persists even with the manual centering (producing a num vector ranging from -2.5 to 2.5).
The model being used is mixed(quest ~ blockID.c + condition * SM.blockDiff + (blockID.c + SM.blockDiff|VP), data=df) – this should be correct, right?
Using the uncentered blockID (num vector ranging from 0-5) only returns the warning about the missing centering but not the missing cells warning.

[henrik]

The warning does not seem to be emitted from afex, but from the package performing the tests (lmerTest or car), so I cannot really explain it. Try using a different method (e.g., method = "S") and hopefully this works.

[Dom42]

Thanks for pointing me to the right direction: it seems to be a specific error/warning of lmerTest when used on centered, continuous predictor variables (so exactly the case here). It could therefore be replicated with afex using either method ("S" or "KR"). Source: Issue on Github

Apparently this has not been resolved yet, however it seems a bit unclear to me, if this is a truly erroneous warning or something that points to a computational error deeper within the lmerTest code.

Nevertheless, thanks again for your help and time! Keep up your great work and support!

[Author]

Posts