Interpretation of Logistic Mixed Interactions

[Trey]

I’m working on what was intended to be a 3x2x3 Repeated-Measures ANOVA. We realized, however that mixed models would handle the complexity better, though we’re less familiar with them.

Could someone help explain one of the interactions and how to discuss it?

The response variable is “Correct”, a binary measure of success, with 0 being incorrect and 1 being correct. Each subject gave 14 responses which were intending, initially, to analyze as proportions. However, as logistic mixed-models can handle binary responses, we went with the raw binary inputs.

The treatment independent variables are Method (3 factors), Levels(3 factors), and MATB (2 factors). SubjectID indicates the individual subject labels.

The interpretation of the initial analysis is pretty straightforward to those used to ANOVA.

> require(afex) package:
> full_mixed = mixed(Correct ~ Method * Levels * MATB + (1 | SubjectID),family = binomial, data= data2clean)
Mixed Model Anova Table (Type 3 tests, LRT-method)

Model: Correct ~ Method * Levels * MATB + (1 | Subject)
Data: data2clean
Df full model: 19
              Effect df      Chisq p.value
1             Method  2 174.54 ***  <.0001
2             Levels  2 135.95 ***  <.0001
3               MATB  1       0.01     .94
4      Method:Levels  4  21.05 ***   .0003
5        Method:MATB  2       0.77     .68
6        Levels:MATB  2     4.67 +     .10
7 Method:Levels:MATB  4       1.69     .79
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘+’ 0.1 ‘ ’ 1

Where I’m uncertain is how to analyze the ‘Fixed Effects’ output:

 > summary(full_mixed)
Generalized linear mixed model fit by maximum likelihood (Laplace Approximation) ['glmerMod']
 Family: binomial  ( logit )
Formula: Correct ~ Method * Levels * MATB + (1 | Subject)
   Data: data2clean

     AIC      BIC   logLik deviance df.resid 
  4603.3   4725.2  -2282.6   4565.3     4511 

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-4.9059 -0.6839  0.3544  0.5920  1.8263 

Random effects:
 Groups  Name        Variance Std.Dev.
 Subject (Intercept) 0.756    0.8695  
Number of obs: 4530, groups:  Subject, 18

Fixed effects:
                       Estimate Std. Error z value Pr(>|z|)    
<strong>(Intercept)            1.181444   0.208896   5.656 1.55e-08 ***
Method1               -0.329280   0.051814  -6.355 2.08e-10 ***
Method2               -0.396058   0.051918  -7.629 2.37e-14 ***
Levels1                0.608708   0.057102  10.660  < 2e-16 ***
Levels2               -0.161221   0.052849  -3.051  0.00228 ** 
MATB1                  0.002876   0.037927   0.076  0.93955    
Method1:Levels1       -0.068669   0.076390  -0.899  0.36869    
Method2:Levels1        0.183151   0.077251   2.371  0.01775 *  
Method1:Levels2        0.182732   0.072137   2.533  0.01131 *  
Method2:Levels2       -0.317815   0.071332  -4.455 8.37e-06 ***
Method1:MATB1          0.012756   0.051708   0.247  0.80515    
Method2:MATB1          0.035272   0.051646   0.683  0.49464    
Levels1:MATB1         -0.121100   0.056619  -2.139  0.03245 *  
Levels2:MATB1          0.050910   0.052799   0.964  0.33493    
Method1:Levels1:MATB1  0.042346   0.076382   0.554  0.57930    
Method2:Levels1:MATB1 -0.012933   0.077183  -0.168  0.86693    
Method1:Levels2:MATB1 -0.033750   0.072194  -0.467  0.64015    
Method2:Levels2:MATB1  0.073972   0.071349   1.037  0.29985    </strong>
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

So, basically, if I’m looking at the output, 1.) I’d assume that the significance of Levels1:MATB1 should be ignored because the main effect for MATB is not present, and 2.) How do I explain what Method 2: Levels 2 means? (I’m pretty certain that I also can’t do a confidence interval because of the logistic transformation.)

• This topic was modified 6 years, 3 months ago by Trey.

[henrik]

To be able to provide a more useful answer to your question, I have to ask you why are you looking at the fixed effects in the first place? What is your goal with this?

One of the design goals of afex is to hide the fixed effects from the user, as they are not particularly useful in cases such as yours (i.e., factorial designs with factors with more than two levels). If your goal is to test specific hypotheses on the level of the factor levels do so using the functionality provided by the lsmeans (or from afex v. 0.19.1 onwards using emmeans). This is described in some detail in the ANOVA vignette and applies in exactly the same way to mixed models as to ANOVA models. For more details on how to set up contrasts (or post-hoc tests) you are interested in, have a lookj at the great vignettes of emmeans: https://cran.r-project.org/package=emmeans

emmeans also automatically calculates CIs for you. This is certainly possible, but needs to be done on the unconstrained (i.e., logistic) scale. Only after calculating the CIs can the values be transformed on the response scale (i.e., probabilities). For example in your case, compare:

emmeans(full_mixed, c("Method", "Levels"))
emmeans(full_mixed, c("Method", "Levels"), type = "response")

Note that you can simply specify any type of contrasts on these estimated marginal means. Several examples of this for logistic mixed models are provided in the supplemental material of a recent paper of mine (https://doi.org/10.1016/j.cogpsych.2017.09.002), which can be found at: https://osf.io/3uajq/

One final thought, as discussed in length by Barr, Levy, Scheepers, and Tily (2013), you most likely do not want a random-intercept only model. Anything they write in this paper applies in the same manner to logistic mixed model. The recommended maximal model has random-slopes for all within-subjects factors (e.g., Correct ~ Method * Levels * MATB + (Method * Levels * MATB | SubjectID)). The model you estimated likely produces anti-conservatie results.

All I have discussed here is also discussed in more length in our recent chapter: http://singmann.org/download/publications/singmann_kellen-introduction-mixed-models.pdf
Again, this deals with linear mixed models, but the parts relevant to your question apply in the same manner to generalized linear mixed models.

[Trey]

First of all, thank you for the detailed response. I had also seen Barr, Levy, Scheepers, and Tily, but was confused as to how it should directly apply to my analysis, so I appreciate the clarification and recommendatoin.

The study’s goal is to assess different interface types (coded as Method, with three conditions) at different cognitive workloads (MATB) and different #’s of signals (3, 4, and 5) on the number of correct responses. So, yes, I’m mainly interested in determining differences at the level of factors.

I was mainly working through Winter’s guide for Mixed Models, prior to this. I’m still digging through the references and may respond back with clarifications, but need to review the materials closely to make sure that I’m asking good questions.

• This reply was modified 6 years, 3 months ago by Trey.

[Author]

Posts