Home forums Mixed Models Interpretation of Logistic Mixed Interactions

Tagged: 

  • This topic has 2 replies, 2 voices, and was last updated 4 years ago by Trey.
Viewing 2 reply threads
  • Author
    Posts
    • #175
      Trey
      Participant

      I’m working on what was intended to be a 3x2x3 Repeated-Measures ANOVA. We realized, however that mixed models would handle the complexity better, though we’re less familiar with them.

      Could someone help explain one of the interactions and how to discuss it?

      The response variable is “Correct”, a binary measure of success, with 0 being incorrect and 1 being correct. Each subject gave 14 responses which were intending, initially, to analyze as proportions. However, as logistic mixed-models can handle binary responses, we went with the raw binary inputs.

      The treatment independent variables are Method (3 factors), Levels(3 factors), and MATB (2 factors). SubjectID indicates the individual subject labels.

      The interpretation of the initial analysis is pretty straightforward to those used to ANOVA.

      > require(afex) package:
      > full_mixed = mixed(Correct ~ Method * Levels * MATB + (1 | SubjectID),family = binomial, data= data2clean)
      Mixed Model Anova Table (Type 3 tests, LRT-method)
      
      Model: Correct ~ Method * Levels * MATB + (1 | Subject)
      Data: data2clean
      Df full model: 19
                    Effect df      Chisq p.value
      1             Method  2 174.54 ***  <.0001
      2             Levels  2 135.95 ***  <.0001
      3               MATB  1       0.01     .94
      4      Method:Levels  4  21.05 ***   .0003
      5        Method:MATB  2       0.77     .68
      6        Levels:MATB  2     4.67 +     .10
      7 Method:Levels:MATB  4       1.69     .79
      ---
      Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘+’ 0.1 ‘ ’ 1
      

      Where I’m uncertain is how to analyze the ‘Fixed Effects’ output:

       > summary(full_mixed)
      Generalized linear mixed model fit by maximum likelihood (Laplace Approximation) ['glmerMod']
       Family: binomial  ( logit )
      Formula: Correct ~ Method * Levels * MATB + (1 | Subject)
         Data: data2clean
      
           AIC      BIC   logLik deviance df.resid 
        4603.3   4725.2  -2282.6   4565.3     4511 
      
      Scaled residuals: 
          Min      1Q  Median      3Q     Max 
      -4.9059 -0.6839  0.3544  0.5920  1.8263 
      
      Random effects:
       Groups  Name        Variance Std.Dev.
       Subject (Intercept) 0.756    0.8695  
      Number of obs: 4530, groups:  Subject, 18
      
      Fixed effects:
                             Estimate Std. Error z value Pr(>|z|)    
      <strong>(Intercept)            1.181444   0.208896   5.656 1.55e-08 ***
      Method1               -0.329280   0.051814  -6.355 2.08e-10 ***
      Method2               -0.396058   0.051918  -7.629 2.37e-14 ***
      Levels1                0.608708   0.057102  10.660  < 2e-16 ***
      Levels2               -0.161221   0.052849  -3.051  0.00228 ** 
      MATB1                  0.002876   0.037927   0.076  0.93955    
      Method1:Levels1       -0.068669   0.076390  -0.899  0.36869    
      Method2:Levels1        0.183151   0.077251   2.371  0.01775 *  
      Method1:Levels2        0.182732   0.072137   2.533  0.01131 *  
      Method2:Levels2       -0.317815   0.071332  -4.455 8.37e-06 ***
      Method1:MATB1          0.012756   0.051708   0.247  0.80515    
      Method2:MATB1          0.035272   0.051646   0.683  0.49464    
      Levels1:MATB1         -0.121100   0.056619  -2.139  0.03245 *  
      Levels2:MATB1          0.050910   0.052799   0.964  0.33493    
      Method1:Levels1:MATB1  0.042346   0.076382   0.554  0.57930    
      Method2:Levels1:MATB1 -0.012933   0.077183  -0.168  0.86693    
      Method1:Levels2:MATB1 -0.033750   0.072194  -0.467  0.64015    
      Method2:Levels2:MATB1  0.073972   0.071349   1.037  0.29985    </strong>
      ---
      Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
      
      

      So, basically, if I’m looking at the output, 1.) I’d assume that the significance of Levels1:MATB1 should be ignored because the main effect for MATB is not present, and 2.) How do I explain what Method 2: Levels 2 means? (I’m pretty certain that I also can’t do a confidence interval because of the logistic transformation.)

      • This topic was modified 4 years ago by Trey.
    • #177
      henrik
      Keymaster

      To be able to provide a more useful answer to your question, I have to ask you why are you looking at the fixed effects in the first place? What is your goal with this?

      One of the design goals of afex is to hide the fixed effects from the user, as they are not particularly useful in cases such as yours (i.e., factorial designs with factors with more than two levels). If your goal is to test specific hypotheses on the level of the factor levels do so using the functionality provided by the lsmeans (or from afex v. 0.19.1 onwards using emmeans). This is described in some detail in the ANOVA vignette and applies in exactly the same way to mixed models as to ANOVA models. For more details on how to set up contrasts (or post-hoc tests) you are interested in, have a lookj at the great vignettes of emmeans: https://cran.r-project.org/package=emmeans

      emmeans also automatically calculates CIs for you. This is certainly possible, but needs to be done on the unconstrained (i.e., logistic) scale. Only after calculating the CIs can the values be transformed on the response scale (i.e., probabilities). For example in your case, compare:

      emmeans(full_mixed, c("Method", "Levels"))
      emmeans(full_mixed, c("Method", "Levels"), type = "response")

      Note that you can simply specify any type of contrasts on these estimated marginal means. Several examples of this for logistic mixed models are provided in the supplemental material of a recent paper of mine (https://doi.org/10.1016/j.cogpsych.2017.09.002), which can be found at: https://osf.io/3uajq/

      One final thought, as discussed in length by Barr, Levy, Scheepers, and Tily (2013), you most likely do not want a random-intercept only model. Anything they write in this paper applies in the same manner to logistic mixed model. The recommended maximal model has random-slopes for all within-subjects factors (e.g., Correct ~ Method * Levels * MATB + (Method * Levels * MATB | SubjectID)). The model you estimated likely produces anti-conservatie results.

      All I have discussed here is also discussed in more length in our recent chapter: http://singmann.org/download/publications/singmann_kellen-introduction-mixed-models.pdf
      Again, this deals with linear mixed models, but the parts relevant to your question apply in the same manner to generalized linear mixed models.

    • #182
      Trey
      Participant

      First of all, thank you for the detailed response. I had also seen Barr, Levy, Scheepers, and Tily, but was confused as to how it should directly apply to my analysis, so I appreciate the clarification and recommendatoin.

      The study’s goal is to assess different interface types (coded as Method, with three conditions) at different cognitive workloads (MATB) and different #’s of signals (3, 4, and 5) on the number of correct responses. So, yes, I’m mainly interested in determining differences at the level of factors.

      I was mainly working through Winter’s guide for Mixed Models, prior to this. I’m still digging through the references and may respond back with clarifications, but need to review the materials closely to make sure that I’m asking good questions.

      • This reply was modified 4 years ago by Trey.
Viewing 2 reply threads
  • You must be logged in to reply to this topic.