Invariance

An EGA structure is only as useful as its comparability across the people you’d like to apply it to. invariance tests whether a network structure — and the strength of each item’s connections within it — holds up across groups, using a permutation-based approach at both the configural (does the same structure emerge?) and metric (do items relate to their dimension similarly?) level.

We’ll return to the wmt2 matrix reasoning data, this time comparing the two groups recorded in its sex column.

# Load {EGAnet}
library(EGAnet)

# Remove missing sex values
complete_sex <- !is.na(wmt2$sex)
wmt <- wmt2[complete_sex, 7:24]
groups <- as.character(wmt2$sex[complete_sex])

table(groups)

groups
Female   Male 
   717    449 

Testing Invariance

wmt_invariance <- invariance(
  data = wmt, groups = groups,
  iter = 500, ncores = 4, seed = 1
)

# Print results
wmt_invariance

Invariance Results 

Comparison: Female vs Male 
      Membership Difference     p  p_BH sig     Direction
wmt1           1      0.019 0.838 0.840                  
wmt2           1     -0.147 0.196 0.457                  
wmt3           1      0.074 0.402 0.704                  
wmt5           1      0.123 0.152 0.426                  
wmt7           2      0.220 0.016 0.112   * Female > Male
wmt8           2      0.196 0.024 0.112   * Female > Male
wmt9           2      0.062 0.522 0.812                  
wmt12          2      0.042 0.640 0.815                  
wmt13          2     -0.013 0.840 0.840                  
wmt14          2     -0.097 0.274 0.548                  
wmt15          2     -0.040 0.610 0.815                  
wmt16          2     -0.186 0.018 0.112   * Female < Male
wmt17          2     -0.175 0.050 0.175   * Female < Male
wmt18          2     -0.020 0.764 0.840                  
----
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 'n.s.' 1

Interpreting the Output

Configural invariance — whether the same community structure emerges in both groups — held for 14 of the 18 items, recovering the same two dimensions seen in the EGA workflow. The remaining four items didn’t replicate closely enough across groups to be carried into the metric stage, and are simply left out of the table above rather than penalized.

At the metric level, invariance permutes group membership to build a null distribution for each item’s loading difference, so the p column reflects how unusual the observed difference would be if sex had no bearing on it at all. Four items clear the uncorrected 0.05 threshold: wmt7 and wmt8 load more strongly for Female respondents, and wmt16 and wmt17 load more strongly for Male respondents.

The p_BH column applies a Benjamini-Hochberg correction for testing 14 items at once — and after that correction, none of the four remain significant (the smallest corrected p is 0.112). This is the more trustworthy conclusion: with 14 simultaneous tests, a few nominally-significant differences are expected by chance alone, and the corrected column is what protects against reading too much into them. Practically, this WMT-2 structure holds up well across sex — both the dimensions themselves and the item loadings within them.