Predict New Data based on Network

General function to compute a network's predictive power on new data, following Haslbeck and Waldorp (2018) and Williams and Rodriguez (2022)

This implementation is different from the predictability in the mgm package (Haslbeck), which is based on (regularized) regression. This implementation uses the network directly, converting the partial correlations into an implied precision (inverse covariance) matrix. See Details for more information

Usage

network.predictability(network, original.data, newdata, ordinal.categories = 7)

Arguments

network

Matrix or data frame. A partial correlation network

original.data

Matrix or data frame. Must consist only of variables to be used to estimate the network. See Examples

newdata

Matrix or data frame. Must consist of the same variables in the same order as original.data. See Examples

ordinal.categories

Numeric (length = 1). Up to the number of categories before a variable is considered continuous. Defaults to 7 categories before 8 is considered continuous

Value

Returns a list containing:

predictions

Predicted values of newdata based on the network

betas

Beta coefficients derived from the network

results

Performance metrics for each variable in newdata

Details

This implementation of network predictability proceeds in several steps with important assumptions:

1. Network was estimated using (partial) correlations (not regression like the mgm package!)

2. Original data that was used to estimate the network in 1. is necessary to apply the original scaling to the new data

3. (Linear) regression-like coefficients are obtained by reserve engineering the inverse covariance matrix using the network's partial correlations (i.e., by setting the diagonal of the network to -1 and computing the inverse of the opposite signed partial correlation matrix; see EGAnet:::pcor2inv)

4. Predicted values are obtained by matrix multiplying the new data with these coefficients

5. Dichotomous and polytomous data are given categorical values based on the original data's thresholds and these thresholds are used to convert the continuous predicted values into their corresponding categorical values

6. Evaluation metrics:

• dichotomous — "Accuracy" or the percent correctly predicted for the 0s and 1s and "Kappa" or Cohen's Kappa (see cite)

• polytomous — "Linear Kappa" or linearly weighted Kappa and "Krippendorff's alpha" (see cite)

• continuous — R-squared ("R2") and root mean square error ("RMSE")

References

Original Implementation of Node Predictability
Haslbeck, J. M., & Waldorp, L. J. (2018). How well do network models predict observations? On the importance of predictability in network models. Behavior Research Methods, 50(2), 853–861.

Derivation of Regression Coefficients Used (Formula 3)
Williams, D. R., & Rodriguez, J. E. (2022). Why overfitting is not (usually) a problem in partial correlation networks. Psychological Methods, 27(5), 822–840.

Cohen's Kappa
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37-46.

Cohen, J. (1968). Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. Psychological Bulletin, 70(4), 213-220.

Krippendorff's alpha
Krippendorff, K. (2013). Content analysis: An introduction to its methodology (3rd ed.). Thousand Oaks, CA: Sage.

Author

Hudson Golino <hfg9s at virginia.edu> and Alexander P. Christensen <alexpaulchristensen@gmail.com>

Examples

# Load data
wmt <- wmt2[,7:24]

# Set seed (to reproduce results)
set.seed(42)

# Split data
training <- sample(
  1:nrow(wmt), round(nrow(wmt) * 0.80) # 80/20 split
)

# Set splits
wmt_train <- wmt[training,]
wmt_test <- wmt[-training,]

# EBICglasso (default for EGA functions)
glasso_network <- network.estimation(
  data = wmt_train, model = "glasso"
)

# Check predictability
network.predictability(
  network = glasso_network, original.data = wmt_train,
  newdata = wmt_test
)
#>          R2   MAE
#> wmt1  0.315 0.253
#> wmt2  0.548 0.135
#> wmt3  0.476 0.198
#> wmt4  0.318 0.300
#> wmt5  0.400 0.249
#> wmt6  0.403 0.262
#> wmt7  0.362 0.291
#> wmt8  0.278 0.321
#> wmt9  0.552 0.232
#> wmt10 0.340 0.278
#> wmt11 0.220 0.270
#> wmt12 0.193 0.287
#> wmt13 0.425 0.270
#> wmt14 0.252 0.325
#> wmt15 0.166 0.291
#> wmt16 0.260 0.287
#> wmt17 0.194 0.203
#> wmt18 0.372 0.236