Significance testing correlations If only y1 is provided a t-test is computed, between the absolute error from yhat1-y1 and yhat2-y1.

If y2 is provided a bootstrapped procedure is used to compare the correlations between y1 and yhat1 versus y2 and yhat2. This is achieved by creating two distributions of correlations using bootstrapping; and then finally compute the distributions overlap.

textPredictTest(
  y1,
  y2 = NULL,
  yhat1,
  yhat2,
  paired = TRUE,
  bootstraps_times = 10000,
  seed = 6134,
  ...
)

Arguments

y1

The observed scores (i.e., what was used to predict when training a model).

y2

The second observed scores (default = NULL; i.e., for when comparing models that are predicting different outcomes. In this case a bootstrap procedure is used to create two distributions of correlations that are compared (see description above).

yhat1

The predicted scores from model 1.

yhat2

The predicted scores from model 2 that will be compared with model 1.

paired

Paired test or not in stats::t.test (default TRUE).

bootstraps_times

Number of bootstraps (when providing y2).

seed

Set different seed.

...

Settings from stats::t.test or overlapping::overlap (e.g., plot = TRUE).

Value

Comparison of correlations either a t-test or the overlap of a bootstrapped procedure (see $OV).

See also

see textTrain textPredict

Examples

# Example random data
y1 <- runif(10)
yhat1 <- runif(10)
y2 <- runif(10)
yhat2 <- runif(10)

boot_test <- textPredictTest(y1, yhat1, y2, yhat2, bootstraps_times = 10)