The hmetad package is designed to fit the meta-d’ model
for confidence ratings (Maniscalco &
Lau, 2012, 2014). The
hmetad package uses a Bayesian modeling approach, building
on and superseding previous development of the Hmeta-d toolbox (Fleming, 2017). A key advance is
implementation as a custom family in the brms package, which itself
provides a friendly interface to the probabilistic programming language
Stan.
This provides major benefits:
R
formulasbrms (e.g.,
tidybayes, ggdist, bayesplot,
loo, posterior, bridgesampling,
bayestestR)hmetad is available via CRAN and can be installed
using:
install.packages("hmetad")Alternatively, you can install the development version of
hmetad from GitHub with:
# install.packages("pak")
pak::pak("metacoglab/hmetad")Let’s say you have some data from a binary decision task with ordinal confidence ratings:
#> # A tibble: 1,000 × 5
#> trial stimulus response correct confidence
#> <int> <int> <int> <int> <int>
#> 1 1 1 0 0 1
#> 2 2 0 0 1 2
#> 3 3 1 1 1 3
#> 4 4 0 1 0 2
#> 5 5 0 0 1 3
#> 6 6 0 0 1 3
#> 7 7 0 0 1 3
#> 8 8 0 1 0 3
#> 9 9 1 0 0 2
#> 10 10 0 1 0 3
#> # ℹ 990 more rowsYou can fit an intercepts-only meta-d’ model using
fit_metad:
library(hmetad)
m <- fit_metad(N ~ 1,
data = d,
prior = prior(normal(0, 1), class = Intercept) +
set_prior("normal(0, 1)", class = c("dprime", "c", metac2_parameters(K = 4)))
)#> Family: metad__4__normal__absolute__multinomial
#> Links: mu = log
#> Formula: N ~ 1
#> Data: data.aggregated (Number of observations: 1)
#> Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
#> total post-warmup draws = 4000
#>
#> Regression Coefficients:
#> Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
#> Intercept 0.09 0.15 -0.21 0.37 1.00 3361 2972
#>
#> Further Distributional Parameters:
#> Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
#> dprime 0.96 0.08 0.81 1.13 1.00 4198 3420
#> c -0.03 0.04 -0.11 0.05 1.00 3956 2995
#> metac2zero1diff 0.52 0.04 0.45 0.59 1.00 4969 2878
#> metac2zero2diff 0.45 0.04 0.38 0.53 1.00 4413 3006
#> metac2zero3diff 0.55 0.05 0.45 0.65 1.00 5463 2815
#> metac2one1diff 0.55 0.04 0.48 0.62 1.00 5035 3158
#> metac2one2diff 0.53 0.04 0.45 0.62 1.00 5315 3096
#> metac2one3diff 0.55 0.05 0.45 0.65 1.00 5463 2880
#>
#> Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
#> and Tail_ESS are effective sample size measures, and Rhat is the potential
#> scale reduction factor on split chains (at convergence, Rhat = 1).Now let’s say you have a more complicated design, such as a within-participant manipulation:
#> # A tibble: 5,000 × 7
#> # Groups: participant, condition [50]
#> participant condition trial stimulus response correct confidence
#> <int> <int> <int> <int> <int> <int> <int>
#> 1 1 1 1 1 1 1 4
#> 2 1 1 2 1 1 1 1
#> 3 1 1 3 1 1 1 2
#> 4 1 1 4 0 0 1 2
#> 5 1 1 5 0 1 0 1
#> 6 1 1 6 0 0 1 4
#> 7 1 1 7 0 0 1 4
#> 8 1 1 8 1 1 1 2
#> 9 1 1 9 0 0 1 1
#> 10 1 1 10 0 0 1 4
#> # ℹ 4,990 more rowsTo account for the repeated measures in this design, you can simply adjust the formula to include participant-level effects:
m <- fit_metad(
bf(
N ~ condition + (condition | participant),
dprime + c +
metac2zero1diff + metac2zero2diff + metac2zero3diff +
metac2one1diff + metac2one2diff + metac2one3diff ~
condition + (condition | participant)
),
data = d, init = "0",
prior = prior(normal(0, 1)) +
set_prior("normal(0, 1)", dpar = c("dprime", "c", metac2_parameters(K = 4)))
)#> Family: metad__4__normal__absolute__multinomial
#> Links: mu = log; dprime = identity; c = identity; metac2zero1diff = log; metac2zero2diff = log; metac2zero3diff = log; metac2one1diff = log; metac2one2diff = log; metac2one3diff = log
#> Formula: N ~ condition + (condition | participant)
#> dprime ~ condition + (condition | participant)
#> c ~ condition + (condition | participant)
#> metac2zero1diff ~ condition + (condition | participant)
#> metac2zero2diff ~ condition + (condition | participant)
#> metac2zero3diff ~ condition + (condition | participant)
#> metac2one1diff ~ condition + (condition | participant)
#> metac2one2diff ~ condition + (condition | participant)
#> metac2one3diff ~ condition + (condition | participant)
#> Data: data.aggregated (Number of observations: 50)
#> Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
#> total post-warmup draws = 4000
#>
#> Multilevel Hyperparameters:
#> ~participant (Number of levels: 25)
#> Estimate Est.Error
#> sd(Intercept) 0.37 0.23
#> sd(condition) 0.21 0.15
#> sd(dprime_Intercept) 1.35 0.23
#> sd(dprime_condition) 0.89 0.15
#> sd(c_Intercept) 1.30 0.20
#> sd(c_condition) 0.89 0.14
#> sd(metac2zero1diff_Intercept) 0.20 0.19
#> sd(metac2zero1diff_condition) 0.15 0.13
#> sd(metac2zero2diff_Intercept) 0.08 0.07
#> sd(metac2zero2diff_condition) 0.05 0.04
#> sd(metac2zero3diff_Intercept) 0.08 0.07
#> sd(metac2zero3diff_condition) 0.05 0.04
#> sd(metac2one1diff_Intercept) 0.17 0.18
#> sd(metac2one1diff_condition) 0.09 0.10
#> sd(metac2one2diff_Intercept) 0.18 0.18
#> sd(metac2one2diff_condition) 0.12 0.12
#> sd(metac2one3diff_Intercept) 0.12 0.10
#> sd(metac2one3diff_condition) 0.08 0.06
#> cor(Intercept,condition) -0.35 0.55
#> cor(dprime_Intercept,dprime_condition) -0.96 0.02
#> cor(c_Intercept,c_condition) -0.96 0.02
#> cor(metac2zero1diff_Intercept,metac2zero1diff_condition) -0.57 0.54
#> cor(metac2zero2diff_Intercept,metac2zero2diff_condition) -0.29 0.58
#> cor(metac2zero3diff_Intercept,metac2zero3diff_condition) -0.32 0.57
#> cor(metac2one1diff_Intercept,metac2one1diff_condition) -0.47 0.57
#> cor(metac2one2diff_Intercept,metac2one2diff_condition) -0.50 0.56
#> cor(metac2one3diff_Intercept,metac2one3diff_condition) -0.33 0.56
#> l-95% CI u-95% CI Rhat
#> sd(Intercept) 0.03 0.92 1.00
#> sd(condition) 0.01 0.60 1.01
#> sd(dprime_Intercept) 0.97 1.86 1.01
#> sd(dprime_condition) 0.64 1.22 1.00
#> sd(c_Intercept) 0.98 1.77 1.01
#> sd(c_condition) 0.67 1.24 1.01
#> sd(metac2zero1diff_Intercept) 0.00 0.64 1.01
#> sd(metac2zero1diff_condition) 0.00 0.46 1.01
#> sd(metac2zero2diff_Intercept) 0.00 0.26 1.00
#> sd(metac2zero2diff_condition) 0.00 0.16 1.00
#> sd(metac2zero3diff_Intercept) 0.00 0.26 1.00
#> sd(metac2zero3diff_condition) 0.00 0.16 1.00
#> sd(metac2one1diff_Intercept) 0.00 0.66 1.00
#> sd(metac2one1diff_condition) 0.00 0.37 1.00
#> sd(metac2one2diff_Intercept) 0.01 0.64 1.01
#> sd(metac2one2diff_condition) 0.00 0.43 1.01
#> sd(metac2one3diff_Intercept) 0.01 0.37 1.00
#> sd(metac2one3diff_condition) 0.00 0.23 1.00
#> cor(Intercept,condition) -0.97 0.88 1.01
#> cor(dprime_Intercept,dprime_condition) -0.99 -0.91 1.01
#> cor(c_Intercept,c_condition) -0.98 -0.91 1.01
#> cor(metac2zero1diff_Intercept,metac2zero1diff_condition) -1.00 0.81 1.01
#> cor(metac2zero2diff_Intercept,metac2zero2diff_condition) -0.99 0.91 1.00
#> cor(metac2zero3diff_Intercept,metac2zero3diff_condition) -0.99 0.88 1.00
#> cor(metac2one1diff_Intercept,metac2one1diff_condition) -1.00 0.84 1.00
#> cor(metac2one2diff_Intercept,metac2one2diff_condition) -1.00 0.85 1.00
#> cor(metac2one3diff_Intercept,metac2one3diff_condition) -0.98 0.89 1.00
#> Bulk_ESS Tail_ESS
#> sd(Intercept) 1038 1395
#> sd(condition) 497 615
#> sd(dprime_Intercept) 1237 2085
#> sd(dprime_condition) 1198 2025
#> sd(c_Intercept) 474 677
#> sd(c_condition) 430 387
#> sd(metac2zero1diff_Intercept) 455 1503
#> sd(metac2zero1diff_condition) 409 1210
#> sd(metac2zero2diff_Intercept) 1982 2046
#> sd(metac2zero2diff_condition) 1934 1606
#> sd(metac2zero3diff_Intercept) 2139 2019
#> sd(metac2zero3diff_condition) 1270 706
#> sd(metac2one1diff_Intercept) 677 1368
#> sd(metac2one1diff_condition) 824 1303
#> sd(metac2one2diff_Intercept) 688 1714
#> sd(metac2one2diff_condition) 575 1858
#> sd(metac2one3diff_Intercept) 1779 2015
#> sd(metac2one3diff_condition) 1124 1708
#> cor(Intercept,condition) 1032 1778
#> cor(dprime_Intercept,dprime_condition) 1125 1767
#> cor(c_Intercept,c_condition) 616 1643
#> cor(metac2zero1diff_Intercept,metac2zero1diff_condition) 570 2153
#> cor(metac2zero2diff_Intercept,metac2zero2diff_condition) 2919 2624
#> cor(metac2zero3diff_Intercept,metac2zero3diff_condition) 2425 2209
#> cor(metac2one1diff_Intercept,metac2one1diff_condition) 1071 2089
#> cor(metac2one2diff_Intercept,metac2one2diff_condition) 983 2289
#> cor(metac2one3diff_Intercept,metac2one3diff_condition) 1866 2267
#>
#> Regression Coefficients:
#> Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS
#> Intercept -0.34 0.24 -0.83 0.13 1.00 1465
#> dprime_Intercept 1.21 0.30 0.60 1.77 1.01 1131
#> c_Intercept 0.12 0.26 -0.38 0.64 1.01 417
#> metac2zero1diff_Intercept -0.82 0.13 -1.09 -0.57 1.00 4220
#> metac2zero2diff_Intercept -0.88 0.13 -1.13 -0.62 1.00 5486
#> metac2zero3diff_Intercept -1.18 0.14 -1.47 -0.90 1.00 3378
#> metac2one1diff_Intercept -1.19 0.14 -1.48 -0.92 1.00 3432
#> metac2one2diff_Intercept -0.91 0.14 -1.20 -0.63 1.00 2572
#> metac2one3diff_Intercept -1.26 0.15 -1.57 -0.96 1.00 4428
#> condition 0.25 0.16 -0.08 0.54 1.00 1387
#> dprime_condition -0.14 0.19 -0.51 0.25 1.01 1125
#> c_condition -0.02 0.17 -0.37 0.31 1.01 442
#> metac2zero1diff_condition -0.11 0.09 -0.28 0.06 1.00 4038
#> metac2zero2diff_condition -0.13 0.08 -0.30 0.03 1.00 5083
#> metac2zero3diff_condition 0.11 0.09 -0.05 0.29 1.00 4423
#> metac2one1diff_condition 0.10 0.09 -0.07 0.27 1.00 3650
#> metac2one2diff_condition -0.06 0.09 -0.24 0.13 1.00 2197
#> metac2one3diff_condition 0.17 0.10 -0.02 0.37 1.00 4381
#> Tail_ESS
#> Intercept 623
#> dprime_Intercept 1845
#> c_Intercept 556
#> metac2zero1diff_Intercept 2332
#> metac2zero2diff_Intercept 2687
#> metac2zero3diff_Intercept 2286
#> metac2one1diff_Intercept 1896
#> metac2one2diff_Intercept 1228
#> metac2one3diff_Intercept 2409
#> condition 526
#> dprime_condition 1771
#> c_condition 593
#> metac2zero1diff_condition 2132
#> metac2zero2diff_condition 2759
#> metac2zero3diff_condition 2967
#> metac2one1diff_condition 2255
#> metac2one2diff_condition 1370
#> metac2one3diff_condition 2362
#>
#> Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
#> and Tail_ESS are effective sample size measures, and Rhat is the potential
#> scale reduction factor on split chains (at convergence, Rhat = 1).