Win odds, win ratio, and net benefit

Setup

Load the package hce and check the version

library(hce)
packageVersion("hce")
#> [1] '0.6.0'

For citing the package run citation("hce") (Samvel B. Gasparyan 2024).

Terminology

The concept of win probability for the binary and continuous outcomes has been described in the paper by Buyse (2010) as “proportion in favor of treatment” (see also Rauch et al. (2014)), while in Verbeeck et al. (2021) it is called “probabilistic index”.

The concept of “win ratio” was introduced in Pocock et al. (2012), which, unlike the win odds, does not account for ties, whereas the win odds is the odds of winning, following G. Dong et al. (2020) (see also Peng (2020); Brunner, Vandemeulebroecke, and Mütze (2021); Samvel B. Gasparyan, Kowalewski, et al. (2021)). The same statistic was named as Mann-Whitney odds in O’Brien and Castelloe (2006). In Samvel B. Gasparyan, Folkvaljon, et al. (2021) the “win ratio” was used as a general term and included ties in the definition. Gaohong Dong et al. (2022) suggested to consider win ratio, win odds, and net benefit together as win statistics.

The concept of winning for the active group is the same as concordance for the active group. For two variables, \(X\) and \(Y,\) this pair is concordant if the observation with the larger value of \(X\) also has the better value of \(Y\) (Agresti 2013). If \(X\) indicates the treatment group with the value 1 for the active group and the value 0 for the control group, and the variable \(Y\) is the ordinal value for the analysis, then concordance means that a patient with the active treatment has a better value than a patient with the control, while the discordance means the patient in the active group has a worse value than the control patient. Therefore, the win ratio is the total number of concordances divided by the total number of discordances. The win ratio can be obtained from the Goodman-Kruskal gamma (Kruskal and Goodman 1954), \(G\), as follows \(WR=(1+G)/(1-G)\). The net benefit is Somers’ D C/R (Somers 1962), while the win odds is the Mann-Whitney odds (Mann and Whitney 1947). Estimation of win statistics in the absence of censoring can be done using the theory of U-statistics (Hoeffding 1948).

Definitions

Two treatment groups are compared using an ordinal endpoint and each comparison results in a win, loss, or a tie for the patient in the active group compared to a patient in the placebo group. All possible (overall) combinations are denoted by \(O\), with \(W\) denoted the total wins for the active group, \(L\) total losses, and \(T\) the total ties, so that \(O=W+L+T.\) Then the following quantities are called win statistics

  • Win probability defined as \(WP=\frac{W+0.5T}{O}\), that is, the total number of wins, added half of the total number of ties, divided by the overall number of comparisons.
  • Number needed to treat defined as \(NNT=\frac{1}{2WP-1}=\frac{O}{W-L}\) (rounded up to the nearest natural number for interpretation).
  • Win ratio defined as \(WR=\frac{W}{L}\).
  • Win odds defined as \(WO=\frac{W+0.5T}{L+0.5T} = \frac{WP}{1-WP}\).
  • Net Benefit defined as \(NB=\frac{W - L}{O}=2WP-1=\frac{1}{NNT}\).

Given the overall number of comparisons \(O,\) the win proportion \(WP\) and the win ratio \(WR\), it is possible to find the total number of wins and losses. \[\begin{align*} &L = O*\frac{2WP-1}{WR-1},\nonumber\\ &W = WR*L = WR*O*\frac{2WP-1}{WR-1},\nonumber\\ &T=O-W-L = O*\left[1 - (WR+1)\frac{2WP-1}{WR-1}\right]. \end{align*}\]

The function propWINS() implements the formula above

args("propWINS")
#> function (WO, WR, Overall = 1, alpha = NULL, N = NULL) 
#> NULL
propWINS(WO = 1.5, WR = 2)
#>   WIN LOSS TIE TOTAL WR  WO
#> 1   0    0   1     1  2 1.5

Suppose there are \(n_1=120\) patients in the placebo group and \(n_2=150\) in the active group. If win ratio is 1.5 and the win odds is 1.25, then the number of wins and losses for the active group can be calculated using the argument Overall for all possible comparisons.

propWINS(WO = 1.25, WR = 1.5, Overall = 120*150)
#>    WIN LOSS  TIE TOTAL  WR   WO
#> 1 6000 4000 8000 18000 1.5 1.25

References

Agresti, Alan. 2013. Categorical Data Analysis. John Wiley & Sons. https://www.wiley.com/en-gb/Categorical+Data+Analysis%2C+3rd+Edition-p-9780470463635.
Brunner, Edgar, Marc Vandemeulebroecke, and Tobias Mütze. 2021. “Win Odds: An Adaptation of the Win Ratio to Include Ties.” Statistics in Medicine. https://doi.org/10.1002/sim.8967.
Buyse, Marc. 2010. “Generalized Pairwise Comparisons of Prioritized Outcomes in the Two-Sample Problem.” Statistics in Medicine 29 (30): 3245–57. https://doi.org/10.1002/sim.3923.
Dong, Gaohong, Bo Huang, Johan Verbeeck, Ying Cui, James Song, Margaret Gamalo-Siebers, Duolao Wang, et al. 2022. “Win Statistics (Win Ratio, Win Odds, and Net Benefit) Can Complement One Another to Show the Strength of the Treatment Effect on Time-to-Event Outcomes.” Pharmaceutical Statistics. https://doi.org/10.1002/pst.2251.
Dong, G, DC Hoaglin, J Qiu, RA Matsouaka YW Chang, J Wang, and M Vandemeulebroecke. 2020. “The Win Ratio: On Interpretation and Handling of Ties.” Statistics in Biopharmaceutical Research 12 (1): 99–106. https://doi.org/10.1080/19466315.2019.1575279.
Gasparyan, Samvel B. 2024. hce: Design and Analysis of Hierarchical Composite Endpoints. CRAN: The Comprehensive R Archive Network, R Package, Version 0.6.0. https://CRAN.R-project.org/package=hce.
Gasparyan, Samvel B, Folke Folkvaljon, Olof Bengtsson, Joan Buenconsejo, and Gary G Koch. 2021. “Adjusted Win Ratio with Stratification: Calculation Methods and Interpretation.” Statistical Methods in Medical Research 30 (2): 580–611. https://doi.org/10.1177/0962280220942558.
Gasparyan, Samvel B, Elaine K Kowalewski, Folke Folkvaljon, Olof Bengtsson, Joan Buenconsejo, John Adler, and Gary G Koch. 2021. “Power and Sample Size Calculation for the Win Odds Test: Application to an Ordinal Endpoint in COVID-19 Trials.” Journal of Biopharmaceutical Statistics 31 (6): 765–87. https://doi.org/10.1080/10543406.2021.1968893.
Hoeffding, W. 1948. “A Class of Statistics with Asymptotically Normal Distribution.” The Annals of Mathematical Statistics 19 (3): 293–325. https://doi.org/10.1214/aoms/1177730196.
Kruskal, William H, and Leo Goodman. 1954. “Measures of Association for Cross Classifications.” Journal of the American Statistical Association 49 (268): 732–64. https://doi.org/10.2307/2281536.
Mann, Henry B, and Donald R Whitney. 1947. “On a Test of Whether One of Two Random Variables Is Stochastically Larger Than the Other.” The Annals of Mathematical Statistics, 50–60.
O’Brien, RG, and JM Castelloe. 2006. Exploiting the Link Between the Wilcoxon–Mann–Whitney Test and a Simple Odds Statistic. Cary, NC: SAS Institute Inc. https://support.sas.com/resources/papers/proceedings/proceedings/sugi31/209-31.pdf.
Peng, Lei. 2020. “The Use of the Win Odds in the Design of Non-Inferiority Clinical Trials.” Journal of Biopharmaceutical Statistics 30 (5): 941–46. https://doi.org/10.1080/10543406.2020.1757690.
Pocock, SJ, CA Ariti, TJ Collier, and D Wang. 2012. “The Win Ratio: A New Approach to the Analysis of Composite Endpoints in Clinical Trials Based on Clinical Priorities.” European Heart Journal 33 (2): 176–82. https://doi.org/10.1093/eurheartj/ehr352.
Rauch, Geraldine, Antje Jahn-Eimermacher, Werner Brannath, and Meinhard Kieser. 2014. “Opportunities and Challenges of Combined Effect Measures Based on Prioritized Outcomes.” Statistics in Medicine 33 (7): 1104–20. https://doi.org/10.1002/sim.3923.
Somers, Robert H. 1962. “A New Asymmetric Measure of Association for Ordinal Variables.” American Sociological Review, 799–811. https://doi.org/10.2307/2090408.
Verbeeck, Johan, Vaiva Deltuvaite-Thomas, Ben Berckmoes, Tomasz Burzykowski, Marc Aerts, Olivier Thas, Marc Buyse, and Geert Molenberghs. 2021. “Unbiasedness and Efficiency of Non-Parametric and UMVUE Estimators of the Probabilistic Index and Related Statistics.” Statistical Methods in Medical Research 30 (3): 747–68. https://doi.org/10.1177/0962280220966629.