Data selection

This vignette describes how to retrieve data from a coin. The main functions to do this are get_dset() and the more flexible get_data().

These functions are important to understand, because many COINr functions use them to retrieve data for plotting, analysis and other functions. Both functions are generics, which means that they have methods for coins and purses.

Data sets

Every time a “building” operation is applied to a coin, such as Treat(), Screen(), Normalise() and so on, a new data set is created. Data sets live in the .$Data sub-list of the coin. We can retrieve a data set at any time using the get_data() function: library(COINr) # build full example coin coin <- build_example_coin(quietly = TRUE) # retrieve normalised data set dset_norm <- get_dset(coin, dset = "Normalised") # view first few rows and cols head(dset_norm[1:5], 5) #> uCode LPI Flights Ship Bord #> 1 AUS 79.96112 12.3223217 66.14497 0.00000 #> 2 AUT 94.07137 27.8763185 0.00000 42.01269 #> 3 BEL 94.56023 23.3967426 97.14314 100.00000 #> 4 BGD 27.63906 0.1243185 45.80661 10.85013 #> 5 BGR 34.29965 10.8828790 37.40495 16.34359 By default, a data set in the coin consists of indicator columns plus the “uCode” column, which is the unique identifier of each row. You can also ask to attach unit metadata columns, such as unit names, groups, and anything else that was input when building the coin, using the also_get argument: # retrieve normalised data set dset_norm2 <- get_dset(coin, dset = "Normalised", also_get = c("uName", "GDP_group")) # view first few rows and cols head(dset_norm2[1:5], 5) #> uCode uName GDP_group LPI Flights #> 1 AUS Australia XL 79.96112 12.3223217 #> 2 AUT Austria L 94.07137 27.8763185 #> 3 BEL Belgium L 94.56023 23.3967426 #> 4 BGD Bangladesh M 27.63906 0.1243185 #> 5 BGR Bulgaria S 34.29965 10.8828790 Data subsets While get_dset() is a quick way to retrieve an entire data set and metadata, the get_data() function is a generalisation: it can also be used to obtain a whole data set, but also subsets of data, based on e.g. indicator selection and grouping (columns), as well as unit selection and grouping (rows). Indicators/columns A simple example is to extract one or more named indicators from a target data set: x <- get_data(coin, dset = "Raw", iCodes = c("Flights", "LPI")) # see first few rows head(x, 5) #> uCode Flights LPI #> 31 AUS 36.05498 3.793385 #> 1 AUT 29.01725 4.097985 #> 2 BEL 31.88546 4.108538 #> 32 BGD 4.27955 2.663902 #> 3 BGR 9.23588 2.807685 By default, get_data() returns the requested indicators, plus the uCode identifier column. We can also set also_get = "none" to return only the indicator columns. The iCode argument can also accept groups of indicators, based on the structure of the index. In our example, indicators are aggregated into “pillars” (level 2) within groups. We can name an aggregation group and extract the underlying indicators: x <- get_data(coin, dset = "Raw", iCodes = "Political", Level = 1) head(x, 5) #> uCode Embs IGOs UNVote #> 31 AUS 82 196 38.46245 #> 1 AUT 88 227 42.63920 #> 2 BEL 84 248 43.00308 #> 32 BGD 52 145 38.60601 #> 3 BGR 67 209 42.95986 Here we have requested all the indicators in level 1 (the indicator level), that belong to the group called “Political” (one of the pillars). Specifying the level becomes more relevant when we look at the aggregated data set, which also includes the pillar, sub-index and index scores. Here, for example, we can ask for all the pillar scores (level 2) which belong to the sustainability sub-index (level 3): x <- get_data(coin, dset = "Aggregated", iCodes = "Sust", Level = 2) head(x, 5) #> uCode Environ Social SusEcFin #> 1 AUS 31.92211 71.88108 55.69987 #> 2 AUT 69.47511 72.76415 62.88150 #> 3 BEL 53.00859 86.16783 50.09020 #> 4 BGD 81.66988 27.51138 64.58884 #> 5 BGR 55.69922 53.30489 61.68677 If this isn’t clear, look at the structure of the example index using e.g. plot_framework(coin). If we wanted to select all the indicators within the “Sust” sub-index we would set Level = 1. If we wanted to select the sub-index scores themselves we would set Level = 3, and so on. The idea of selecting indicators and aggregates based on the structure of the index is useful in many places in COINr, for example examining correlations within aggregation groups using plot_corr(). Units/rows Units (rows) of the data set can also be selected (also in combination with selecting indicators). Starting with a simple example, let’s select specified units for a specific indicator: get_data(coin, dset = "Raw", iCodes = "Goods", uCodes = c("AUT", "VNM")) #> uCode Goods #> 1 AUT 278.4264 #> 51 VNM 269.0766 Rows can also be sub-setted using groups, i.e. unit groupings that are defined using variables input with iMeta$Type = "Group" when building the coin. Recall that for our example coin we have several groups (a reminder that you can see some details about the coin using its print method):

coin
#> --------------
#> A coin with...
#> --------------
#> Input:
#>   Units: 51 (AUS, AUT, BEL, ...)
#>   Indicators: 49 (Goods, Services, FDI, ...)
#>   Denominators: 4 (Area, Energy, GDP, ...)
#>   Groups: 4 (GDP_group, GDPpc_group, Pop_group, ...)
#>
#> Structure:
#>   Level 1 Indicator: 49 indicators (FDI, ForPort, Goods, ...)
#>   Level 2 Pillar: 8 groups (ConEcFin, Instit, P2P, ...)
#>   Level 3 Sub-index: 2 groups (Conn, Sust)
#>   Level 4 Index: 1 groups (Index)
#>
#> Data sets:
#>   Raw (51 units)
#>   Denominated (51 units)
#>   Imputed (51 units)
#>   Screened (51 units)
#>   Treated (51 units)
#>   Normalised (51 units)
#>   Aggregated (51 units)

The first way to subset by unit group is to name a grouping variable, and a group within that variable to select. For example, say we want to know the values of the “Goods” indicator for all the countries in the “XL” GDP group:

get_data(coin, dset = "Raw", iCodes = "Goods", use_group = list(GDP_group = "XL"))
#>    uCode GDP_group     Goods
#> 1    AUS        XL  288.4893
#> 8    CHN        XL 1713.6190
#> 11   DEU        XL 1919.1940
#> 13   ESP        XL  447.1229
#> 16   FRA        XL  849.3303
#> 17   GBR        XL  778.9052
#> 21   IDN        XL  222.4186
#> 22   IND        XL  288.9806
#> 24   ITA        XL  658.1981
#> 25   JPN        XL  732.2078
#> 28   KOR        XL  568.9920
#> 45   RUS        XL  343.8504

Since we have subsetted by group, this also returns the group column which was used.

Another way of sub-setting is to combine uCodes and use_group. When these two arguments are both specified, the result is to return the full group(s) to which the specified uCodes belong. This can be used to put a unit in context with its peers within a group. For example, we might want to see the values of the “Flights” indicator for a specific unit, as well as all other units within the same population group:

get_data(coin, dset = "Raw", iCodes = "Flights", uCodes = "MLT", use_group = "Pop_group")
#>    uCode Pop_group  Flights
#> 6    BRN         S  2.01900
#> 9    CYP         S  8.75467
#> 14   EST         S  3.12946
#> 19   HRV         S  9.24529
#> 23   IRL         S 34.17721
#> 30   LTU         S  5.37919
#> 31   LUX         S  4.84458
#> 32   LVA         S  6.77976
#> 33   MLT         S  6.75251
#> 35   MNG         S  0.98951
#> 38   NOR         S 25.64994
#> 39   NZL         S 13.37242
#> 48   SVN         S  1.51736

Here, we have to specify use_group simply as a string rather than a list. Since MLT is in the “S” population group, it returns all units within that group.

Overall, the idea of get_data() is to flexibly return subsets of indicator data, based on the structure of the index and unit groups.

Manual selection

As a final point, it’s worth pointing out that a coin is simply a list of R objects such as data frames, other lists, vectors and so on. It has a particular format which allows things to be easily accessed by COINr functions. But other than that, its an ordinary R object. This means that even without the helper functions mentioned, you can get at the data simply by exploring the coin yourself.

The data sets live in the .$Data sub-list of the coin: names(coin$Data)
#> [1] "Raw"         "Denominated" "Imputed"     "Screened"    "Treated"
#> [6] "Normalised"  "Aggregated"

And we can access any of these directly:

data_raw <- coin$Data$Raw

#>    uCode      LPI  Flights      Ship Bord
#> 31   AUS 3.793385 36.05498 14.004198    0
#> 1    AUT 4.097985 29.01725  0.000000   35
#> 2    BEL 4.108538 31.88546 20.567121   48
#> 32   BGD 2.663902  4.27955  9.698165   16
#> 3    BGR 2.807685  9.23588  7.919366   18

The metadata lives in the .$Meta sub-list. For example, the unit metadata, which includes groups, names etc: str(coin$Meta$Unit) #> 'data.frame': 51 obs. of 11 variables: #>$ uCode        : chr  "AUS" "AUT" "BEL" "BGD" ...
#>  $uName : chr "Australia" "Austria" "Belgium" "Bangladesh" ... #>$ GDP_group    : chr  "XL" "L" "L" "M" ...
#>  $GDPpc_group : chr "XL" "XL" "L" "S" ... #>$ Pop_group    : chr  "L" "M" "L" "XL" ...
#>  $EurAsia_group: chr "Asia" "Europe" "Europe" "Asia" ... #>$ Time         : num  2018 2018 2018 2018 2018 ...
#>  $Area : num 7741220 83871 30528 148460 110879 ... #>$ Energy       : num  81.3 27 41.83 27.92 9.96 ...
#>  $GDP : num 1304.5 390.8 468 220.8 53.2 ... #>$ Population   : num  24451 8735 11429 164670 7085 ...

The point is that if COINr tools don’t get you where you want to go, knowing your way around the coin allows you to access the data exactly how you want.