cat2cat algorithm was designed to offer
an easy and clear interface to apply a mapping table (transition table)
which was provided by a data maintainer or built by a researcher. The
objective is to unify an inconsistent coded categorical variable in a
panel dataset, where a transition table is the core element of the
Examples of a dataset with such inconsistent coded categorical variable are ISCO (The International Standard Classification of Occupations) or ICD (International Classification of Diseases) based one. The both classifications are regularly updated to adjust to e.g. new science achievements. More clearly we might image that e.g. new science achievements brings new occupations types on the market or enable recognition of new diseases types.
The categorical variable encoding changes are typically provided by datasets providers in the transition table form, for each time point the changes occurred. A transition table conveys information needed for matching all categories between two periods of time. More precisely it contains two columns where the first column contains old categories and the second column contains the new ones. Sometimes a transition table has to be created manually by a researcher.
The main rule is to replicate the observation if it could be assigned to a few categories. More precisely for each observation we look across a transition table to check how the original category could be mapped to the opposite period one. Then using simple frequencies or statistical methods to approximate weights (probabilities) of being assigned to each of them. For each observation that was replicated, the probabilities have to add up to one. The algorithm distinguishes different mechanics for panel data with and without unique identifiers.
There are 3 important elements:
occup dataset is an example of unbalance panel dataset.
This is a simulated data although there are applied a real world
characteristics from national statistical office survey. The original
survey is anonymous and take place every two years.
trans dataset containing transitions between old (2008)
and new (2010) occupational codes. This table could be used to map
encodings in both directions.
library(cat2cat) library(dplyr) data(occup) data(trans) <- occup[occup$year == 2008, ] occup_old <- occup[occup$year == 2010, ] occup_new <- occup[occup$year == 2006, ] occup_2006 <- occup[occup$year == 2008, ] occup_2008 <- occup[occup$year == 2010, ] occup_2010 <- occup[occup$year == 2012, ]occup_2012
There were prepared two graphs for forward and backward mapping.
These graphs present how the
works, in this case under a panel dataset without the unique identifiers
and only two periods.