There is offered the cat2cat procedure to map a categorical variable according to a mapping (transition) table between two different time points. The mapping (transition) table should to have a candidate for each category from the targeted for an update period. The main rule is to replicate the observation if it could be assigned to a few categories, then using simple frequencies or statistical methods to approximate probabilities of being assigned to each of them.
This algorithm was invented and implemented in the paper by (Nasinski, Majchrowska and Broniatowska (2020)).
For more details please read the paper by (Nasinski, Gajowniczek (2023)).
$ pip install cat2cat
For more examples and descriptions please vist the example notebook
# cat2cat datasets from cat2cat.datasets import load_trans, load_occup trans = load_trans() occup = load_occup()
from cat2cat.mappings import get_mappings, get_freqs, cat_apply_freq # convert the mapping table to two association lists mappings = get_mappings(trans) # get a variable levels freqencies codes_new = occup.code[occup.year == 2010].values freqs = get_freqs(codes_new) # apply the frequencies to the (one) association list mapp_new_p = cat_apply_freq(mappings["to_new"], freqs) # mappings for a specific category mappings["to_new"]['3481'] # probability mappings for a specific category mapp_new_p['3481']
from cat2cat import cat2cat from cat2cat.dataclass import cat2cat_data, cat2cat_mappings, cat2cat_ml from pandas import concat # split the panel by the time variale # here only two periods o_old = occup.loc[occup.year == 2008, :].copy() o_new = occup.loc[occup.year == 2010, :].copy() # dataclasses, core arguments for the cat2cat function data = cat2cat_data( old = o_old, new = o_new, cat_var_old = "code", cat_var_new = "code", time_var = "year" ) mappings = cat2cat_mappings(trans = trans, direction = "backward") # apply the cat2cat procedure c2c = cat2cat(data = data, mappings = mappings) # pandas.concat used to bind per period datasets data_final = concat([c2c["old"], c2c["new"]])
Interested in contributing? Check out the contributing guidelines. Please note that this project is released with a Code of Conduct. By contributing to this project, you agree to abide by its terms.
cat2cat was created by Maciej Nasinski. It is licensed under the terms of the MIT license.
cat2cat was created with cookiecutter and the py-pkgs-cookiecutter template.