1. Home
  2. Volume 23 Number 3, October 2023
  3. SUBIANTO

Application of SHAP on CatBoost classification for identification of variabels characterizing food insecurity occurrences in Aceh Province households

MUHAMMAD SUBIANTO, INA YATUL ULYA, EVI RAMADHANI, BAGUS SARTONO, ALFIAN FUTUHUL HADI

Abstract


Classification is the process of building a model that can distinguish between different classes of data. The model aims to predict the class of testing data based on patterns or relationships learned from training data. One of the data processing algorithms used to build classification models is Categorical Boosting (CatBoost). However, in general, the resulting models are difficult to interpret. To facilitate the interpretation of complex classification models, methods such as SHAP (SHapley Additive exPlanations) are needed. SHAP is a method to explain individual predictions. SHAP is based on the game theoretically optimal shapley values. In this study, an analysis of important SHAP variables was conducted on the CatBoost classification model to identify variables characterizing occurrences of food insecurity in households. The data used in this study was obtained from the Survei Sosial Ekonomi Nasional (Susenas) in March 2021 in Aceh Province, sourced from the Badan Pusat Statistik (BPS). There are 13,126 observations in the research data. The results from four evaluated classification models on the testing data showed that the best model had accuracy, sensitivity, specificity, and AUC values of 0.703, 0.349, 0.798, and 0.637, respectively. Furthermore, the results of the analysis of important SHAP variables showed that the variables number of household members who smoke ( ), education of the household head ( ), wall types ( ), drinking water source ( ), and decent sanitation ( ) significantly contributed to the occurrences of food insecurity in households in Aceh Province in the year 2021.

Keywords


food insecurity; Shapley Additive Explanations (SHAP); Categorical Boosting (CatBoost); Area Under the Curve (AUC)

References


Arena, F.; Pau, G. 2020. An overview of big data analysis. Bull. Electr. Eng. Informatics. 9 (4):1646–1653.

Ying, X. 2019. An Overview of Overfitting and its Solutions. J. Phys. Conf. Ser. 1168 (2).

Arrahimi, AR.; Ihsan, MK.; Kartini, D.; Faisal, MR.; Indriani, F. 2019. Teknik Bagging Dan Boosting pada Algoritma CART Untuk Klasifikasi Masa Studi Mahasiswa. J. Sains dan Inform. 5 (1):21–30.

Mahesh, B. 2020. Machine Learning Algorithms - A Review. Int. J. Sci. Res. 9 (1):381–386. Available from: https://www.ijsr.net/getabstract.php?paperid=ART20203995

Prokhorenkova, L.; Gusev, G.; Vorobev, A.; Dorogush, AV.; Gulin, A. 2018. Catboost: Unbiased boosting with categorical features. In: Advances in Neural Information Processing Systems. Neural information processing systems foundation; 2018-December. p. 6638–6648.

Rodríguez-Pérez, R.; Bajorath, J. 2020. Interpretation of machine learning models using shapley values: application to compound potency and multi-target activity predictions. J. Comput. Aided. Mol. 34:1013–1026. DOI: 10.1007/s10822-020-00314-0

Mayapada, R.; Yanti, RW.; Syarifuddin S. 2022. Analisis Tingkat Kepentingan terhadap Faktor-Faktor yang Mempengaruhi Indeks Pembangunan Manusia di Indonesia. J. Math. Theory Appl. 4 (2):45–49. Available from: https://ojs.unsulbar.ac.id/index.php/Mathematics/article/view/2030

Asyiva, A.; Susetyo, B.; Sartono, B.; Hadi, AF.; Ramadhani, E. 2022. Interpretable machine learning to characterize food insecurity in Aceh and West Java provinces. Proceeding Cgant Unej. Available from: https://proceedingcgantunej.or.id/index.php/proceedingcgant/article/view/13

Pusat Ketersediaan dan Kerawanan Pangan. 2021. Indeks Ketahanan Pangan. Available from: https://repository.pertanian.go.id/handle/123456789/15396

Reagan, HA. 2018. Measuring Food Insecurity Experience Scale (FIES) in Indonesia. International Workshop on Sustainable Development Goal (SDG) Indicators. June:26–28.

Nasional, BP. 2022. Peta Ketahanan dan Kerentanan Pangan. Food Secur. VVulnerability Atlas Tahun 2022. 41.

Dorogush, AV.; Ershov, V.; Gulin, A. 2018. CatBoost: Gradient Boosting with Categorical Features Support. Available from: https://catboost.ai. DOI: 10.48550/arXiv.1810.11363

Hunter, JD. 2007. Matplotlib: A 2D graphics environment. Comput. Sci. & Eng. 9 (3):90–95.

McKinney, W. 2010. Data Structures for Statistical Computing in Python. In: Proceedings of the 9th Python in Science Conference. p. 51–56. DOI: 10.25080/majora-92bf1922-00a

Harris, CR.; Millman, KJ.; van der Walt, SJ.; Gommers, R.; Virtanen, P.; Cournapeau, D.; et al. 2020. Array Programming with NumPy. Nature. 585:357–362. DOI: 10.1038/s41586-020-2649-2

Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; et al. 2011. Scikit-learn: Machine Learning in Python. J. Mach. Learn. Res. 12 (Oct):2825–2830.

Supranto, J. 2000. Statistik Teori dan Aplikasi, edisi Keenam Jilid 1. Ed 1, Cet. Jakarta: Erlangga.

Chawla, NV.; Bowyer, KW.; Hall, LO.; Kegelmeyer, WP. 2002. SMOTE: Synthetic Minority Over-Sampling Technique. J. Artif. Intell. Res. 16:321–357. DOI: 10.1613/jair.953

Chicco, D.; Tötsch, N.; Jurman, G. 2021. The matthews correlation coefficient (Mcc) is more reliable than balanced accuracy, bookmaker informedness, and markedness in two-class confusion matrix evaluation. BioData Mining. 14:1–22. DOI: 10.1186/s13040-021-00244-z

Bewick, V.; Cheek, L.; Ball, J. 2004. Statistics Review 13: Receiver Operating Characteristic Curves. Critical Care. 8 (6):508. DOI: 10.1186/cc3000

Saito, T.; Rehmsmeier, M. 2015. The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets. PLoS One. 10 (3). DOI: 10.1371/journal.pone.0118432

Sihite, NW.; Tanziha, I. 2021. Faktor-Faktor yang Mempengaruhi Ketahanan Pangan Rumah Tangga di Kota Medan. AcTion Aceh Nutr. J. 6 (1):15–24. Available from: https://ejournal.poltekkesaceh.ac.id/index.php/an/article/view/395

Kang, SY.; Cho, HJ. 2022. Association Between the Use of Tobacco Products and Food Insecurity Among South Korean Adults. Int. J. Public Health. 67:1604866. DOI: 10.3389/ijph.2022.1604866

Berry, KM.; Drew, JAR.; Brady, PJ. Widome, R. 2023. Impact of smoking cessation on household food security. Ann. Epidemiol. 79:49--55.e3. DOI: 10.1016/j.annepidem.2023.01.007

Jones, AD.; Ngure, FM.; Pelto, G.; Young, SL. 2013. What Are We Assessing When We Measure Food Security? A Compendium and Review of Current Metrics. Adv. Nutr. 4 (5):481–505.

Sreeramareddy, CT.; Ramakrishnareddy, N. 2011. Association of adult tobacco use with household food access insecurity: results from Nepal demographic and health survey. BMC Public Health. 18 (1):48. DOI: 10.1186/s12889-017-4579-y


Full Text: PDF

DOI: 10.24815/jn.v23i3.33548

Refbacks

  • There are currently no refbacks.