Komparasi Teknik Hyperparameter Optimization pada SVM untuk Permasalahan Klasifikasi dengan Menggunakan Grid Search dan Random Search

Muhamad Fajri; Aji Primajaya

doi:10.30871/jaic.v7i1.5004

Muhamad Fajri Universitas Singaperbangsa Karawang
Aji Primajaya Universitas Singaperbangsa Karawang

DOI: https://doi.org/10.30871/jaic.v7i1.5004

Keywords: Grid Search, Hyper Parameter Optimization, Random Search, SVM

Abstract

Classification is one of the important tasks in the field of Machine Learning. Classification can be viewed as an Optimization Problem (Optimization Problem) with the aim of finding the best model that can represent the relationship/pattern between data with labels. Support Vector Machine (SVM) Is an algorithm in Machine Learning used to solve problems such as Classification or Regression. The performance of the SVM algorithm is strongly influenced by parameters, for example error prediction in non-linear SVM results in parameters C and gamma. In this study, an analysis of the technique was carried out to obtain good parameter values using Grid search and Random Search on seven datasets. Evaluation is done by calculating the value of accuracy, memory usage and validity test time of the best model by the two techniques.

Downloads

Download data is not yet available.

References

D. K Srivastave and L. Bhambu, “Data Classification Using Support Vector Machine,” J. Theor. Appl. Inf. Technol., vol. 1, pp. 3–6, 2010, doi: 10.1109/ICTAI.2010.9.

F. E. B. Otero, A. A. Freitas, and C. G. Johnson, “Inducing decision trees with an ant colony optimization algorithm,” Appl. Soft Comput. J., vol. 12, no. 11, pp. 3615–3626, 2012, doi: 10.1016/j.asoc.2012.05.028.

C. Cortez and V. Vapnik, “Support-Vector Networks,” Mach. Learn., vol. 7, no. 5, pp. 63–72, 1995, doi: 10.1109/64.163674.

R. A. Rizal, I. S. Girsang, and S. A. Prasetiyo, “Klasifikasi Wajah Menggunakan Support Vector Machine (SVM) Reyhan,” Ris. dan E-Jurnal Manaj. Inform. Komput., vol. 3, no. 2, pp. 275–280, 2019, doi: 10.33480/pilar.v15i2.693.

Y. Wang et al., “Minimum distribution support vector clustering,” Entropy, vol. 23, no. 11, 2021, doi: 10.3390/e23111473.

A. Klein, S. Falkner, S. Bartels, P. Hennig, and F. Hutter, “Fast Bayesian optimization of machine learning hyperparameters on large datasets,” Proc. 20th Int. Conf. Artif. Intell. Stat. AISTATS 2017, vol. 11, no. June, pp. 4945–4968, 2017.

T. Yu and H. Zhu, “Hyper-Parameter Optimization: A Review of Algorithms and Applications,” pp. 1–56, 2020, [Online]. Available: http://arxiv.org/abs/2003.05689.

I. Syarif, A. Prugel-Bennett, and G. Wills, “SVM Parameter Optimization using Grid Search and Genetic Algorithm to Improve Classification Performance,” TELKOMNIKA (Telecommunication Comput. Electron. Control., vol. 14, no. 4, p. 1502, 2016, doi: 10.12928/telkomnika.v14i4.3956.

S. Yuanyuan, W. Yongming, G. Lili, M. Zhongsong, and J. Shan, “The comparison of optimizing SVM by GA and grid search,” ICEMI 2017 - Proc. IEEE 13th Int. Conf. Electron. Meas. Instruments, vol. 2018-Janua, pp. 354–360, 2017, doi: 10.1109/ICEMI.2017.8265815.

T. Horváth, R. G. Mantovani, and A. C. P. L. F. de Carvalho, “Effects of random sampling on SVM hyper-parameter tuning,” Adv. Intell. Syst. Comput., vol. 557, pp. 268–278, 2017, doi: 10.1007/978-3-319-53480-0_27.

V. K. Chauhan, K. Dahiya, and A. Sharma, “Problem formulations and solvers in linear SVM: a review,” Artif. Intell. Rev., vol. 52, no. 2, pp. 803–855, 2019, doi: 10.1007/s10462-018-9614-6.

G. H. Saputra, A. H. Wigena, and B. Sartono, “Penggunaan Support Vector Regression Dalam Pemodelan Indeks Saham Syariah Indonesia Dengan Algoritme Grid Search,” Indones. J. Stat. Its Appl., vol. 3, no. 2, pp. 148–160, 2019, doi: 10.29244/ijsa.v3i2.172.

R. Valarmathi and T. Sheela, “Heart disease prediction using hyper parameter optimization (HPO) tuning,” Biomed. Signal Process. Control, vol. 70, no. August, p. 103033, 2021, doi: 10.1016/j.bspc.2021.103033.

Moch. Lutfi and Mochamad Hasyim, “Penanganan Data Missing Value Pada Kualitas Produksi Jagung Dengan Menggunakan Metode K-Nn Imputation Pada Algoritma C4.5,” J. Resist. (Rekayasa Sist. Komputer), vol. 2, no. 2, pp. 89–104, 2019, doi: 10.31598/jurnalresistor.v2i2.427.

A. R. Alfarisi, H. Tjandrasa, and I. Arieshanti, “Perbandingan Performa antara Imputasi Metode Konvensional dan Imputasi dengan Algoritma,” Mach. Learn., vol. 2, no. 1, pp. 1–4, 2013.

A. Peryanto, A. Yudhana, and R. Umar, “Klasifikasi Citra Menggunakan Convolutional Neural Network dan K Fold Cross Validation,” J. Appl. Informatics Comput., vol. 4, no. 1, pp. 45–51, 2020, doi: 10.30871/jaic.v4i1.2017.

P. Fabian et al., “Scikit-learn: Machine Learning in Python Fabian,” Mach. Learn. Res., vol. 12, no. 9, pp. 2825–2830, 2011, doi: 10.1289/EHP4713.