The potential of synthetic minority oversampling technique to enhance the precision of gender prediction: an investigation of artificial neural networks with cephalometry

Full Text

Background

Radiography is an essential component of forensic odontology because it is a straightforward, cost-effective, and noninvasive method of identification, particularly for identifying corpses by comparing premortem and postmortem radiographs [1]. Dental radiographs can provide reliable information, such as comparison data of the anatomical shape of teeth, periapical anomalies, fillings, cavities, outlines, and positions of impacted teeth, among others [1]. Broadbent invented lateral cephalometry in 1931, which has the benefit of gaining a comprehensive visual representation of the cranial structure and soft tissue contour [2]. In addition, lateral cephalometry permits the evaluation of several anatomical components, such as the nasal bones, frontal sinuses, sphenoid sinuses, and other images that assist in the gender identification process [3-5].

Patil et al, used artificial neural networks (ANN) to predict gender using mandibular radiograph [6]. ANN is a type of computing system that utilizes machine learning (ML) to generate intricate information (7). An ANN is a computational model that draws inspiration from the biological nervous system of the brain [7]. The ANN can obtain and retain knowledge (information-based) and can be characterized as a collection of processing units represented by artificial neurons, interconnected by numerous interconnections (artificial synapses), and implemented by vectors and matrices of synaptic weights [8, 9]. The main advantage ANN is its capacity to acquire knowledge and maintain stability even when confronted with minor errors by generating information through the learning process [7, 8, 10].

A previous study conducted by Handayani et al demonstrated how well both of these models predict the gender of adult Indonesians by utilizing 297 lateral cephalometry ANN models with 80% of the samples for training, 10% of the samples for testing, and 10% of the samples for validating. Although an overall accuracy of 80% is achieved, data imbalance reduces the accuracy of predicting the male gender. Data imbalance occurs when the number of people in each class is unequal. In other words, one class is not well represented (minority class), whereas the other class has more cases (majority class). In many real-life situations, the class mismatch problem occurs [11]. The most well-known method to deal with uneven data is the synthetic minority oversampling technique (SMOTE). SMOTE creates new fake data patterns by combining samples from the minority class with their K closest neighbors in a straight line [11]. This study uses an ANN model to predict gender and examines the possible effects of applying SMOTE.

Aim

This study seeks to determine the potential impact of using SMOTE on gender prediction using the ANN model.

Materials and Method

A total of 297 cephalometry images were taken from the medical records of patients at the Airlangga University Dental and Mouth Hospital (RSGMP Unair) in Surabaya, part of this data is taken from a previous study [12]. Subsequently, the sample will be partitioned into three subsets, i.e., 70% of the cephalometry images will be allocated for training purposes, 15% will be reserved for validation tests, and the remaining 15% will be utilized for testing. Training, development, and analysis of the data will be conducted by cephalometric algorithms using ANN in Python. The following inclusion criteria will be employed:

1) Cephalometric images are obtained from preexisting cephalometric photographs at the RSGMP FKG Airlangga University in Surabaya.

2) Cephalometric pictures are acquired from individuals of Indonesian descent.

3) Cephalometric photographs are captured using standardized instruments and identical equipment.

4) Cephalometric photographs are in a satisfactory state, with no evidence of superimposition.

5) Cephalometric photographs exhibit a satisfactory state of preservation, devoid of any discernible distortions.

6) Cephalometric photographs are captured by operators possessing a minimum of D3 radiology education, together with a minimum of 1 year of practical experience operating cephalometric equipment.

7) Cephalometric images of individuals between the ages of 18 and 40 years. The cephalometric photograph exhibits a comprehensive dentition in both the mandibular and maxillary regions, except for the third molar.

8) Cephalometric photographs of individuals who lack either orthodontic intervention or prior records of orthodontic intervention are taken.

9) Cephalometric photographs are obtained from individuals lacking prior records of orthognathic surgery.

10) Cephalometric photographs of individuals who do not possess any documented instances of jaw injuries are taken.

This study employs the following instruments:

1) The cephalometric photographs were captured using equipment that adheres to defined protocols.

2) The cephalometric device employed in this study is the ZULASSUNG THA/HV-GEN Type THA100.

3) Operators of cephalometric equipment possess standardized skills.

4) A computer system equipped with a minimum of two 8-GB random access memory modules and a solid-state drive with a storage capacity of 1 TB.

5) The computer is equipped with the NVIDIA GeForce RTX 3060 graphics processing unit.

6) The utilization of Python in web development and Google Collaboration.

7) The utilization of Pandas.

8) The utilization of the SHapley Additive exPlanation (SHAP).

9) The utilization of WebCeph application.

Variables and Definition

The following variables are used in this study:

1. Dependent variables: The precision of cephalometric analysis utilizing the ANN techniques, considering the variables listed in Table 1.
2. Independent variable: Gender-based variations in cephalometric data employing the artificial intelligence computational analysis method, ANN.
3. Control variables: Age, cephalometric instrument, operator skill, Indonesian people.

 

Table 1. Definition of the operational variables

Parameter	Definition
Age	The length of life of the respondent is seen from the age shown in the cephalometric photo. The ratio is 18–40 years
Cephalometric photo	The cephalometric photo was taken at RSGMP Unair by an operator with the same level of skill-using a standard device. There was no superimposition, and the teeth were complete, except for the third molar. The photo was taken from a patient who has never undergone jaw and maxilla surgery, or orthodontic treatment, or has never experienced any jaw and maxilla trauma
Instrument	The standardized tool used to capture cephalometric photos at RSGMP Unair is the ASAHI Hyper-XCM Panoramic Cephalometric Type D.052 SB
Operator skill	The required qualifications for the operator position include a minimum of a D3 degree in Radiography and at least 1 year of experience operating cephalometric radiology equipment
Indonesian people	The cephalometric photo was taken from a Mongoloid race, which was generalized from cephalometric photo results originating from Indonesian citizens. Photo selection was performed by an operator from RSGMP Unair
SNA angle	Angles formed from the points SellaNasion–A (Wyle)
SGA angle	Angles formed from the points Sella_Glabella_A on the cephalometric photographs (Johnson)
Mandibular length	A lining of the lower mandibular boundary that is projected to the most posterior point from the head of the condyle to the more anterior point of the jaw (Jarabak)
Mandibular angle	Angles formed from the Gonion_Gnathion line (Go_Gn) together with the base of the anterior skull (S_N; Steiner)
Diagnosis	The differential diagnosis of the skeletal pattern and the dental classification of malocclusion

 

Results

Cephalometric radiographs were obtained from patients who sought treatment at Airlangga University Dental Hospital in Surabaya, Indonesia, as part of this research endeavor. This study has passed the ethics review with ethical number 316/HRECC.FODM/III/2023 conducted by Airlangga University Faculty of Dental Medicine Health Research Ethical Clearance Commission. We choose a suitable shape for people between the ages of 18 and 40 years. The collection comprises 297 cephalometric photographs, including 229 images in the female category and 68 images in the male category. The observed disparity in data between males and females can be attributed to the predominant representation of female patients seeking orthodontic therapy, as indicated by the majority of cephalometry data collected. The Python programming language is utilized to partition the photos into three distinct segments, i.e., 80% for training, 15% for validation, and 15% for testing, as depicted in Table 2.

 

Table 2. Distribution of sample frequency based on gender

	Training (70%)	Validation (15%)	Testing (15%)	Total sample
Female	153	38	38	229
Male	46	11	11	68
Total sample	199	49	49	297

 

A. Measurement of the variables

We employed the backpropagation method in this investigation. In this methodology, two computational processes, i.e., sophisticated computations to ascertain the disparity between the output of the ANN and the intended goal, are executed. The next step entails a reverse calculation that utilizes the obtained errors to modify the weights of all of the neurons in the system.

The ANN cephalometric parameters were measured using the end web application, which is a medical record application for orthodontic care. This application is utilized because of its cost-free nature and its capability to reduce errors associated with manual procedures. The visual representation of the measurement obtained using the front-end web application is depicted in Fig. 1.

 

Fig. 1. Analysis of cephalometric parameters using the front-end web application [13].

 

After obtaining the value of the parameters searched using the front-end web application, the results of the four parameters were analyzed using Pandas in Python, and the ANN was trained based on the diagnosis of malocclusion of the patient as listed in Table 3. Notably, the sample ratio is in Class I malocclusion angle with normal occlusion.

 

Table 3. Distribution of sample frequency based on diagnosis

Diagnosis	Gender		Total sample
	Female	Male
Class I malocclusion angle with normal occlusion	45	17	62
Class I malocclusion angle with maxilla protrusion	40	3	43
Class I malocclusion angle with open bite	0	1	1
Class II malocclusion angle with normal occlusion	57	23	80
Class II malocclusion angle with maxilla protrusion	64	16	80
Class III malocclusion angle with normal occlusion	11	7	18
Class III malocclusion angle with maxilla protrusion	10	1	11
Class III malocclusion angle with maxillary retrusion	2	0	2
Total sample	229	68	297

 

The average values of the four parameters were determined by analyzing them using Pandas in Python. The parameters obtained from the cephalometric photographs were used as a gender reference in the ANN algorithm. The results are shown in Table 4.

 

Table 4. Average values of the cephalometric parameters based on gender and diagnosis

Diagnosis	Gender	Mean of the parameter
		SNA angle	Mandibular length (mm)	Mandibular angle	SGA angle
Class I malocclusion angle with normal occlusion	Male	84.78	83.46	30.45	72.81
	Female	82.75	78.32	32.5	72.46
Class I malocclusion angle with maxilla protrusion	Male	88.7	83.18	28.43	74.4
	Female	87.79	81.11	30.69	75.02
Class I malocclusion angle with open bite	Male	80.47	72.73	32.95	70.5
	Female	_	_	_	_
Class II malocclusion angle with normal occlusion	Male	84.24	80.53	31.47	72.71
	Female	82.51	75.74	37	71.45
Class II malocclusion angle with maxilla protrusion	Male	89.47	78.94	26.90	76.04
	Female	87.52	79.23	31.91	75.76
Class III malocclusion angle with normal occlusion	Male	84.29	57.85	49.573	71.1
	Female	82.3	79.12	30.38	68.98
Class III malocclusion angle with maxilla protrusion	Male	88.96	81.22	20.52	64.9
	Female	86.94	84.98	25.49	75.01
Class III malocclusion angle with maxillary retrusion	Male	_	_	_	_
	Female	74.37	73	35.35	71.35

 

B. Network training and validation

Python is used to develop the ANN architecture. The first step involves training and validating the model before conducting the tests. This study employs four distinct scenarios. The first scenario involves the absence of SMOTE and normalization. The second scenario entails the presence of SMOTE without normalization. The third scenario involves the combination of SMOTE and normalization. The fourth scenario involves the presence of normalization without SMOTE. The training and validation methods of photocephalometry are exemplified in Fig. 2 and 3. The observation of a decreasing loss graph in both the training and validation stages provides additional support for this claim.

 

Fig. 2. Graph of the success of the training and validation of ANN: a — training and validation accuracy without SMOTE and normalization; b — training and validation accuracy with SMOTE and without normalization; c — training and validation accuracy with SMOTE and normalization; d — training and validation accuracy with normalization and without SMOTE.

 

Fig. 3. Graph of the failure of the training and validation of ANN: a — training and validation loss without SMOTE and normalization; b — training and validation loss with SMOTE and without normalization; c — training and validation loss with SMOTE and normalization; d — training and validation loss with normalization and without SMOTE.

 

C. ANN model analysis

The results of accuracy, precision, recall, f1 score, and support of the ANN model under four different scenarios are presented in Table 5. The first scenario involves developing the model without incorporating SMOTE and normalization techniques. The accuracy outcome for the first scenario is 0.78, or 78%, with a male f1 score of 0.00 and a female f1 score of 0.87. The second scenario of the ANN model involves the use of SMOTE without normalization. The accuracy score obtained is 0.22, which corresponds to 22%. The male f1 score is 0.00, whereas the female f1 score is 0.37. The third ANN model scenario involves the use of SMOTE and normalization. The accuracy result is 0.71, which corresponds to a 71% accuracy rate. The male f1 score is 0.30, whereas the female f1 score is 0.82. In the final scenario, the absence of SMOTE with normalization leads to an accuracy result of 0.76, or 76%. The male f1 score is 0.14, whereas the female f1 score is 0.86.

 

Table 5. Accuracy, precision, recall, f1 score, and support of the ANN model

Scenario		Precision	Recall	f1 score	Support
Without SMOTE and normalization	Female	0.78	1.00	0.87	38
	Male	0.00	0.00	0.00	11
	Accuracy	0.78	49
	Macro average	0.39	0.50	0.44	49
	Weighted average	0.60	0.78	0.68	49
With SMOTE and without normalization	Female	0.00	0.00	0.37	39
	Male	0.22	1.00	0.00	11
	Accuracy			0.22	49
	Macro average	0.11	0.50	0.18	49
	Weighted average	0.05	0.22	0.08	49
With SMOTE and normalization	Female	0.80	0.84	0.82	38
	Male	0.33	0.27	0.30	11
	Accuracy	0.71	49
	Macro average	0.57	0.56	0.56	49
	Weighted average	0.70	0.71	0.70	49
With normalization and without SMOTE	Female	0.78	0.95	0.86	38
	Male	0.33	0.09	0.14	11
	Accuracy			0.76	49
	Macro average	0.90	0.57	0.57	49
	Weighted average	0.88	0.80	0.74	49

 

A matrix classification provides a concise representation of the classification accuracy of a classifier in relation to a given set of test data. A two-dimensional matrix is created, with one dimension containing the true class of an object and the other dimension containing the class assigned by the classifier. This study employed a two-class design, with one class representing females and the other class representing males, which were classified as the positive class and the negative class, respectively. Within this framework, the four cells of the matrix are classified as true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN), as specified in Table 6.

 

Table 6. Matrix classification results of males and females using ANN

Without SMOTE and normalization
n=49	Predicted: YES	Predicted: NO
Actual: NO	TP=38	TN=0
Actual: YES	FP = 11	FN=0
	38	11
With SMOTE and without normalization
n=49	Predicted: YES	Predicted: NO
Actual: NO	TP=0	TN=38
Actual: YES	FP = 0	FN=11
	38	11
With SMOTE and normalization
n=49	Predicted: YES	Predicted: NO
Actual: NO	TP=32	TN=6
Actual: YES	FP = 8	FN=3
	38	11
With normalization and without SMOTE
n=49	Predicted: YES	Predicted: NO
Actual: NO	TP=36	TN=2
Actual: YES	FP = 10	FN=1
	38	11

Note. TP ― true positives; TN ― true negatives; FP ― false positives; FN ― false negatives.

 

In this study, the ANN model utilizes four cephalometric factors as the foundation for gender prediction. However, certain parameters may undermine the accuracy of the prediction. The SHAP model was employed in this work to determine the most relevant parameters in predicting gender. Fig. 4 illustrates that the mandibular angle had the strongest positive impact on gender prediction, whereas the SGA angle had the most negative impact on gender prediction.

 

Fig. 4. SHapley Additive exPlanation values attribute to each feature the change in the expected model prediction when conditioning on that feature.

 

Discussion

The creation of a machine or autonomous mechanism with intelligence has long been a cherished aspiration of researchers across several scientific and technical disciplines, and ANN can be utilized to address various technical and scientific difficulties [7]. Although established more than 50 years ago, ANN has garnered substantial interest and research in the early 1990s and still shows great promise for further exploration [9]. The applications of intelligent systems cover a wide range of areas, including the field of forensic odontology [9]. This study utilizes a combined approach of parameter measurement using deep learning, e.g., WebCeph, and ANN to conduct gender determination.

The accurate interpretation of the results of an ANN prediction model is crucial. This interpretation provides dependable user confidence, valuable insights for model enhancement, and an improved understanding of the modeling process [14]. However, the effectiveness of the ANN technique is dependent on a significant volume of acquired data, as demonstrated in previous research conducted by Vathsala Patil et al., Tanvi et al., Deeepthi Bharadwaj et al., Naveen et al., and Usha Jambunath et al. The investigations conducted had a sample size of over 500 radiographs [6, 15–17]. By assessing the precision rate, the ANN demonstrated effectiveness and accuracy in detecting gender, regardless of the specific diagnosis of the sample used. These systems can forecast future values of a certain process by considering several past samples within its domain [9]. As a result, ANN shows promise in the field of forensic odontology, particularly in the prediction identification of gender [18].

Lateral cephalometry has long been an essential tool for diagnosing orthodontic issues and creating treatment programs; it is now also applicable in the field of forensic odontology [18, 19]. In a prior investigation, Bao discovered that automatic analysis software conducts cephalometric measurements with a level of effectiveness that is nearly sufficient for clinical application [19].

A sizable dataset is required to create a ML model [20]. Handayani et al. determined that collecting data on cephalometry presents a greater challenge than panoramic radiography, as RSGMP Unair exclusively uses cephalometric radiography for patients undergoing orthodontic therapy [13]. In addition, the statistics between males and females exhibit a significant discrepancy, which can be attributed to the fact that women are more aware of dental esthetics than men. Handayani et al. also discovered data imbalances that prevent the accuracy of the results from exceeding 90%. The composition of the ANN model is 80% for training, 10% for validation, and 10% for testing, with precision values for males and females being 25% and 88%, respectively, in that study [13].

Table 5 summarizes the four scenarios of the ANN model. The first scenario is the ANN model without SMOTE and normalization, which achieves a precision rate of 78% in predicting gender. The second scenario is the ANN model with SMOTE and without normalization, which leads to a decrease in accuracy by 22%. The third scenario is the ANN model with SMOTE and normalization, which achieves a better accuracy result of 71%. The fourth scenario is the ANN model with normalization and without SMOTE, which yields a better result, of 76%. The discrepancy in these precision values can be attributed to the variations in the distribution of the number of samples for male and female cephalometry. In cases where the sample numbers are imbalanced, the accuracy value tends to favor larger sample sizes [21]. This finding is corroborated by the metrics of accuracy, precision, and recall obtained from ANN [22].

SMOTE is a resampling strategy that seeks to augment the number of samples in the minority class by generating synthetic samples in that class. SMOTE is utilized to stabilize datasets with a significantly imbalanced ratio. Upon analyzing the results of this study, the matrix classification presented in Table 6 shows that SMOTE can enhance the accuracy of male predictions but fall short in predicting females. Overall, the absence of normalization in SMOTE does not contribute to the improvement in accuracy and reduces the accuracy of the model. The findings of Elreedy and Atiya and Duan et al. indicated that the implementation of SMOTE can enhance the accuracy of samples in the minority class [23, 24]. However, when normalization is applied, SMOTE exhibits improved performance, which can be attributed to its capability to mitigate the problem of overfitting. In addition, the procedure for generating fresh synthetic samples differs from that of the multiplication method.

Cephalometric radiographs are examined to assess the SNA angles, SGA angles, and mandibular length parameters. We utilize the SHAP method, which is a Python-based metric, to measure the level of integration of the cephalometric components used in the ANN and determine their importance. The SHAP approach is utilized to aid ANN models in acquiring precise parameters to identify a novel category of parameter measurements [25]. Moreover, the SHAP approach aids in determining whether there exists a singular answer in this category that possesses a collection of desirable attributes. Hence, using the SHAP methodology in our research, we have ascertained that the mandibular angle derived from cephalometric radiographs emerges as the most dependable parameter for assessing gender prediction in the Indonesian population. This research pertains to the studies conducted by Sikka et al. and Patil et al, which showed significant variations in the mandible between males and females. This research shows that the selected characteristics, particularly the SGA angle, pose a problem in accurately predicting gender [6, 26].

ANNs possess a notable advantage in that they do not require any prior familiarity with system models [27]. This attribute is highly advantageous when used for the processing of files that exhibit missing or corrupted data. However, because of the exclusive operation of ANNs on the tasks for which they have been trained, a substantial amount of data needs to be used. To accomplish any objective, the data need to be retrained, and the augmentation of the image count within the cephalometry picture collection has the potential to exhibit improved performance [27, 28]. This study is hampered by the shortage of male samples in comparison to female samples, as well as the limited parameters employed, which can negatively impact the accuracy of gender prediction and do not measure the density of the cranium.

Conclusion

This study proposes the use of SMOTE as a solution to address the issue of imbalanced data in an ANN model used for gender identification. The efficacy of SMOTE in augmenting male samples has been demonstrated, although this study did not observe any improvement in accuracy. Among the four selected characteristics, the mandibular angle has a significant influence on gender prediction. To enhance performance, future research could involve expanding the dataset and conducting a thorough analysis of gender sample distribution. Furthermore, researchers should evaluate additional characteristics and parameters, such as skull density, that can be utilized in the study of unidentified cranium and mass disasters.

Additional information

Funding source. This research was funded by Indonesian Health Ministry.

Competing interests. The authors declare that they have no competing interests.

Authors’ contribution. All authors made a substantial contribution to the conception of the work, acquisition, analysis, interpretation of data for the work, drafting and revising the work, final approval of the version to be published and agree to be accountable for all aspects of the work. V.W. Handayani — data collecting, editing the article; A. Yudianto — supervisor; M. Sylvia M.A.R. — cephalometric manual data measures; R. Rulaningtyas — algorithm development; M.R. Caesarardhi — algortithm revised; R.H. Putra — image data filtering.

Acknowledgements. We express our gratitude to the Indonesian Health Ministry for providing financing for this project, as well as to Airlangga University for their cooperation in conducting this study.

About the authors

Vitria Wuri Handayani

Universitas Airlangga; Pontianak Polytechnic Health Ministry

Email: vitriawuri@gmail.com
ORCID iD: 0000-0002-5076-0118

MD, Medical Faculty; Nursing Department

Индонезия, Surabaya; Pontianak

Ahmad Yudianto

Universitas Airlangga

Author for correspondence.
Email: ahmad-yudianto@fk.unair.ac.id
ORCID iD: 0000-0003-4754-768X

MD, PhD, Professor, Department of Forensics and Medicolegal, Faculty of Medicine; Magister of Forensic Sciences, Postgraduate School

Индонезия, Surabaya

MAR Mieke Sylvia

Univesitas Airlangga

Email: mieke-s-m-a-r@fkg.unair.ac.id
ORCID iD: 0000-0001-8821-0157

MD, PhD, Professor, Forensic Odontology Department, Dental Medical Faculty

Индонезия, Surabaya

Rulaningtyas Riries

Universitas Airlangga

Email: riries-r@fst.unair.ac.id
ORCID iD: 0000-0001-7058-1566

MD, Physics Department, Sains and Technology Faculty; Biomedical Department, Sains and Technology Faculty

Индонезия, Surabaya

Muhammad Rasyad Caesarardhi

Institut Teknologi Sepuluh Nopember

Email: mrasyadc@gmail.com
ORCID iD: 0000-0002-6986-0346

MD, Department of Information Systems

Индонезия, Surabaya

Ramadhan Putra

Universitas Airlangga

Email: ramadhan.hardani@fkg.unair.ac.id
ORCID iD: 0000-0002-0622-3892

MD, Department of Dentomaxillofacial Radiology, Faculty of Dental Medicine

Индонезия, Surabaya

References

Supplementary files