Journal of Medical Systems

, Volume 36, Issue 5, pp 2721–2729

Diagnosis of Diabetes Diseases Using an Artificial Immune Recognition System2 (AIRS2) with Fuzzy K-nearest Neighbor

Authors

  • Mohamed Amine Chikh
    • Biomedical Engineering LaboratoryTlemcen University
    • Biomedical Engineering LaboratoryTlemcen University
  • Nesma Settouti
    • Biomedical Engineering LaboratoryTlemcen University
ORIGINAL PAPER

DOI: 10.1007/s10916-011-9748-4

Cite this article as:
Chikh, M.A., Saidi, M. & Settouti, N. J Med Syst (2012) 36: 2721. doi:10.1007/s10916-011-9748-4

Abstract

The use of expert systems and artificial intelligence techniques in disease diagnosis has been increasing gradually. Artificial Immune Recognition System (AIRS) is one of the methods used in medical classification problems. AIRS2 is a more efficient version of the AIRS algorithm. In this paper, we used a modified AIRS2 called MAIRS2 where we replace the K- nearest neighbors algorithm with the fuzzy K-nearest neighbors to improve the diagnostic accuracy of diabetes diseases. The diabetes disease dataset used in our work is retrieved from UCI machine learning repository. The performances of the AIRS2 and MAIRS2 are evaluated regarding classification accuracy, sensitivity and specificity values. The highest classification accuracy obtained when applying the AIRS2 and MAIRS2 using 10-fold cross-validation was, respectively 82.69% and 89.10%.

Keywords

Pima Indians diabetes data setDiagnosisAIRS2Fuzzy k- nearest neighbors

Introduction

Diabetes is a chronic illness that requires continuing medical care and patient self-management education to prevent acute complications and to reduce the risk of long-term complications. People develop diabetes because the pancreas does not make enough insulin or because the cells do not use insulin properly, or both. As a result, the amount of glucose in the blood increases while the cells are starved of energy. Insulin is a hormone, made in the pancreas by beta cells, that regulates blood sugar. Over the years, high blood glucose, also called hyperglycemia, damages nerves and blood vessels, which can lead to complications such as heart disease, stroke, kidney disease, blindness, nerve problems, gum infections, and amputation. The classification of diabetes includes two main clinical classes: Type 1 diabetes, which results from Beta-cell destruction, usually leading to absolute insulin deficiency, Type 2 diabetes, which results from a progressive insulin secretory defect on the background of insulin resistance [1].

A medical diagnosis is a classification process. Using the computer science to perform this classification is becoming more frequent. Even if the expert’s decision is the most important factor in diagnosis, classification systems provide substantial help as they reduce the errors due to fatigue and the time needed for diagnosis. There is a great variety of methods related to classification and diagnosis of diabetes disease in literature. In [2], a principal component analysis and adaptive neuro-fuzzy inference were used for diagnosing Pima Indian diabetes. They have reported 89.47% classification accuracy. Purnami et al. [3], obtained 93.2% classification accuracies using a new smooth support vector machine and its applications in diabetes disease diagnosis. In [4], Polat et al. used attribute weighted artificial immune System with 10-fold cross validation method; they obtained a classification accuracy of 75.87%. Ster and Dobnikar [6] accomplished 77.5%, 76.6%, 76.5%, 76.4%, 75.8% and 75.8% using respectively linear discriminant analysis, ASI, Fisher discriminant analysis, MLP + BP, LVQ and LFC with 10-fold cross validation.

Artificial immune recognition system has shown an effective performance on several problems such as medical classification problems. In [5], Polat et al. have used a fuzzy resource allocation combined with the AIRS algorithm. The classification accuracy obtained using 10-fold cross-validation was, respectively, 98.53% and 99.00% for classification of WBCD; 79.22% and 84.42% for classification of the Pima Indians diabetes data set; and 100% and 92.86% for classification of the ECG arrhythmia data set. Polat and Güneş used the same Fuzzy-AIRS algorithm to classify the diagnosis of lymph diseases in [16] and to improve the prediction accuracy of Escherichia coli promoter gene sequences in [17]; the proposed system has obtained respectively 90.00% and 90% prediction accuracy using 10-fold cross-validation. Polat et al [18] classified Breast cancer and liver disorders using artificial immune recognition system with fuzzy resource allocation mechanism and 10-fold cross-validation they have reported respectively an accuracy of 98.51% for breast cancer and 83.36% for the Liver Disorders dataset.

In this work we have proposed a new approach called MAIRS2 to recognize diabetes disease. In the first phase, we used the AIRS2 learning algorithm to reduce the size of the diabetes dataset; we obtained a reduced database named Memory Cells Pool. In the second phase, we apply the fuzzy k-nearest neighbor to overcome the limitations of the k-nn classifier by assigning a class membership to each patient. We evaluate the performances of our MAIRS2 algorithm by using the Pima Indians diabetes dataset.

This paper is organized as follows: "Theory" section introduces the used methods: artificial immune recognition systems and fuzzy-Knn algorithms. The results obtained in applications are given in "Experimentation and results" section. In "Conclusion" section, we conclude the paper.

Theory

Natural and artificial immune system

The immune system (IS) is a complex of cells, molecules and organs that represent an identification mechanism capable of perceiving and combating dysfunction from our own cells (infectious self) and the action of exogenous infectious microorganisms (infectious nonself). Two of the most important cells in this system are T cells, and B cells. The immune system consists of two complementary mechanisms, the innate and adaptive immune systems. The innate immune system has the ability to recognize certain microbes and immediately destroy them on first encounter. The adaptive immune system enables the body to recognize and respond to any microbe, even if it has never faced the invader before [7, 8].

The vertebrate immune system is particularly interesting due to its several computational capabilities, like: recognition, feature extraction, diversity, learning, memory, distributed detection, self-regulation, metadynamics, and immune network [9].

Artificial Immune Systems is the collective name for a number of algorithms inspired by the human immune system. AIS emerged in the 1990s as a new computational research area inspired by theoretical immunology and observed immune functions, principles and models like: clonal selection theory, negative selection theory, positive selection theory and immune network theory. Artificial Immune Systems (AIS) are being used in many applications such as anomaly detection, pattern recognition, data mining, computer security, adaptive control and fault detection [1012].

Artificial immune recognition system

Artificial Immune Recognition System (AIRS), is a supervised learning AIS that has shown significant success on broad range of classification problems. The immune mechanisms used by AIRS resource competition, clonal selection, affinity maturation and memory cell formation. The terms and concepts used in AIRS are [13]:
  • Affinity: a measure of “closeness” or similarity between two antibodies or antigens.

  • Affinity threshold scalar (ATS): a value between 0 and 1 that, when multiplied by the affinity threshold, provides a cut-off value for memory cell replacement in the AIRS training routine.

  • Artificial Recognition Ball (ARB): also known as a B-Cell. It consists of an antibody, a count of the number of resources held by the cell, and the current stimulation value of the cell.

  • Candidate Memory Cell: the antibody of an ARB, of the same class as the training antigen, which was the most stimulated after exposure to the given antigen.

  • Clonal rate: an integer value used to determine the number of mutated clones a given ARB is allowed to attempt to produce.

  • Hyper-mutation rate: an integer value used to determine the number of mutated clones a given memory cell is allowed to inject into the cell population.

  • Mutation rate: a parameter between 0 and 1 that indicates the probability that any given feature (or the output) of an ARB will be mutated.

  • Resources: a parameter which limits the number of ARBs allowed in the system. Each ARB is allocated a number of resources based on its stimulation value and the clonal rate.

  • Stimulation threshold: a parameter between 0 and 1 used as a stopping criterion for the training on a specific antigen.

The AIRS algorithm

This algorithm is composed of four main stages (Fig. 1) [5, 1317]:
  1. 1.
    Initialization:
    1. a.

      Normalize all items in the data set such that the Euclidean distance between the feature vectors of any two items is in the range of [0, 1].

       
    2. b.
      After normalization, the affinity threshold is calculated.
      $$ affinity\;threshold = \frac{{\mathop{\sum }\nolimits_{{i = 1}}^n \mathop{\sum }\nolimits_{{j = i + 1}}^n affinity(a{g_i},a{g_j})}}{{\frac{{n(n - 1)}}{2}}} $$
      (1)
      Where n is the number of training data items (antigens), agi and agj are the ith and jth training antigen in the antigen training data set.
      $$ affinity(a{g_i},a{g_j}) = 1 - Euclidean{ }distance(a{g_i},a{g_j}) $$
      (2)
       
    3. c.

      Create a random base called the memory pool (M) and the ARB pool (P) from training data.

      Antigenic Presentation: for each antigenic pattern do:

       
     
  2. 2.
    Memory cell identification and ARB generation:
    1. a.
      Clonal Expansion: For each element of M determines their affinity to the antigenic pattern, which resides in the same class. Select highest affinity memory cell (mcmatch) and clone mc in proportion to its antigenic affinity to add to the set of ARBs (P).
      $$ m{c_{{match}}} = argmax(stimulation(mc,ag)) $$
      (3)
      $$ stimulation(mc,ag) = \{_{{1 - affinity{ }otherwise}}^{{affinity\left( {mc,ag} \right)if{ }mc.class = ag.class}} $$
      (4)
       
    2. b.

      Affinity Maturation: Mutate each ARB descendant of mcmatch. Place each mutated ARB into P.

       
     
  3. 3.
    Competition for resources and development of a candidate memory cell:
    1. a.
      Metadynamics of ARBs: Process each ARBs through the resource allocation mechanism. This will result in some ARB death, and ultimately controls the population. Calculate the average stimulation for each ARB, and check for stopping criterion.
      $$ {s_i} = \frac{{\mathop{\sum }\nolimits_{{j = 1}}^{{\left| {AR{B_i}} \right|}} ar{b_j}.stimulation}}{{\left| {AR{B_i}} \right|}}ar{b_j} \in AR{B_i} $$
      (5)

      The stopping criterion is reached if sistimulation-threshold for all elements in s = {s1, s2,…, snc}.

       
    2. b.

      Clonal Expansion and Affinity Maturation: Clone and mutate a randomly selected subset of the ARBs left in P based in proportion to their stimulation level.

       
    3. c.

      Cycle: While the average stimulation value of each ARB of the same class as the antigen is less than a given stimulation threshold repeat from step 3.a.

       
     
  4. 4.

    Memory cell introduction:

    Metadynamics of Memory Cells: Select the highest affinity ARB from the last antigenic interaction. If the affinity of this ARB with the antigenic pattern is better than that of the previously identified best memory cell mc then add the candidate (mc-candidate) to memory set M. additionally, if the affinity of mcmatch and mc-candidate is below the affinity threshold, and then remove mcmatch from M.

    Cycle: Repeat steps 2, 3, 4 until all antigenic patterns have been presented.
    https://static-content.springer.com/image/art%3A10.1007%2Fs10916-011-9748-4/MediaObjects/10916_2011_9748_Fig1_HTML.gif
    Fig. 1

    AIRS algorithm

     

A more efficient AIRS: AIRS2

The changes made to the AIRS algorithm are small, but end up having an interesting impact on both the simplicity of implementation and on the quality of results. These changes can be identified as follows [10]:
  1. a.

    Only the Memory Cell pool is seeded during initialization and the ARB pool is seeded for each antigen. If we initialize the ARB Pool during initialization, then ARBs left over from previous ARB refinement passes (for past antigens) are maintained and participate in the competition for limited resources. So the algorithm spends more time rewarding and refining ARBs.

     
  2. b.

    During the clonal expansion from the matching memory cell used to populate P, the newly created ARBs are no longer allowed to mutate class because only clones of the same class as the antigen are considered in the ARB pool.

     
  3. c.

    Due to changes cited previously; resources are only allocated to ARBs of the same class as the antigen.

     
  4. d.

    During affinity maturation (mutation), a cell’s stimulation level is taken into account. Each individual gene is only allowed to change over a finite range. This range is centered with the gene’s pre-mutation value and has a width size of the difference of 1.0 and the cell’s stimulation value. In this way the mutated offspring of highly stimulated cells (those whose stimulation value is closer to 1.0) are only allowed to explore a very tight neighborhood around the original cell, while less stimulated cells are allowed a wider range of exploration.

     
  5. e.

    Knowing that the ARB pool contains only the cells of the same class as the antigen, the training stopping criterion no longer takes into account the stimulation value of ARBs in all classes, but now only accounts for the stimulation value of the ARBs of the same class as the antigen.

     

K-nearest neighbor

The classification is performed in a k-nearest neighbor approach. K-nearest-neighbor (kNN) is a classification algorithm and one of the most important methods in nonparametric algorithm. Each memory cell is iteratively presented with each data item for stimulation. The system’s classification of a data item is determined by using a majority vote of the outputs of the k most stimulated memory cells. The algorithm is as follows [19]:
https://static-content.springer.com/image/art%3A10.1007%2Fs10916-011-9748-4/MediaObjects/10916_2011_9748_Figa_HTML.gif

Fuzzy-Knn

There are some problems with k-NN algorithms. One of these problems is that normally each of the neighbors is considered equally important in the assignment of the class label of the input vector. Another problem is that when an input vector is assigned to a class, it does not determine the strength of membership in this class. Keller et al. [16], propose fuzzy k-NN classifier to overcome the limitations of the k-NN classifier. The fuzzy k-NN algorithm assigns class membership to a sample vector rather than assigning the data to a particular class. The basis of the algorithm is to assign membership as a function of the vector’s distance from its k-nearest neighbors and those neighbors’ membership in the possible classes. The algorithm is as follows [19, 20]:
https://static-content.springer.com/image/art%3A10.1007%2Fs10916-011-9748-4/MediaObjects/10916_2011_9748_Figb_HTML.gif

Where {x1, x2,… xn} be the set of n labeled samples. uij is the membership in the ith class of the jth vector of labeled sample set, the parameter m determines how heavily the distance is weighted when calculating the class membership and \( \left\| {x - {x_j}} \right\| \) is the distance between x and its jth nearest neighbor xj . The pattern x is assigned to the class given by : \( \arg \max_{{i = 1}}^{{nc}}(ui(x)) \).

There are different methods to assign membership for the labeled data. A crisp labeling method is to assign each labeled sample complete membership in its known class and zero membership in all other classes. A fuzzy method is to assign membership to the labeled samples according to a k-nn rule. The following equation assigns class membership to the k-nn of each sample x (say x in class i):
$$ {u_j}(x) = \{_{{\left( { \frac{{{n_j}}}{k} } \right)*0.49{ }if{ }i \ne j}}^{{0.51 + \left( { \frac{{{n_j}}}{k} } \right)*0.49{ }if{ }i = j}} $$
(6)

Where nj is the number of neighbors amongst the k closest labeled reference patterns which are labeled in class j.

Experimentation and results

Pima Indians diabetes data set

The proposed method has been tested using the public Pima Indian Diabetes dataset of National Institute of Diabetes and Digestive and Kidney Diseases. Pima Indians of Arizona have the highest prevalence and incidence of diabetes Type 2 of any population in the world. This dataset contains information on various medical measurements on 768 individuals, 500 of these samples belong to persons with no diabetes problem while the remaining 268 sample are of persons with diabetes. Noting that there are some records with missing data. After removing these cases with unreasonable physical data, the total number of cases is 392 where 262 are normal cases and 130 are diabetes cases. The class information contained in this data set is given by 0 for healthy persons and by 1 for diabetic patients. The number of attributes in samples is 8. These attributes are [4]:
  1. 1.

    Number of times pregnant.

     
  2. 2.

    Plasma glucose concentration a 2 h in an oral glucose tolerance test.

     
  3. 3.

    Diastolic blood pressure (mm Hg).

     
  4. 4.

    Triceps skin fold thickness (mm).

     
  5. 5.

    2-h serum insulin (mu U/ml).

     
  6. 6.

    Body mass index (weight in kg/(height in m)^2).

     
  7. 7.

    Diabetes pedigree function.

     
  8. 8.

    Age (years).

     
  9. 9.

    Class variable (0 or 1).

     

K-fold cross validation

In k-fold cross-validation the data is first partitioned into k equally (or nearly equally) sized segments or folds. Subsequently k iterations of training and validation are performed such that within each iteration a different fold of the data is held-out for validation while the remaining k-1 folds are used for learning. Then the average error across all k trials is computed. The advantage of this method is that it matters less how the data gets divided since every sample gets to be in a test set exactly once, and gets to be in a training set k-1 times. The variance of the resulting estimate is reduced as k is increased. The disadvantage of this method is that the training algorithm has to be rerun from scratch k times, which means it takes k times as much computation to make an evaluation [4].

Evaluation parameters

In this paper, we use two methods to classify the Pima Indian Diabetes dataset: AIRS2 and AIRS2 with fuzzy-Knn (MAIRS2). In this section, the performance evaluations of the proposed methods are presented; the experimental results are given and discussed. The parameters values of training algorithm are shown in Table 1.
Table 1

Parameters used in training algorithm

Parameters

Value

Mutation rate

0.2

Stimulation threshold

0.9

Clonal rate

10

Hyper clonal rate

2

k value

3

K Fold

10

The performances of the two methods have been tested with different measures: the classification accuracy, sensitivity and specificity values, and confusion matrix.
  1. a.
    Classification accuracy (Acc%)
    $$ accurcy(T) = \frac{{\mathop{\sum }\nolimits_{{i = 1}}^{{\left| T \right|}} assess({t_i})}}{{\left| T \right|}} $$
    $$ assess(t) = \{_{{0{ }otherwise}}^{{1{ }if{ }classif(t) = t.c}} $$

    Where T is the test set, t ∈ T, t.c is the class of the item t and classify (t) returns the classification of t by the algorithm.

     
  2. b.
    Sensitivity (Se%)
    $$ Sensitivity = \frac{{TP}}{{TP + FN}} $$
     
  3. c.
    Specificity (Sp%)
    $$ Specificity = \frac{{TN}}{{TN + FP}} $$
    Where TP, TN, FP and FN denote respectively:
    • True positives: classifies Diabetic as Diabetic;

    • True negatives: classifies No Diabetic as No Diabetic;

    • False positives: classifies No Diabetic as Diabetic;

    • False negatives: classifies Diabetic as No Diabetic.

     
  4. d.

    Confusion matrix

     
A confusion matrix contains information about actual and predicted classifications done by a classification system. Confusion matrix identifies the common misclassifications of the proposed classification schema (Table 2).
Table 2

Confusion matrix

Actual

Predicted

Negative

Positive

Negative

a

b

Positive

c

d

The entries in the confusion matrix have the following meaning:
  • a is the number of correct predictions that an instance is negative,

  • b is the number of incorrect predictions that an instance is positive,

  • c is the number of incorrect of predictions that an instance negative,

  • d is the number of correct predictions that an instance is positive.

Results and discussion

In this study, diabetes disease diagnosis was conducted using a hybrid medical decision support system based on MAIRS2. The experimental study was performed with 392 sample (262 healthy, 130 diabetics) using 10-fold cross validation method.

The experiments were performed to generate the optimum number of memory cells that will be used for classification purposes using fuzzy K-nn algorithm. It is shown that the ATS (Affinity Threshold Scalar) is the parameter which determines the number of memory cells. The value of this parameter was selected in the range [0 1]. If this selected ATS values are too low, more memory cells are generated but if they are too high the number of memory cells will be too low. Another parameter which has an effect on classification accuracy was the number of resources. We have performed several trials in order to obtain the optimum values of the ATS and the number of resources that give highest classification accuracy. The best classification accuracy (89.10%) was obtained with ATS = 0.12 and number of resources = 200.

The relation between resource number, classification accuracy, sensitivity and specificity values, size of memory cells set and ATS is shown in Table 3. This table presents the results obtained using the AIRS2 learning algorithm with the k-nn method. Tables 4 and 5, present the results obtained by the MAIRS2 using respectively the crisp and the fuzzy labeling methods.
Table 3

The classification accuracies, sensitivity and specificity values for the AIRS2

Resource number

ATS

MC set

Accuracy (%)

sensitivity

specificity

200

0.1

365

80.12

61.81

89.31

200

0.2

132

69.14

58.97

74.40

200

0.12

280

82.69

73.47

87.55

300

0.12

282

81.16

73.46

84.35

300

0.13

248

81.62

70.79

88.25

Table 4

The classification accuracies, sensitivity and specificity values for the MAIRS2 (Crisp method)

Resource number

ATS

MC set

Accuracy (%)

Sensitivity (%)

Specificity (%)

200

0.1

365

91.40

85.85

94.90

200

0.2

132

72.24

61.19

76.29

200

0.12

280

89.10

85.18

91.50

300

0.12

282

85.56

85.56

85.44

300

0.13

248

86.26

76.78

91.89

Table 5

The classification accuracies, sensitivity and specificity values for the MAIRS2 (Fuzzy method)

Resource number

ATS

MC set

Accuracy (%)

Sensitivity (%)

Specificity (%)

200

0.1

365

86.75

75.50

93.50

200

0.2

132

73.78

61.23

81.36

200

0.12

280

84.51

74.57

89.90

300

0.12

282

86.76

81.67

89.46

300

0.13

248

84.96

73.12

90.94

In Table 3, AIRS2 showed an accuracy of 82.69% using 280 memory cells and a degree of data reduction of 28.57%, whereas in Table 4 and 5, MAIRS2 with the crisp labeling method and MAIRS2 with the fuzzy labeling method obtained respectively 89.10% and 84.51% using the same memory cells pool. With 248 memory cells, we have a degree of data reduction of 36.73% and an accuracy of 81.62%, 86.26% and 84.96% using respectively AIRS2, MAIRS2 with crisp labeling method and MAIRS2 with fuzzy labeling method.

Table 6 gives the confusion matrix showing the classification results of this classifier on 90–10% training-test split from the diabetes diseases dataset. In this case, the classification accuracy obtained with AIRS2 classifier algorithm was 82.5%. The MAIRS2 with crisp labeling method obtains 87.5% classification accuracy using 169 memory cells (data reduction is 56.89%).
Table 6

Confusion matrices of the AIRS2 and MAIRS algorithms using a 90%–10% training–test split (for resources = 200, ATS = 0.12 and 169 memory cells)

Actual

Predicted

Diabetics

Non- Diabetics

Method used

Diabetics

11

3

AIRS2

Non- Diabetics

4

22

Diabetics

12

2

Fuzzy-AIRS

Non- Diabetics

3

23

The medical decision support system presented in literature for classification of Pima Indian Diabetes dataset use the complete dataset, Table 7 presents the obtained classification accuracies of these classifiers. It can be shown from these results that the obtained classification accuracy by combination of AIRS2 and fuzzy-Knn classifier for diabetes disease was among the best classifier report from literature. Whereas our method couldn’t reach the highest classification accuracy for the problem, it has been over-performed to another AIS algorithm: AWAIS, AIRS and AIRS with fuzzy resource allocation mechanism which obtained respectively 75.87%, 79.22%, 84.42% classification accuracies.
Table 7

The classification accuracy of our method and classification accuracies obtained by other methods in the literature

Method

Accuracy (%)

smooth support vector machine

93.2%

principal component analysis and adaptive neuro-fuzzy inference

89.47%

MAIRS2

89.10%

AIRS with fuzzy resource allocation mechanism

84.42%

AIRS2

82.69%

AIRS

79.22%

linear discriminant analysis

77.5%

ASI

76.6%

Fisher discriminant analysis

76.5%

MLP + BP

76.4%

AWAIS

75.87%

LFC

75.8%

LVQ

75.8%

Conclusion

Classification systems that are used in clinical diagnosis allow medical data to be examined in a shorter time and in more detail. In the research reported in this paper, MAIRS2 was applied to recognize diabetes disease. Our method was validated on the Pima Indian diabetes dataset. We noted that an increase in the number of memory cells led to a better recognition rate but in other hand, it diminishes the degree of data reduction. In this work, we have achieved a good tradeoff between classification accuracy and data reduction. Thus the combination of AIRS2 and fuzzy K-nn led us to a new system (MAIRS2) that performed better than classical AIRS2 implemented in this paper and others AISs algorithms cited in literature. Finally, we hope that the use of others classification techniques such as SVM and others affinity measure methods such as Manhattan distance increase the recognition rate of the diabetes disease.

Copyright information

© Springer Science+Business Media, LLC 2011