Research and Reviews - International Journals

International Journal of Advanced Research in Electrical, Electronics and Instrumentation Engineering

Menu

ISSN ONLINE(2278-8875) PRINT (2320-3765)

All submissions of the EM system will be redirected to Online Manuscript Submission System. Authors are requested to submit articles directly to Online Manuscript Submission System of respective journal.

Image Fusion Using Type-2 Fuzzy Systems

Swathy Nair1, Bindu Elias1, VPS Naidu2
  1. Department of Electrical and Electronics, Mar Athanasius College of Engineering, Kothamangalam, Kerala, India
  2. Multi Sensor Data Fusion Lab, CSIR- National Aerospace Laboratories, Bangalore, Karnataka, India
Related article at Pubmed, Scholar Google

Visit for more related articles at International Journal of Advanced Research in Electrical, Electronics and Instrumentation Engineering

International Journal of Advanced Research in Electrical, Electronics and Instrumentation Engineering

Abstract

Image Fusion is a process of combining images from different sensors in order to get a single image having relevant information from all the sensors. Fuzzy logic based image fusion is introduced in order to incorporate uncertainty to the fusion logic since pixel calculation of the input image is not that certain and crisp. Recently studies are going on in Type-2 Fuzzy sets which can handle higher levels of uncertainties. Image Fusion algorithms using different types of Type-2 FLS s are developed and tested. It was observed that type-2 FLSs gives better values of Fusion quality performance metrics than Type-1 FLS. Among Type-2 FLSs, Type-2 Sugeno outperformed Mamdani. In Type-2 Mamdani FLSs, Type-2 Non-singleton type-2 Mamdani FLS was showing good results than the other two.

Keywords
Type-2 fuzzy systems, Singleton, non-singleton, performance metrics.

INTRODUCTION
Image Fusion is a process of combining images from different sensors in order to get a single image having relevant information from all the sensors. Pixel Level image fusion is a fusion method in which Fusion is done pixel by pixel on input images. Different image Fusion techniques have been discussed in many literatures such as weighted Average, High Pass Filter (HPF), Intensity Hue Saturation (IHS), Principal Component Analysis (PCA), Pyramid Based Decomposition, Discrete Cosine Transform(DCT), Discrete Wavelet Transform (DWT), Stationary Wavelet Transform (SWT), Fuzzy logic etc [1].
In [2], an image fusion technique using DCT is introduced and image fusion quality is analyzed. In DCT only spatial correlation of the pixels inside the single 2-D block is considered and the correlation from the pixels of the neighboring blocks is neglected, and it is impossible to completely decorrelate the blocks at their boundaries using it. DWT based image fusion has the advantage of multi scale and multi resolution. Because of their inherent multiresolution nature, wavelet coding schemes are especially suitable for applications where scalability and tolerable degradation are important. Also, it allows good localization both in time and spatial frequency domain. In [3] an image fusion technique using DWT and PCA is introduced and different image fusion techniques are compared using performance evaluation metrics. SWT based image fusion is discussed in [4], in which the translational variance of DWT has been eliminated and there by fusion quality is increased.
Real time systems have high levels of linguistic and numerical uncertainties. With the invention of fuzzy logic by Professor Lotfi Zadeh in 1965, it is treated as the adequate methodology for treating uncertainties and imprecision in real time systems. Fuzzy logic based image fusion is introduced in order to incorporate uncertainty to the fusion logic since pixel calculation of the input image is not that certain and crisp [5]. In fuzzy based image fusion imprecision of image fusion algorithm is also taken into consideration. By proper tuning of membership function and proper formulation of rule base, good quality image fusion results can be obtained [6][7]. The advantages of both SWT and Fuzzy algorithms are incorporated together for image fusion to get high quality image fusion results [8]. FLSs using type-1 fuzzy sets are considered as the first generation fuzzy sets. Recently studies are going on in Type-2 Fuzzy sets which can handle higher levels of uncertainties [9] [10].
This paper is to apply Type-2 Fuzzy logic in image fusion. In the paper different types of Type-2 Fuzzy systems are implemented for image fusion and fusion quality is compared based of Fusion quality Performance Metrics [3]. A software tool for Type-2 Fuzzy logic system has been developed in [11]. This software tool is used in Matlab for the implementation of image fusion algorithm. The prerequisite of image fusion is image registration. In this paper, it is assumed that input images are registered.

TYPE-2 FUZZY LOGIC
Type-2 fuzzy system proponents argue that Type-1 fuzzy systems are too crisp. This is because the membership function edges are crisp even if they are uncertain. The main difference between type-1 and type-2 fuzzy sets lies in the creation of their membership functions. There lie some uncertainties in how to define the edges of membership function. The term for this area is “Footprint of Uncertainty,” or FOU. If the domain of interest is not well understood, it is difficult to model the data. By fuzzifying the edges of the membership functions, the FOU can be modeled. Interval Type-2 fuzzy logic System (IT2FLS), a special case of type-2 fuzzy systems are used to handle the data with high levels of uncertainties.
An IT2FLS A with its membership function μA(x,u) can be defined as:
image
Where, x∈X is the primary variable and u∈J is the secondary variable (which has values in between 0 and 1). An IT2FLS can be pictorially represented as in [9].
image
In Fig.1, it is seen that the membership grade for each value of x is an interval unlike that of a Type-1 fuzzy set in which membership grade for each value of x is a number. So the membership function (MF) is bounded with two
image
image
The major difference between the Type-2 FIS from Type -1 is at least one of the fuzzy system in the rule base is Type-2. So the output of the inference engine will be Type-2 and thus a type reducer is needed to convert the type-2 inference output to type-1. Then it is undergone defuzzification to get the crisp set. This is the case of Type-2 Mamdani FIS. In Sugeno FIS the output of inference engine is Type-1 and hence type reducer is not needed. Different stages in IT2FIS for image fusion are:
• Formulation of Rule base: Determining a set of fuzzy rules ( usually If…Then rules) according to which fusion is to be done
• Input Fuzzification: Type-2 Membership functions defined on input images are applied to their actual values so that the degree of truth for each rule premise can be determined and there by input images are converted to type-2 fuzzy sets
• Inference: Truth value for the premise of each rule is computed and applied to the conclusion part of each rule. This results in one output type-2 fuzzy set to be assigned to each output variable for each rule. The consequent of a fuzzy rule assigns an entire type-2 fuzzy set to the output. If the antecedent is only partially true, then the output fuzzy set is truncated according to the implication method
• Type Reduction: The outputs of each rule are combined into a single type-2 fuzzy set. The type-2 fuzzy set from the inference engine is converted to a type-1 Fuzzy set using any of the type reduction method
• Defuzzification: The fuzzy output set is converted to a crisp number
In this paper, the linguistic variables are chosen based on the pixel value, which indicates level of brightness. So the linguistic variables are selected as VH (Very High), H (High), M (Medium), L (Low) and VL (Very Low). Rules are formulated according to Table-1.
image
image
I. Type-2 Mamdani FLS
Both Mamdani and Sugeno kind of FLS s are characterized by If…Then rules. In a Mamdani FLS, the consequents of its rules are fuzzy sets. Type-2 Mamdani rules gives type-2 fuzzy sets as its consequents. The rule formulation is same for all types of Type-2 Mamdani FLSs. The difference comes in the type of fuzzification. The classification is done according to the type of fuzzifier used [12].
A. Singleton FLS
In a singleton FLS, the fuzzifier maps crisp inputs x =(x1 , x2 ......x p ) into a singleton type-2 fuzzy set as shown in Fig.4:
image
B. Non singleton FLS
In non singleton Fuzzy sets, inputs are modeled as fuzzy numbers. A type-2 FLS whose inputs are modeled as Type-1 fuzzy numbers is called as Type-1 Non singleton type-2 fuzzy FLS (as shown in fig. 5). Where as a type-2 FLS whose inputs are modeled as Type-2 fuzzy numbers is called as Type-2 Non Singleton type-2 FLS (as shown in Fig.6).
image
In Figs. 4-6, x =(x1 , x2 ......x p ) shows the inputs and F&f show the fuzzified inputs.
II. Type-2 Sugeno FLS
The difference from Sugeno from Mamdani is, the consequent of a Mamdani rule is a fuzzy set whereas that of a Sugeno rule is a function. In a Type-2 Sugeno FLS, the output of inference engine is a type-1 fuzzy set (because it is a linear combination of type-1 fuzzy sets). Thus for a Type-2 Sugeno FLS, there is no need of type reduction just like there is no need of defuzzification in Type-1 Sugeno FLS.
The algorithm used for image fusion is as follows:
Step 1: Read input images ( 1 2 I & I ) to be fused and covert them to column vector
Step 2: Form a Matlab ‘fis’ file with two inputs and decide type (Type-2 Singleton Mamdani/Type-1 Non Singleton Type-2 Mamdani/ Type-2 Non Singleton Type-2 Mamdani/Type-2 Sugeno) and number of membership functions for input images and output image
Step 3: Formulate rules according to which output fuzzy sets are obtained from input fuzzy sets
Step 4: According to the rule base, inference is done to get a inferred type-2 fuzzy set for each rule. Aggregate it to get the output Type-2 fuzzyset
Step 5: Do type reduction to get a type-1output fuzzyset from the type-2 fuzzy set
Step 4: Defuzzify the output type-1 fuzzy set to get crisp output and convert the column form to matrix form to get fused image f I
In this paper, image fusion is implemented using all the above mentioned FLS types and comparison is made based upon Performance Evaluation metrics. The performance evaluation metrics used for comparison in this paper is discussed in next section.

QUALITY EVALUATION METRICS USED FOR IMAGE FUSION
The quality of fused images obtained from algorithms using different types of Type-2 FLSs are compared using Fusion Quality Performance Evaluation Metrics. Some performance metrics uses a reference image for calculation and others not. Evaluation metrics are calculated for all algorithms and compared to find out the best algorithm [3].
A. With Reference Image
For datasets having reference image, fusion quality could be evaluated using the following evaluation metrics:
1. Root Mean Square Error(RMSE)
RMSE is computed as the root mean square error of the corresponding pixels in the reference image r I and the fused image f I . The RMSE between a reference image and the fused image is given by:
image'
image
image
image

RESULTS AND DISCUSSIONS
Image fusion algorithms using different types of Type-2 FLSs described in section II are implemented on 2 datasets in Matlab. For developing type-2 ‘fis’ file in Matlab, software tool mentioned in [11] is used. For evaluating a type-2 ‘fis’ file, a Matlab code was developed in [13]. An image set consisting CSIR-NAL indigenously developed SARAS images is used for testing different algorithms. The input images ( I1 and I 2 ) are obtained by blurring the true image (reference image r I ) as shown in Figs 7 and 8. The comparison of different algorithms is done using the fusion quality performance evaluation metrics described in section-III.
image
image
Image fusion algorithms using different type of type-2 fuzzy sets are done on Dataset-1. Results of each algorithm are discussed in following sub sections. Here the ‘fis’ parameters taken for all the methods are as following:
And Method: 'min'
Or Method: 'max'
Type Reduction Method: 'center of sets'
Defuzzification Method: 'centroid'
Implication Method: 'prod'
Aggregation Method: 'max'
Membership Function Type: ‘Gaussian’
No: of Membership Functions: 5
No: of Rules: 25 (According to Table-1)
i. Type-2 Singleton Mamdani
As explained before the input set of Type-2 Singleton Mamdani is a singleton set. So here the algorithm is tested for different FOU (uncertainty width) of the membership functions. The performance metrics obtained for different FOUs are tabulated in Table-2 & 3.
image
image
Here execution time is so large for all types because all the programs are written in Matlab. If the evaluation code can be written in C/C++, the execution time can be reduced to a great extend. So here execution time is not taking as an evaluation metric. The bold digits show the best values of performance metrics. From the Table- 2and 3 it is observed that Type-2 is giving better fusion results than Type-1 (with FOU=0). The fusion quality is the best for a FOU of 0.05 which is in between 0 and 0.1. Thus it is observed that Type-2 singleton Mamdani FLS is suitable for systems which have a medium uncertainty level. Image fusion quality decreases with the increase and decrease of FOU from 0.05.
ii. Type-1 Non-Singleton Type-2 Mamdani
In this type of FLS, the inputs are modeled as Type-1 Fuzzy sets. In this case, for a ‘fis’ structure having particular parameter set, the fusion quality depends on FOU and the Standard deviation of input fuzzy set. Performance metrics are calculated for each case and tabulated in Tables 4 and 5.
image
image
From Tables 4 and 5 it is observed that for a fixed input SD, Fusion quality increases as FOU increases till 0.05 and then decreases. Also from Tables 2-5, it is observed that fusion quality is better for Type-1 Non-singleton type-2 FLS than type-2 singleton FLS. It is also observed that increase in SD also result in increase in Fusion quality till SD is 0.05 and then decreases. So in the case of Type-1 Non-singleton type-2 FLS, it is observed that an FLS with input SD 0.05 and membership FOU 0.05 gives the best fusion result.
iii. Type-2 Non-Singleton type-2 Mamdani
In this type of FLS, the inputs are modeled as Type-2 Fuzzy sets. In this case, for a ‘fis’ structure having particular parameter set, the fusion quality depends on FOU of the membership function and FOU of input fuzzy set. Performance metrics are calculated for each case and tabulated in Tables 6 and 7.
 
image
image
image
From Tables 6 and 7 it is observed that for a fixed input FOU, Fusion quality increases as FOU increases till 0.05 and then decreases. Also from Tables 2-7, it is observed that fusion quality is better for Type-2 Non-singleton type-2 FLS than type-2 singleton FLS and Type-1 Non-singleton type-2 FLS. It is also observed that increase in SD also result in increase in Fusion quality till SD is 0.05 and then decreases. So in the case of Type-2 Non-singleton type-2 FLS, it is observed that an FLS with input FOU 0.05 and membership FOU 0.05 gives the best fusion result.
iv. Type-2 Sugeno FLS
In a Type-2 Sugeno FLS, the output of inference engine is a type-1 fuzzy set (because it is a linear combination of type-1 fuzzy sets). Thus for a Type-2 Sugeno FLS, there is no need of type reduction just like there is no need of defuzzification in Type-1 Sugeno FLS. So time needed for type-2 Sugeno FLS will be less than that of Mamdani. Here the image fusion algorithm using type-2 Sugeno FLS is tested for different FOU (uncertainty width) of the membership functions for Dataset-1. The performance metrics obtained for different FOUs are tabulated in Table- 8& 9.
image
Table-9 Performance Evaluation Metrics for Different FOUs of type-2 Sugeno without reference image for Dataset-1
image
From the Table- 8 and 9 it is observed that Type-2 is giving better fusion results than Type-1 (with FOU=0). The fusion quality is the best for a FOU of 0.05 which is in between 0 and 0.1 in case of Sugeno also. From Tables 2-9, it is observed that Type-2 Sugeno gives better result than Mamdani. And in Type-2 Mamdani, Type-2 non-singleton type-2 Mamdani outperforms the other two. The fused and error images with best parameters of all FLSs are shown in Figs 9 and 10.
image

CONCLUSION
Image Fusion algorithms using different types of Type-2 FLS s are developed and tested o. It was observed that type-2 FLSs gives better values of Fusion quality performance metrics than Type-1 FLS. Among Type-2 FLSs, Type-2 Sugeno outperformed Mamdani. In Type-2 Mamdani FLSs, Type-2 Non-singleton type-2 Mamdani FLS was showing good results than the other two.

References