Neural Networks for Classification and Pattern Recognition of Biological Signals RENATO M.E. SABBATINI, PhD, SENIOR MEMBER, IEEE Center for Biomedical Informatics, State University of Campinas, P.O. Box 6005, Campinas, SP 13081-970, BRAZIL Artificial neural networks (ANN) can be used to represent and to process information by means of networks of interconnected processing elements, similar to neurones. Several emerging global properties of ANNs, such as associative memory, distributed parallel processing, learning, etc., have favored its applications in a large variety of tasks involving pattern classification and recognition, particularly in biological signals and images and neural-based intelligent biomedical instrumentation. This paper reviews the field in its latest developments and points out potential applications for intelligent biomedical instrumentation and control. INTRODUCTION Artificial neural networks, also interchangeably called connectionist or neuromorphic systems, have been shown to be very useful and effective alternatives for implementing intelligent systems (application software and instruments) in several areas of Biology and Medicine, mainly in those needing complex pattern recognition and classification, such as processing and understanding signals and images [1, 2]. In this paper, we review the current status of ANN applications in biosignal pattern recognition and classification. General information on ANN history, available techniques and applications can be found elsewhere [e.g., 3] ANNs In Biological Signal Processing Artificial neural networks lend themselves very naturally to applications involving biological signal processing and recognition [4]. It has been proved that ANNs perform better and quicker than conventional methods in situations where noise and uncertainty is present, and that they can be used to implement several functions, such as data compression [5], digital filters, artifact detection, function optimization, time-series and Fourier analysis, classification and recognition of complex patterns, etc. Detection, identification and classification of patterns which may be present in a biosignal record is of great importance for the development of a variety of biomedical applications: - closed-loop control of automatic systems [6]; - intelligent real-time computer-based instruments [7]; - non-invasive intelligent monitoring [8]; - prediction of time-ordered trends and sequences [9]; - automatic interpretation and diagnosis of biological signal recordings [10,11]; - adaptive control of bioprosthetic devices [12]; - voice control of biomedical devices [13] - localization of signal sources [14]; - adaptive noise cancellation and signal separation [15]; - automatic peak detection [16], selection and segmentation [17]; - signal decomposition [18]; - complex multichannel signal analysis and classification [19]; - morphological and template matched filtering [20]; PRACTICAL IMPLEMENTATION The most common use of ANNs is as a pattern classifier or recognizer. For example, in a feedforward, multilayer ANN, an input vector I is applied to the n nodes of the input layer. The activation pattern processed by this layer is propagated to the successive layers, until it reaches the output layer, whereby it will produce an output vector O. This operation corresponds to a mapping of I into O, and it has several interesting applications. In a heteroassociative network, the nature of input and output patterns is different. For example, I could be a sequence of time-ordered ECG samples, while O would represent diagnostic classes (beat-to-beat ECG interpretation). In this way, ANNs can be easily used to classify and to interpret signals and images. In an autoassociative network, the input and output patterns are of the same nature. In the same ECG example given above, the output layer would be also an array of signal samples and the network could be used to perform signal manipulation tasks, such as filtering, compression, spectral analysis, etc. There are many variations to this basic paradigm. The input nodes can be used to provide the following information to the ANN classifier: - Raw data: in this application, each input node holds a time- or space-ordered sample from the digitized signal, as they came out of the A/D converter. Not all biological signals can be processed this way, however. For example, event-synchronized signals, such as brain evoked potentials (BEP) and periodical, self-synchronized or paced signals, such as electrocardiograms (ECG), are easily time- locked onto the ANN's input vector. On the other hand, non- synchronized signals, such as electroencephalograms (EEG), electromyograms (EMG), etc., are too variable to be used in raw format by the ANN. Sometimes, other considerations preclude the direct use of raw data, such as the presence of baseline drift, amplitude variability, noise, artifacts, undesired features, etc. - Pre-processed signals: in this application, the raw signal sampling data are pre-processed before applying it to the ANN input layer. Thus, it can be synchronized, segmented, transformed, normalized, filtered, detrended, etc., using standard algorithmic, mathematical or neural network-based methods. - Extracted features: in this kind of application, features extracted from the original signal are fed to the ANN input layer, instead of the signal itself. Again, standard algorithmic, mathematical or connectionist methods can be used to extract a host of useful features which characterize some signal properties, such as integrated or derived amplitude, phase, frequency spectrum, latency, duration, counts, crossings, thresholdings, presence/absence, rates, statistical measures, etc. For instance, computing translation- invariant moments is useful when the signal is not time-locked [21]. Learning methods Many ANN architectures and learning paradigms have been used for biological signal processing. In supervised learning, a series of different pairs of input and output patterns, called examplars, are presented to the network in the training phase. At each training cycle, the network computes an error or mismatch function between desired and obtained output, and adjusts the weights of all connections in such a way as to decrease this error. When learning is effective, the error function converges to a minimum after a suitable (finite) number of iterations through the examples. There are ANNs capable of continuous, or on-line learning, i.e., after the introduction of a new, non-classifiable example, the network adapts itself quickly to the new class; but, more usually, learning is performed off-line, i.e., there are separated training and classification phases, and the introduction of a new example forces relearning. A large number of learning algorithms has been developed for ANNs. One of the most used for adaptive biological signal classification is the back-propagation algorithm. The so-called Hopfield network, a single-layer associative memory, has also been used for several biosignal optimization and classification tasks [22] Supervised methods require that previous knowledge about the exemplar classification exists, i.e., training exemplars must have fully known (and correct) inputs and outputs. For some biosignal applications, however, this is an unreasonable demand (for example, classification of occurring patterns in the signal must be carried out a posteriori), so use can be made of non-supervised learning networks. The training examples are presented solely at the input layer, and the network organizes itself progressively, in order to reach maximal separation between the naturally-occurring classes of examples. Thus it operates in a way similar to the classical divisive clustering algorithms found in multivariate analysis methods [23]. Kohonen's LVQ (linear vector quantizer) and SOM (self-organizing maps) networks have been used in biosignal processing, as well as the less common ART (adaptive resonance theory) and CPN (counterpropagation) networks. Testing and Validation As with any development of a new pattern recognition and classification technique, ANNs are not different in the sense that they require extensive testing and validation procedures before they are released for practical use and/or publication. This involves gathering sufficient, complete and accurate case data, performing random (preferrably blind) subdivision of the dataset into training and testing subsets, and then testing the performance of a variety of network configurations against a gold standard. A proper approach requires also the quantification of the network's performance, using sensitivity analysis, ROC (Receiver Operating Characteristics) curve analysis and statistical significance tests [24], as well as the comparison of the ANN with other competing classification methods. AREAS OF APPLICATION Except for a few pioneering applications (eg., [15]), the use of ANNs in biosignal processing is quite a novel development, and it is still confined to a few medical and biological fields. In electrocardiology, ANNs have been used to detect and to analyze the propagation of intracardiac signals [14], ECG signals [10,17,20], exercise [25] and 24-hour ECG analysis [5]. In neurophysiology, ANNs have been applied to the analysis and classification of EMG signals [4,12,18,22], EEG [11], brain evoked potentials [16] and sleep staging [19], among others. Especially interesting is the fact that ANNs may help the study of biological neural networks, by means, for example, of the analysis of complex multi-unit electrical activity [26], thus closing the circle of mutual influence between the Neurosciences and Connectionist Science. Aside the aforementioned biological signals, ANN aplications are still rarely found [8,9], thus giving an indication of its huge potential for new developments. DISCUSSION AND CONCLUSIONS Many investigations have provided evidence that ANN systems are the preferred method to be used when it is required an adaptive solution to ill-defined and complex problems of pattern classification and recognition. The automatic interpretation of biological signals requires the definition of numerous diagnostic criteria and parameters. Rule-based expert system often fail in this domain, either because logical knowledge is hard to define and to acquire, or it's not available. They are too sensitive to noise and are usually too slow for real-time applications. Neural networks are easier to implement and to train using examples, rather than complex and unreliable heuristics. They do not require rigid analytical or causal assumptions, have a higher reliability in presence of noise and uncertainty and are able to provide graded or fuzzy responses. Complex nonlinear as well as time-ordered phenomena are easily treated by ANNs, in contrast to other approaches. In addition, ANNs have the strong incentive that, once true massively- parallel neurocomputers are available, they will provide unprecedented, blinding-fast devices to implement intelligent applications in this field. Among the current hurdles to an increased utilization of ANNs to biosignal processing are: long training times; many factors influence network classification performance and no reliable heuristics is available for choosing the best configuration (thus experimentation is needed); ANN's black-box nature [27]; difficulties in finding gold standards and large and reliable signal databases for training; and lengthy and elaborate procedures for validating ANN performance. REFERENCES [1] Reggia, J.A.; Sutton III, G.G. - Self-processing networks and their biomedical implications. Proc. IEEE, 76(6): 580-92, 1988. [2] Sabbatini, R.M.E. - Applications of connectionist systems in Biomedicine. Proceed. 7th World Congress on Medical Informatics (MEDINFO 92). International Federation of Medical Informatics. North Holland, Amsterdam, p. 418-426, 1992. [3] Simpson, P.K. - Artificial Neural Systems: Foundations, Paradigms, Applications and Implementations. Pergamon Press, London, 1990 [4] Graupe, D.; Liu, R.W.; Moschytz, G.S. - Applications of neural networks to medical signal processing. Proc. 27th IEEE Conf. Decis. Contr. New York: IEEE Press, Vol.1, p. 343-7, 1988. [5] Iwata, A.; Nagasaka, Y.; Suzumura, N. - Data compression of the ECG using neural network for digital Holter monitor. IEEE Engineer. Med. Biol. Magaz., 9(3): 53-57, 1990. [6] Sebald, A.V.; Schlenzig, J. - Use of Slayr neural strategies for closed loop control of drug infusion. Proceed. 12th. Ann. Conf. IEEE Engineer. Med. Biol. Soc., Philadelphia, PA, USA, 1990. [7] Farrell, R.M.; Orr, J.A.; Kuck, K.; Westenskow, D.R. - Differential features for a neural network based anesthesia alarm system. Biomed. Sci. Instrum., 28: 99-104, 1992. [8] Baker, P.D.; Orr, J.A.; Westenskow, D.R. - Nonoinvasive blood pressure monitoring using neural networks. Anesthesiology, 73(3A): A446, 1990. [9] Blumenfeld, B. - A connectionist approach to the recognition of trend in time ordered medical parameters. Comput. Methods Progr. Biomed., 32 (1): 53-62, 1990. [10] Bortolan, G.; Degani, R.; Willems, J.L. - Design of neural networks for classification of electrocardiography signals. Proceed. 12th. Ann. Conf. IEEE Engineer. Med. Biol. Soc., Philadelphia, PA, USA, 1990. [11] Eberhart, R.C.; Dobbins, R.W.; Webber, W.R.S. - CASENET: a neural network tool for EEG waveform classification. In: Proc. II Ann. IEEE Symp. Comput.-Based Med. Syst. Washington, DC: IEEE Comput. Soc. Press, p. 60-8, 1989. [12] Khoshaba, T.; Badie, K.; Hashemi, R.M. - EMG patterns classification based on backpropagation neural network for prosthesis control. Proceed. 12th. Ann. Conf. IEEE Engineer. Med. Biol. Soc., Philadelphia, PA, USA, 1990. [13] Cliquet Jr., A.; Mendeleck, A.; Quesnel, D.R.F.; Sovi, F.X.; Felipe Jr., P.; Oberg, T.D., Lemos, G.J.P.; GuimarĈes, E.A.; Quevedo, A.A.F. - Restoration of upper limb function to tetraplegics using neural networks and a voice processing system. In: Proceed. VI Mediterrean Conf. Med. Biol. Engineer. International Federation for Medical and Biological Engineering, Capri, Italia, 1992. [14] Dassen, W.R.; Muleneers. R.G.; Den Dulk. K.; Smeets, J.R.; Cruz, F.; Penn, O.C.; Wellens, H.J. - An artificial neural network to localize atrioventricular accessory pathways in patients suffering from the Wolff-Parkinson-White syndrome. PACE 13(12 Pt 2):1792-6, 1990. [15] Widrow, B.; Glover Jr., J.R.; McCool, J.M.; Kaunitz, J.; Williams, C.S. et al - Adaptive noise cancelling: principles and applications. Proceed. IEEE, 63(12): 1692-1716, 1975. [16] Freeman, D.T. - Computer recognition of brain stem auditory evoked potential wave V by a neural network. Proceed. 15th Symposium of Computer Applications in Medical Care, AMIA, Washington, DC., p. 305-309, 1991. [17] Suzuki, Y.; Ono, K. - Personal computer system for ECG ST-segment recognition based on neural networks. Med. Biol. Eng. Comput., 30(1):2-8, 1992. [18] Spitzer, A.R.; Hassoun, M.; Wang, C.; Bearden, F. - Signal decomposition and diagnostic classification of the electromyogram using a novel neural network technique. Proceed. 14th. Annual Symposium on Computer Applications in Medical Care. New York: IEEE Press, p. 552-556, 1990. [19] Pfurtscheller, G.; Litscher, G. - Analysis of sleep patterns in babies using neural networks - preliminary results. Medical Informatics Europe MIE'91. Berlin: Springer-Verlag, 1991. [20] Xue, Q.; Hu, Y.H.; Tompkins, W.J. - Neural-network-based adaptive matched filtering for QRS detection. IEEE Trans. Biomed. Eng., 39(4): 317-29, 1992. [21] Lee, S.C. - Using a translation-invariant neural network to diagnose heart arrhythmia. Proc. Ann. Int. Conf. IEEE Med. Biol. Soc., p. 2025-6, 1989. [22] Kelly, M.F.; Parker, P.A.; Scott, R.N. - The application of neural networks to myoelectric signal analysis - A preliminary study. IEEE Trans. Biomed. Engineer., 37(3):221-230, 1990. [23] Lippman, R.P. - An introduction to computing with neural nets. IEEE Trans. Acoust. Speech & Sign. Process., 36: 4-22, 1987. [24] Meistrell, M.L.; Spackman, K.A. - Evaluation of neural network performance by receiver operating characteristic (ROC) analysis: examples from the biotechnology domain. Comput. Methods Programs. Biomed. 32(1):73-80, 1990. [25] Patil, S.; Morise, A.; Detrano, R.; Bobbio, M.; Diamond, G. - Comparison of neural network and logistic regression algorithms for the interpretation of exercise electrocardiography. J. Amer. Coll. Cardiol., 21(2): 98A, 1993. [26] Bower, J.M.; Wilson, M.; Banik, J.; Nelson, M.; Rasnow, B. - Approaches to monitoring and interpreting the dynamics of real neural networks. Proc. Ann. Int. Conf. IEEE Eng. Med. Biol. Soc. New York: IEEE, p.1921-2, 1988. [27] Hart, A.; Wyatt, J. - Evaluating black-boxes as medical decision aids: issues arising from a study of neural networks. Medical Informatics, 15(3): 229-36, 1990. -//-