2nd Session Machine learning: feed-forward neural networks and - - PowerPoint PPT Presentation
2nd Session Machine learning: feed-forward neural networks and self-organizing maps 1 Recommended reading J. Zupan, J. Gasteiger, Neural Networks in Chemistry and Drug Design: An Introduction, Wiley-VCH, Weinheim, 1999.
1
2
and Drug Design: An Introduction, Wiley-VCH, Weinheim, 1999. Chemoinformatics - A Textbook, eds. Johnann Gasteiger and Thomas Engel, Wiley-VCH, 2003. Handbook of Chemoinformatics, ed. Johnann Gasteiger, Wiley-VCH, 2003.
3
Information processing systems inspired on biological nervous systems.
Ability to learn from observations: Extract knowledge Identify relationships Identify structures Generalize
4 Statistical methods process information and ‘learn’. The brain learns with no statistical methods! Neural networks simulate nervous systems using algorithms and mathematical models NNs are interesting from a neuroscience point of view as models of the brain. NNs are interesting for computer science as computational tools.
5
input
A black box ?
6
input
Connected functional units
NEURONS
7
Cell body Dendrites Axon The human nervous system has ca. 1015 neurons. Transmission of an electric signal between dendrites and axons occurs through the transport of ions. Axon terminal
8
Neurons in the superficial layers of the visual cortex in the brain of a mice.
PLoS Biology Vol. 4, No. 2, e29 DOI: 10.1371/journal.pbio.0040029
9
Axon – Dendrite : chemical signal (neurotransmitter). Signal is transmitted in only one direction. Some neurons are able to modify the signal transmission at the synapses.
10
Loss of connections between neurons in the Alzheimer disease
11
Similar neurons in different species. The same type of signal. What is essential is the whole set of neurons, and the connections.
12
The transmitted signal depends on the received signal and the synaptic strength. In artificial neurons, the synaptic strength is called weight. w s p = ws Signal s sent from a previous neuron Synapse with weight w Signal p arriving at the neuron after crossing a synapse
13
14
Each neuron receives signals (si) from many neurons. 0.1
0.2 0.4
0.5 0.2 0.2
Net input = 0.04 = 0.4×0.2 – 0.1×0.1 – – 0.5×0.3 + 0.2×0.2 inputs synapses
15
The net input is modified by a transfer function into an output Out = f (Net)
16
Out = 1 / (1 + e -Net) Important: it is non-linear! Derivative is easy to calculate: d(Out) / d(Net) = Out (1-Out)
17
http://lcn.epfl.ch/tutorial/english/aneuron/html/index.html
18
A neuron recovers approximately one millisecond (10-3 s) after firing. The human brain is able to perform intelligent processes, such as recognizing a friend's face or reacting to some danger, in approximately one tenth of a second. Highly complex tasks have to be performed in less than 100 steps ?! Conclusion: many tasks must be performed simultaneously and in parallel.
19
Input layer Input layer Hidden layer Hidden layer Output layer Output layer Input data Output values
20
21
Input data
22
computation of an error.
the last layer.
the penultimate layer.
23 Introduction of a momentum parameter µ. Correction = computed correction + µ × previous correction
24
Analysis of the problem Which inputs ? How many ? Which output(s) ? How many ? Data pre-processing Normalization (output varies within ]0,1[ !). Splitting into training, test, and prediction sets. Training with the training set and monitoring with the test set (to decide when training shall be stopped). Repetition of training with different parameters (nr of hidden neurons, rate, and momentum) until the best network is found for the test set. Application of the best network found to the prediction set. Evaluation
25
Stop training
26
http://www.dq.fct.unl.pt/staff/jas/jatoon
Training set Test set
Optimum nr of epochs
27
Example: prediction of 1H NMR chemical shifts
O A A B C C D E F G
A B C D E F G
Chemical shift (ppm)
BPG NNs
Training set with exp. values Input: descriptors of H-atoms Output: chemical shift
28
Test with 952 + 259 protons
1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9
Predicted Chemical Shift Experimental Chemical Shift
Aromatics-Set A Pi-Set A Aliphatics-Set A Rigids-Set A Aromatics-Set B Pi-Set B Aliphatics-Set B Rigids-Set B
R2= 0.9830
29
The SPINUS program: www.dq.fct.unl.pt/spinus
30
31
Algebraic view of a data set (values, signals, magnitudes,...) vs. Topological view of a data set (relationships between information)
32
These are two-dimensional arrays of neurons that reflect as well as possible the topology of information, that is, the relationships between individual pieces of data and not their magnitude.
Compression of information Mapping on a 2D surface. “Self-Organized Topological Features Maps” Preserve topology.
33
34
Similar signals in neighbor neurons Do similar signals correspond to the same class?
YES NO
35
One layer of neurons.
36
One layer of neurons.
n weights for each neuron (n = number of inputs)
37
Definition of distance between neurons
Neuron
1st neighborhood 2nd neighborhood
The output of a neuron
neurons
38
Neighborhood
Neuron 1st neighborhood 2nd neighborhood
39
After the input, only one neuron is activated (central neuron or winning neuron) The central neuron is the one with the most similar weights to the input. Traditionaly, similarity = Euclidean distance
2 1
i n i i
n – number of inputs w – value of the weight x – value of the input
40
winning neuron weights
41
The weights of the winning neuron are corrected to make them even more similar to the input. The weights of neighbor neurons are also adapted with the same goal but to a lesser extent. Neuron 1st neighborhood 2nd neighborhood
42
The correction of the neighbor neurons after the activation of a neuron depends on: 1. The distance to the winning neuron (the farther, the smaller the correction) 2. The stage of the training (at the beginning corrections are more drastic) 3. The difference between the weight and the input (the larger the difference, the stronger the correction).
43
The activation of neurons, and the corrections, depend on the Euclidean distance. If the values of a descriptor are in a wider range than another, it will have a larger impact on the result. Therefore, for all descriptors to make a similar impact, NORMALIZATION of data is required.
44
Example of normalization:
descriptor.
(now the descriptor varies between 0 and 1)
(the descriptor will vary between 0.1 and 0.9, useful for BPG networks)
45
Another example of normalization (z normalization):
descriptor.
(the normalized descriptor will have average = 0 and standard deviation = 1)
46
Geographical classification of crude oil samples for the identification of spill sources. From chemical features of oils. Database of chemical features of oils from different geographical origins. Sample (chemical features ) NEURAL NETS Geographical class
classification of crude oils by Kohonen self-organizing maps", Anal. Chim. Acta 2006, 556 (2), 374-382.
47
Content in several compounds determined by GC / MS Examples
3- Methylphenanthrene
H H H H
18α(H)-Oleanane
48
Vector input GC/MS descriptors for a sample of oil
Weights Winning neuron
49 Test set:
Test set:
Training set:
geographical origins
Training set:
geographical origins
50
Input layer Output layer
51
Submission of input input
Correction of the weights at the input layer Correction of the corresponding weights at the
52
Submission of input input
prediction
53
Prediction
Input layer Output layer
Winning neuron
Training
54
Ability of a compound to bind GPCR (G-Protein-Coupled Receptors)
P.Selzer, P. Ertl, QSAR Comb. Sci. 2005, 24, 270-276; J. Chem. Inf. Model. 2006, 46 (6), 2319 -2323.
55
Prediction of the ability to bind GPCR (G-Protein-Coupled Receptors)
P.Selzer, P. Ertl, QSAR Comb. Sci. 2005, 24, 270-276; J. Chem. Inf. Model. 2006, 46 (6), 2319 -2323.
CPG network of size 250×250 Training set: 24870 molecules randomly taken from catalogs (“drug-like”) 1709 known GPCR ligands Input: 225 descriptors (RDF descriptors) Output: 9 levels (GPCR and sub-family “adrenalin, bradykinin, dopamine, endothelin, histamine, opioid, serotonin, vasopressin”). Binary values (0/1) according to ‘YES’ or ‘NO’.
56
P.Selzer, P. Ertl, QSAR Comb. Sci. 2005, 24, 270-276;
Results: 1st output level (GPCR ligand) Weight values are translated into colors.
Regions activated by ligands
57
P.Selzer, P. Ertl, QSAR Comb. Sci. 2005, 24, 270-276; J. Chem. Inf. Model. 2006, 46 (6), 2319 -2323.
Results:
58
P.Selzer, P. Ertl, QSAR Comb. Sci. 2005, 24, 270-276; J. Chem. Inf. Model. 2006, 46 (6), 2319 -2323.
Results: Test set
(25096 non-GPCR and 1490 GPCR)
71% of ligands correctly predicted 18% false positives
59
http://www.dq.fct.unl.pt/staff/jas/jatoon ‘Paste’ data
60
http://www.dq.fct.unl.pt/staff/jas/jatoon Visualization of the distribution of the objects. Neurons colored according to the classes
them.
61
http://www.dq.fct.unl.pt/staff/jas/jatoon Distribution of the
62
http://www.dq.fct.unl.pt/staff/jas/jatoon Inspection of the weights at level 2 of the input layer.