Colin Fyfe

Exploratory data analysis, also known as data mining or knowledge discovery from databases, is typically based on the optimisation of a specific function of a dataset. Such optimisation is often performed with gradient descent or variations thereof. In this book, we first lay the groundwork by reviewing some standard clustering algorithms and projection algorithms before presenting various non-standard criteria for clustering. The family of algorithms developed are shown to perform better than the standard clustering algorithms on a variety of datasets. We then consider extensions of the basic mappings which maintain some topology of the original data space. Finally we show how reinforcement learning can be used as a clustering mechanism before turning to projection methods. We show that several varieties of reinforcement learning may also be used to define optimal projections for example for principal component analysis, exploratory projection pursuit and canonical correlation analysis. The new method of cross entropy adaptation is then introduced and used as a means of optimising projections. Finally an artificial immune system is used to create optimal projections and combinations of these three methods are shown to outperform the individual methods of optimisation.

This book is the outcome of a decade’s research into a speci?c architecture and associated learning mechanism for an arti?cial neural network: the - chitecture involves negative feedback and the learning mechanism is simple Hebbian learning. The research began with my own thesis at the University of Strathclyde, Scotland, under Professor Douglas McGregor which culminated with me being awarded a PhD in 1995 [52], the title of which was “Negative Feedback as an Organising Principle for Arti?cial Neural Networks”. Naturally enough, having established this theme, when I began to sup- vise PhD students of my own, we continued to develop this concept and this book owes much to the research and theses of these students at the Applied Computational Intelligence Research Unit in the University of Paisley. Thus we discuss work from • Dr. Darryl Charles [24] in Chapter 5. • Dr. Stephen McGlinchey [127] in Chapter 7. • Dr. Donald MacDonald [121] in Chapters 6 and 8. • Dr. Emilio Corchado [29] in Chapter 8. We brie?y discuss one simulation from the thesis of Dr. Mark Girolami [58] in Chapter 6 but do not discuss any of the rest of his thesis since it has already appeared in book form [59]. We also must credit Cesar Garcia Osorio, a current PhD student, for the comparative study of the two Exploratory Projection Pursuit networks in Chapter 8. All of Chapters 3 to 8 deal with single stream arti?cial neural networks.

Exploratory data analysis, also known as data mining or knowledge discovery from databases, is typically based on the optimisation of a specific function of a dataset. Such optimisation is often performed with gradient descent or variations thereof. In this book, we first lay the groundwork by reviewing some standard clustering algorithms and projection algorithms before presenting various non-standard criteria for clustering. The family of algorithms developed are shown to perform better than the standard clustering algorithms on a variety of datasets. We then consider extensions of the basic mappings which maintain some topology of the original data space. Finally we show how reinforcement learning can be used as a clustering mechanism before turning to projection methods. We show that several varieties of reinforcement learning may also be used to define optimal projections for example for principal component analysis, exploratory projection pursuit and canonical correlation analysis. The new method of cross entropy adaptation is then introduced and used as a means of optimising projections. Finally an artificial immune system is used to create optimal projections and combinations of these three methods are shown to outperform the individual methods of optimisation.

This book is the outcome of a decade’s research into a speci?c architecture and associated learning mechanism for an arti?cial neural network: the - chitecture involves negative feedback and the learning mechanism is simple Hebbian learning. The research began with my own thesis at the University of Strathclyde, Scotland, under Professor Douglas McGregor which culminated with me being awarded a PhD in 1995 [52], the title of which was “Negative Feedback as an Organising Principle for Arti?cial Neural Networks”. Naturally enough, having established this theme, when I began to sup- vise PhD students of my own, we continued to develop this concept and this book owes much to the research and theses of these students at the Applied Computational Intelligence Research Unit in the University of Paisley. Thus we discuss work from • Dr. Darryl Charles [24] in Chapter 5. • Dr. Stephen McGlinchey [127] in Chapter 7. • Dr. Donald MacDonald [121] in Chapters 6 and 8. • Dr. Emilio Corchado [29] in Chapter 8. We brie?y discuss one simulation from the thesis of Dr. Mark Girolami [58] in Chapter 6 but do not discuss any of the rest of his thesis since it has already appeared in book form [59]. We also must credit Cesar Garcia Osorio, a current PhD student, for the comparative study of the two Exploratory Projection Pursuit networks in Chapter 8. All of Chapters 3 to 8 deal with single stream arti?cial neural networks.