Eugene Zayden's office hours will
be held regulary at the following times:
Eugene's office is room 108 of the CNS building, 677 Beacon Street. Eugene's e-mail is ezaydens@cns.bu.edu.
Course description
The curriculum of the Department of Cognitive and Neural
Systems (CNS) is highly interdisciplinary, drawing from fields such as
mathematics, neuroscience, psychology, and computer science. The goal of
this course is to provide a background for proper interdisciplinary training by
introducing students to a number of computational and mathematical techniques
that are useful in the research activities of the CNS department. Rather than
simply surveying many topics at an introductory level, the course aims to reach
a sufficiently advanced level for each technique introduced, that the student
should be able to use that technique in his or her research and in other
graduate courses. Several steps have been taken in the design of this course to
maximize the usefulness and depth of the course as well as the breadth of
topics covered. (1) The course makes use of current literature in order to show
from practical examples how a given technique is used in a current field of
research. (2) The course is subdivided into (almost) self-contained modules to
take into account the varied background of participating students. (3) Each
module is designed to facilitate the integration of students with varying
levels of background knowledge for the given topic. Within each module, the
topic covered is introduced with a combination of theoretical and applied
approaches, drawing from concrete examples in the literature. (4) The course
was designed in collaboration with all CNS faculty members to ensure that the
most important topics are covered and that each module contains material
directly relevant to current interdisciplinary research topics.
Covered topics include: (a) linear algebra; (b) ordinary
differential equations; (c) linear system analysis; (d) digital signal
processing; (e) statistics. The course also provides an introduction to the
mathematical software Matlab. Matlab demos will be given in class to show examples of
applications of the covered theories in various research areas (including, for
example, computer vision, control theory, and neuroscience). In addition,
students will be required to use Matlab to
solve the computational sections of the weekly-distributed homework assignments.
The following sections give a more detailed overview of the material covered
in this course. The included links allow downloading of the lecture notes and
some Matlab demos.
Note: Only computers that are part of the BU network are enabled to
download handouts of the lecture notes. Downloading will not work from external
computers.
Introduction
Introductory
Lecture: An overview of
computational methods in cognitive and neural systems.
Module 1 - Linear Algebra A
This series of lectures will provide an introduction to the
solution of systems of linear equations and to the two important factorizations of a matrix: the LU and QR
decompositions. The theory of vector spaces and orthogonality
will be reviewed.
Linear
Equation and LU Factorization: Gaussian elimination and the Gauss-Jordan
elimination algorithms. LU factorization. Computational considerations in solving systems of linear
equations. Pivoting and round-off errors.
Introduction to Matlab and the solution of linear equations.
LU of an
image (this is a Unix tar file).
Vector spaces:
Vector spaces, subspaces and linear combinations. Rank of a matrix and linear
independency. Span, basis and dimension of a subspace.
The four fundamental subspaces associated with a matrix.
Simple Matlab demo that illustrates the importance of choosing a good basis to analyze experimental data.
Orthogonality and QR factorization: Orthogonal vectors
and subspaces. Orthogonality of the fundamental subspaces of a matrix. Least squares. QR factorization.
Matlab Least Squares demo
Module 2 - Fourier analysis
This module provides an introduction to the Fourier analysis
of functions and signals. Covered topics range from formal definitions of
Fourier series and integrals to the use of the Fast Fourier Transform in digital
signal processing.
Fourier Series: Trigonometric expansion of periodic functions.
Convergence of the Fourier Series.
Matlab Fourier Series demo
Fourier
Transform: The Fourier integral. Convergence of the Fourier Transform.
Properties of the Fourier Transform.
Convolution of
two functions
Convolution with
a delta
On the importance of phase and amplitude
Follow this link to download additional notes on the Fourier Transform
Fourier Analysis of discrete-time signals: The Discrete Fourier
Transform. The Fast Fourier Transform.
Follow this link to download additional notes on the Discrete Fourier Transform
Windowing
Module 3 - Linear Algebra B
The concept of a linear transformation will be reviewed together
with the calculation and use of eigenvalues and
eigenvectors. The singular value decomposition of a matrix will also be
reviewed.
Linear
Transformations: Basic concepts on linear transformations and their
relation to matrices.
Eigenvalues and Eigenvectors: Definition of eigenvalues and eigenvectors. Similar
matrices.
Computation
of eigenvalues and eigenvectors
see also Determinant
of a Matrix
The
spectral theorem and SVD: Eigenvalues and
eigenvectors of symmetric matrices. The Singular value decomposition of a mxn matrix.
SVD demo
(this is a Unix tar file)
see also Extension
to complex matrices
Module 4 - Ordinary Differential Equations
This module gives an introduction to the world of
differential equations. After an overview of different types of ordinary
differential equations, we will focus on the case of linear equations. The
lectures will review the theoretical analysis of linear differential equation
as well as the most common computational methods for the solution of ODEs.
First Order
Differential Equations: Different types of equations. Theorems
of existence and uniqueness of solutions. RL and RC
circuits.
Second
Order Differential Equations:
Higher
Order Differential Equations:
Systems
of linear Differential Equations:
See also the ODE Overview and the tar file ODE Matlab demo
Interesting chapters on computational methods for ODEs
can be found on Numerical Recipes in C online. Documentation
on Matlab ODE solvers can be found at MathWorks.
See also the following chapters:
Chapter 1,
Chapter 2,
Chapter 3
Module 5 - Signals and systems
Introduction to the analysis of systems
and signals and to digital signal processing. The focus will be on
linear, time-invariant systems.
Signals
& Systems See also the matlab demos
in SigSys Matlab demos (unix tar file)
FT of a
train of spikes
Reconstruction
of a sampled signal
Aliasing
Module 6 - Probability and Statistics
Introduction to the analysis of
probabilistic data. The module is subdivided into three topics:
Hypothesis testing; Practical issues in statistics; and Signal detection
theory.
Introduction
to probability
Introduction
to statistics
Course requirements and grades
Requirements: All students must complete two
in-class tests, weekly homework assignments, and a final exam.
Note: The course is intended primarily for CNS
graduate students. All students wishing to take this course should discuss it
with their advisor and the CNS faculty member teaching the course prior to the
start of the semester.
In-class tests and final exam: These tests will involve solving a
number of numerical exercises on different topics covered in class. Exercises
may include simple research questions in which the goal is to find the correct
technique (among those analyzed in class) to solve the given problem. All the
exams will last 1.5 hours. In-class tests will be given during regular class
hours. Both in-class tests will concentrate on a limited portion of the
material covered during the course (the first and second one-third of the
material in each of the two tests). The final exam may include material covered
at any time during the course, although an emphasis will be given to the topics
covered during the third part of the course.
First Test: Oct.
17, 2006
Second Test: Nov. 14, 2006
Final Exam: Dec. 15, 2006
Homework assignments: Assignments will be distributed
every week (usually on a Thursday) and will be due for the following week (i.e. the following Thursday). Assignments
will be collected in class. Every homework assignment will include two
sections: (1) Pen and paper exercises: these are standard math exercises
that involve a review of the theory covered in class and the direct application
of the theory to the solution of numerical problems. For example, pen and paper
exercises may include solving a differential equation or performing the
singular value decomposition of a matrix. (2) Computational exercises: these
are brief simulations based on the theory covered in class. Computational
exercises may include the application of a technique to a particular research
problem or the analysis of problems emerging from the use of digital computer
to implement mathematical methods. Examples of computational exercises are the
numerical implementation of an algorithm for solving differential equations and
the evaluation of the power spectrum of a sampled signal. In a number of
assignments, submission of your Matlab code
will be required. In that case, a copy of the developed Matlab
code must be sent via email to both the faculty member teaching the
course and the course teaching fellow. In addition, a printout of your
code must be included with the hardcopy submission.
Note: Homework assignments can be downloaded online
from www.cns.bu.edu/~rucci/Internal/CN500/Homework
Note: All enrolled students have access to facilities adequate
for doing the assignments. In the past many students have successfully used
their own personal computers or machines provided at their place of employment.
If you anticipate difficulties in performing simulations (programming,
graphical plotting, machine access), see the course teaching fellow
immediately.
Grades: Course grades will be based on a conventional 100 point scale,
with A = 93 or better, A-minus = 90-92, etc. All the assignments and exams
contribute equally to the final grade:
Assignments turned in late are eligible for a maximum of
80% credit. The maximum credit allowed for a late assignment will decrease
linearly from 80% to 20% every day for the two weeks following the assignment
due date. Assignments will not be accepted more than 2 calendar weeks
after their due date. Homework assignments will be graded on a 100-point scale,
with 100-88 indicating "excellent" work, 80-70 indicating
"good" work, and less than 70 indicating a "less than B-minus
trajectory", which is inadequate for graduate work.
Note: While participation in class is believed to be crucial
for a thorough understanding of the techniques introduced, no extra credit will
be given for attendance or participation in class discussions. Given the wide
range of topics covered in the course, students that are already familiar with
the material presented in a specific module may skip that module without
penalty in their final grade. Even if not attending to a module, students are
still required to turn in their regular homework assignments.
Supplementary readings and "case-study" papers: The following papers provide examples of application of the methods covered in this course in the field of computational neuroscience::
Linear Algebra:
- Chance FS, Nelson SB, Abbott
LF., Complex cells as cortically amplified simple cells, Nat Neurosci. 1999 Mar;2(3):277-82.
An example of systems of equations in a neural network. (download
pdf)
- GB Stanley, FF Li,
and Y Dan, Reconstruction of Natural Scenes from Ensemble
Responses in the Lateral Geniculate Nucleus
The Journal of Neuroscience, September 15, 1999, 19(18).
An example of calculation of a pseudo-inverse. (download
pdf)
- M.C. Wiener et al. (2001), Consistency
of encoding in the monkey visual cortex, The Journal of Neuroscience, 21
(20), 8210-8221. An example of search for a suitable basis, including an orthonormal basis. (download
pdf)
- De Angelis et al. (1999),Functional Micro-Organization of Primary Visual Cortex: Receptive Field Analysis of Nearby Neurons,
The Journal of Neuroscience, 1999 May 15;19(10):4046-64. An example of the use of orthogonal vectors to model neuronal responses. (download
pdf)
- J. Touryan et al. (2002), Isolation of relevant visual features from random stimuli for cortical complex cells.
The Journal of Neuroscience, 2002, 22(24):10811-10818. An example of applications of eigenvalues and eigenvectors to the analysis of neuronal responses. (download
pdf)
- Field, D. J. (1999) Wavelets,
vision and the statistics of natural scenes. Philosophical Transactions:
Mathematical, Physical and Engineering Sciences 357, no. 1760 : 2527. Searching for convenient representations
of natural stimuli.
- E. Oja
(1982), "A simplified neuron model as a principal component
analyzer", J. Math. Biology, 15, 267-273.
- B. Widrow
and M.E. Hoff (1960). "Adaptive switching networks", IRE WESCON,
96-104.
- H. Bourland
and Y. Kamp (1988). "Auto-association by
multilayer perceptrons and the singular value
decomposition", Biological Cybernetics, 59:291-294.
- Kosko,
B. (1987). Constructing an associative memory. BYTE: The Small Systems
Journal, 12, 137-144. This article gives an interesting and
particularly simple example of the use of matrix algebra in a certain class
of associative networks.
- Kohonen,
T. (1997). Self-organizing maps. Berlin: Springer Verlag.
Ch.1, Sections 1.1 and 1.2 introduce several linear algebra concepts.
- Jordan, M. (1988). An
introduction to linear algebra in parallel distributed processing. In
McClelland, J. and Rumelhart, D. (Eds.) Parallel
Distributed Processing. I: Foundations. Chapter 9, 365-421.
Fourier Analysis:
- Campbell, F. W. & Robson,
J. G. 1968 Application of Fourier analysis to the visibility of gratings.
J. Physiol. 197, 551-556. An example of
application of Fourier theory to visual psychophysics.
- J. Daugman
(1980). Two dimensional spectral analysis of the
cortical receptive field profiles. Vision Research 20 (10), 847-856. An
example of application of Fourier theory to visual neurophysiology.
- J. J. Atick
and A. N. Redlich (1992) What
does the retina know about natural scenes? Neural Computation, 4, 196-210.
- E. D. Lumer,
G. M. Edelman and G. Tononi (1997). Neural
Dynamics in a Model of the Thalamocortical System. I. Layers, Loops and
the Emergence of Fast Synchronous Rhythms, Cerebral Cortex Apr/May 1997;7:207. An example of use of Fourier analysis in a
model.
Differential Equations:
- P. Gaudiano
(1992). "A unified neural model of spatiotemporal processing in X and
Y retinal ganglion cells", 67, 11-21.
- J. Hopfield and D. Tank
(1985). "Neural computation of decision in optimization
problems", Biological Cybernetics, 52, 141-152.
- T. LoFaro,
N. Kopell, E. Marder
and S.L. Hooper (1994) "Subharmonic
coordination in networks of neurons with slow conductances",
6, 69-84.
- J. Rinzel
and B. Ermentrout. Analysis of neural
excitability and oscillations. Chapter 7 of "Methods in Neural
Modeling," C. Koch and I. Segev (Eds.).
Cambridge, MA: MIT Press, 251--291 (1998).
Signal processing:
- E. de Boer and H. R. De Jongh (1978), "On cochlear encoding:
potentialities and limitations of the reverse-correlation technique",
Journal of the Acoustical Society, 63(1), 115-137.
- P. Perona
and J. Malik (1990). "Scale-space and edge
detection using anisotropic diffusion", IEEE PAMI, 12(7): 629-639.
- P. Burt and E.H. Adelson (1983). "The Laplacian
Pyramid as a compact image code", IEEE Transactions on
Communications, COM-31: 532-540.
Control theory:
- H. Gomi
and M. Kawato (1993), "Neural network
control for a closed-loop system using feedback-error-learning",
Neural Networks, 6, 933-946.
- M. Jordan (1996).
Computational aspects of motor control and motor learning. In Handbook
of Perception and Action, 2, 71-120. Read through the end of
Sec.4 only.
- D.K. Anand
(1984). Introduction to control systems. Oxford: Pergamon
Press. Read pp. 1-8, 10-20, 55-66, 95-102, 158-161 and 169-173.
Statistics:
- L. Pessoa,
J. Beck and E. Mingolla (1996). Perceived
texture segregation in chromatic element-arrangement patterns: high
intensity interference. Vision Research 36,
1745-1760. A good example of hypothesis testing.
- Statistical Methods, Chapter
11 in A.C. Bajpai, L.R. Mustoe
and D. Walker Advanced Engineering Mathematics (2nd Ed.), New York: John
Wiley and Sons, 417-467 (1990).
Practical issues in statistics:
- G.R. Loftus and M.E.J.
Masson (1994), "Using confidence intervals in within-subject
designs", Psychonomic Bullettin
and Review, 1(4), 476-490.
- "The Earth is round
(p<.05)" American Psychologist, 49(12), 997-1003 (1994)
- "Psychology will be a
much better science when we change the way we analyze data", Current
Directions in Psychological Science 5(6), 161-171, (1996).
Signal detection theory and probabilistic models:
- W. P. Tanner and J. A. Swets, (1954). "A decision-making theory of
visual detection", Psychological Review, 61, 401-409.
Additional material: Additional readings will be made available in
class in the form of photocopied packets of articles and book chapters.
Online access: In addition, copies of the lecture notes and
other material will be available for downloading from the course web page at http://www.cns.bu.edu/~rucci/CN500.html (this
page if you are accessing the syllabus online) .
Note: you must connect from a computer that is part of the Boston
University network in order to be able to download the material.
This is the updated version of the CN500 syllabus. Please
direct all your questions and comments by electronic mail to Michele Rucci (rucci@cns.bu.edu)
Last modified: September 4, 2006