CN 530: Neural and Computational Models of Vision
Course Syllabus, Spring 2004
Professor Ennio Mingolla Teaching Fellow: Arash Yazdanbaksh
Office: 677 Beacon Street, Room 210 Office: 677 Beacon Street, Room 110
Office hours: Tuesdays, 1:00--3:00 PM, and Office Hours: Tuesdays , 10:00 AM to noon, and
by appointment by appointment
Tel: 617-353-9485; email: ennio@cns.bu.edu Tel: 617-353-6426; email: yazdan@cns.bu.edu
Overview: This course explores the psychological, biological, mathematical and computational foundations of visual perception. Lectures and readings combine with simulation and essay assignments to provide an intensive and self-contained examination of core issues in early and middle visual processing. The structure and dynamics of the mammalian visual system are elucidated by mathematically specified neural and computational models. Emphasis is placed on understanding the psychophysics and physiology of mammalian vision, both as a means of better understanding our own human intelligence, and as a foundation for tomorrowís machine vision architectures and algorithms. While the models developed in recent years at Boston Universityís Center for Adaptive Systems (CAS) and Department of Cognitive and Neural Systems (CNS) are covered in depth, selected models by a variety of researchers are compared and contrasted.
Meeting times: Lectures will be given on Wednesdays, beginning on January 14 and ending on April 28, from 1:00-4:00 PM in Room B02 of 677 Beacon Street. An additional hour-long discussion period at a different time will be arranged.
SUMMARY OF WEEKLY TOPICS
Week 1 Jan 14 Fundamental problems of vision
Week 2 Jan 21 Shunting competitive networks and representation in early vision
Week 3 Jan 28 Sensitivity, linear systems approach, and spatial scales
Week 4 Feb 4 Early visual pathways
Week 5 Feb 11 Brightness and lightness perception
Week 6 Feb 18 Perceived brightness and lightness
Week 7 Feb 25 Parallel visual pathways
Week 8 Mar 3 Boundary detection, completion, and sharpening
Mar 10 NO CLASS
Week 9 Mar 17 Approaches to textural segmentation and grouping
Week 10 Mar 24 Binocular vision
Week 11 Mar 31 An in-class examination, covering topics in the readings and lectures
from the first 10 weeks, will be given during this class period.
Week 12 Apr 7 The phenomena of motion perception
Week 13 Apr 14 Models of motion perception
Apr 21 NO CLASS
Week 14 Apr 28 Visual attention, pop out, and search
COURSE REQUIREMENTS AND GRADES: All students must complete two simulation assignments, an in-class midterm examination, and a written final course report. Students will also complete three short ìreading commentaryî essays on issues raised in required readings. Course grades will be based on a conventional 100 point scale, with A = 93 or better, A-minus = 90-92, etc. The weighting of assignments and exams on the final grade is:
20% Two simulation assignments: Each counts 10% of the total for the course.
15% Three reading commentary assignments, each counting for 5% of the course grade:
25% In-class midterm examination (Week 11, Mar 31)
20% Final report (Due on Thursday, May 6, 4:00 pm)
10% Discussion meeting participation
10% Professional growth, as documented in a personal journal
ASSIGNMENTS (OVERVIEW): Written assignments are of two kinds: (1) Simulation assignments involve performing short computer simulations; facility for graphical plotting of black-and-white data curves or line segments will be required. One of the assignments will involve the numerical solution of ordinary differential equations. All enrolled students have access to facilities adequate for doing the assignments. In the past many students have successfully used their own personal computers or machines provided at their place of employment. If you anticipate difficulties in performing simulations (programming, graphical plotting, machine access), see the course teaching fellow immediately. ) (2) Reading commentary assignment consists of a short essays whose purpose is to help the student engage the problems, phenomena and mechanisms described in the required readings. Suggested themes for each essay are described in an assignment packet to be distributed. Credit for participation in discussion meetings will be based on the studentís understanding of core topics from readings and lectures, as expressed in comments initiated by students or in response to questions from the professor.
ASSIGNMENT SUBMISSION, CONTENTS, AND FORMAT: Read the following carefully.
1) Due dates and submission: Assignments are due by 4:00 pm on Mondays, not Wednesdays. Do not even THINK of asking me if it is ìokayî to submit an assignment late; it is not okay, not even by a few minutes, and your assignment grade will suffer. Please assume that printers will not work in the hours just before an assignment is due, that the subway will run late, etc., and get your assignments turned in on time anyway. My preference is to receive hardcopies at my office; assignments may be placed under my door if I am not in. If you are physically distant from CNS on a due date, you may submit assignments electronically in Adobe Acrobat ìpdfî format only.
2) Cover sheet and anonymous grading: Turn in all work on 8 1/2î x 11î paper, and include your name, the course number, the date of submission, and the words ìReading Commentary Assignment Nî or ìSimulation Assignment Nî (N = assignment number) on the upper right corner of the first page only. Written documents are to be evaluated based on their content only, so do not include information about your identity on any other pages.
3) Length: Reports are expected to be brief. ìBriefî means up to 2500 words for total report text; simulation assignments will have additional pages for graphs and diagrams. Software such as MSWord or LaTeX typically generates approximately 250 to 350 words per page, depending on settings for margins and line spacing.
4) Abstracts: All reports, including Reading Commentaries, Simulation Assignments, and the Final Report, must include abstracts of approximately 200-300 words.
5) Graphical plots: Describe simulation results in words, but with the help of graphical plots whenever possible. Include scales on all axes, and explicitly label the quantities being plotted on all axes. Your plots can be generated by computer or by hand, or in combination. (For example, you may be able to plot locations of points by computer, but prefer to label the scale of axes by hand.) Be judicious in choosing which outputs to display overall. Choose figures that contribute materially to the readerís understanding (by showing crucial ìbefore and afterî or ìwith-crucial-parameter-value-equals-this-or-thatî juxtapositions), rather than showing dozens of simulations. A typical mistake made by novice simulators is to display the results of many computer runs that vary only by a single parameter on several pages, making it nearly impossible for the reader to discern the overall impact of parameter variation on simulation results. Often, some manual ìcut and paste,î perhaps with photocopier reduction, is necessary to get a coherent graphical presentation. Note that it is almost never sufficient to report results only by showing plots; some accompanying verbal description -- well keyed to the plots -- is generally necessary. Irrespective of the contents of the data plots, a ìsprawlingî presentation where the relevant variation occurs across pages or with insufficient labeling (of axes and parameter variations) will be awarded reduced credit. Listing of computer code is not desired.
ASSIGNMENT DUE DATES AND GRADING POLICY: Assignments are to be turned in to Prof. Mingollaís office; please slip them under the door by 4:00 PM on the following dates. Note that all are Mondays, except for May 6, which is a Thursday.
Reading Commentary Assignment 1 Feb 2 Week 4
Simulation Assignment 1 Feb 9 Week 5
Reading Commentary Assignment 2 Feb 23 Week 7
Simulation Assignment 2 Mar 2 Week 8
Reading Commentary Assignment 3 Apr 26 Week 14
Final report and personal journal May 6
NOTE: Assignments turned in late are eligible for a maximum of 80% credit. Assignments will not be accepted more than 2 calendar weeks after their due date. Simulation assignments will be graded on a 15-point scale, with 15 indicating ìexcellentî work, 12 indicating ìgoodî work, and less than 12 indicating a ìless than B-minus trajectoryî, which is inadequate for graduate work. Do not underestimate the effect of missing a single assignment deadline on your final course grade. For example, a 20% decrement from the 15 points available from a single simulation assignment accounts for 3% of total available course credit, which is enough to change an ìA-î to a ìB+î. On rare cases, a credit score slightly greater than 15 may be awarded for an assignment containing elaboration beyond that specified in the assignmentís requirements. NOTE: ìElaborationî means that once you have completed all required aspects of the assignment, you may -- time and your own enthusiasm permitting -- do additional simulations, mathematical analysis, or prose argument. It does not mean that you may substitute something that you would rather do for the requirements of the assignments. Reading commentary assignments will be graded on a 0 to 5 scale and the final report on a 0 to 20 scale.
SIMULATION SOFTWARE: While it is assumed that every student in this course is capable of writing simple applications programs for coding assigned simulations, this is not a programming course, and programming will not be taught. The assignments are not meant to burden students with days of software development, and they have been constructed in such a way as to minimize program development time. Many students in past courses have found that MATLAB or comparable packages offer a useful development environment. In any case, sophistication in programming and the use of software tools varies widely among students, and sometimes the best programmer runs into an unanticipated and stubborn quirk. If this happens, feel free to ask the Teaching Fellow (TF) for assistance. You many use any commercially available software that you feel enhances your productivity. You may legitimately ask questions of me or of the course TF while doing the assignments. You may discuss simulation development with fellow students, up to the point of exchanging programming ìtipsî or information about available resources (for graphics, word processing, etc.), but you are not to work in groups for ìdivision of labor.î
WRITING STYLE: Imagine being the reader of your document. It is not enough to ask ìHave I put down what I know?î or ìIs this what I mean to say?î Ask yourself also: ìWould someone who cannot read my thoughts understand what I have written?î and ìAre there equally probable interpretations of what I have written other than what I mean?î Considerations of proper usage (spelling, margins, etc.) pertain both to human decency -- why subject the reader to your sloppiness? -- and to good professional practice. Do not take the attitude that ìIíll turn in something proper only when it really matters,î insofar as bad habits are easy to establish and hard to break. For those who may need additional motivation, I assure you that ìit really mattersî right here and now. The impact on your grade of stylistic lapses of the sorts described in this syllabus can be UP TO 20% of the total.
SIMULATION ASSIGNMENT FORMAT AND STYLE: All assignments must include an abstract, a short introduction, and a short concluding section, each designated by an appropriate header. Your introductions should be sufficiently self-contained to make sense to someone besides me or the course teaching assistant. That is, you should not use jargon or assume familiarity with terms not expected to be in general use in the field of vision research. For simulation assignments you must explicitly label which part of which question (e.g. 3a or 1b) a given section of your report addresses. All assignments must be produced on a word-processor. Clear handwritten annotation is acceptable in small amounts, including correction of typos, insertion of notation in mathematical equations, graphs, or figures. Assignments containing extensive handwritten passages will be returned ungraded. You are expected to employ proper English spelling and grammar, and to adhere to reasonable stylistic conventions (such as the use of margins, references, headings and so forth.) I am aware that for many of you English is not a first language, and I do not expect you to become accomplished authors overnight. I am referring to basic requirements that can and should be met by all: Make sure that your sentences contain verbs and end with a period. If you use a pronoun in a sentence, make sure its antecedent is unambiguous. Do not employ slang. Check your writing for clarity; do not expect to be given ìthe benefit of the doubtî if your words are ambiguous or vague.
NEW GRADING FACTOR FOR 2004: Students are entering CN530 with widely varying degrees of prior training and modeling. Although most aspects of the course will be graded with reference to common standards of achievement, 10% of the final grade will be based on each invidual studentís professional growth, as documented in a personal journal, to be turned in at the same time as the course final report.
PLAGERISM: What you write is to be the original expression of your own learning. If you must employ a phrase or more of words written by another person, clearly mark the passage used as a quotation, and cite the source in full. This requirement applies even if the source is the course lecture notes, or any ìstudy guideî informally circulated among students, whether in paper or electronic form.
ìEXTRA CREDITî WORK: There will be none. The course already contains many pointers for ìextraî work within its assignments. The answer to any request that a student be allowed to bring their grade up to some level (say, B-) through work not already described in course materials -- as opposed to doing a proper job on the regular assignments -- will be ìNO.î
MAKE-UPî WORK: (a) Assignments: Assignments turned in after the due date for whatever reason are eligible for a maximum of 80% credit. For example, a student receiving less than a 12 on a given simulation assignment may resubmit a corrected version of that assignment within 5 weeks of the original due date, in order to bring the grade up to 12 (i.e. 80% of full credit). If a less-than- perfect assignment is submitted on the second round, the grade will be 80% of what that assignment would have earned on the first round. No amount of subsequent extra work on that assignment can make the resulting grade higher than 12. (b) Examinations: Students who are unavoidably absent from the in-class examination will take a special make-up examination consisting of written and oral portions as soon as one can be scheduled.
EMAIL: An alias called cn530@cns.bu.edu has been set up in order to broadcast information of interest to people in this class. Email will never be used as the sole source of important information, but may serve to speed up administrative matters. The accounts of the professor and teaching fellow are included. Any student may send a message to the account, for items likely to be of general interest.
CONFIDENTIALITY OF PERSONAL WORK: All students using University computers for doing simulations, word processing, or figure generation for assignments or take-home examinations are expected to read-protect their files.
LECTURE NOTES AND MISCELLANEOUS READINGS: Copies of all lecture notes for the course are available for download from the course web site. Required readings that are not contained in the textbooks will be made available for photocopying, as will be explained during the first class. Some readings will be available via electronic means.
BU POLICY: The syllabus, course descriptions, and handouts created by Professor Mingolla, and all class lectures, are copyrighted by Boston University and/or Professor X. Except with respect to enrolled students as set forth below, the materials and lectures may not be reproduced in any form or otherwise copied, displayed or distributed, nor should works derived from them be reproduced, copied, displayed or distributed without the written permission of Professor Mingolla. Infringement of the copyright in these materials, including any sale or commercial use of notes, summaries, outlines or other reproductions of lectures, constitutes a violation of the copyright laws and is prohibited. Students enrolled in the course are allowed to share with other enrolled students course materials, notes, and other writings based on the course materials and lectures, but may not do so on a commercial basis or otherwise for payment of any kind. Please note in particular that selling or buying class notes, lecture notes or summaries, or similar materials both violates copyright and interferes with the academic mission of the College, and is therefore prohibited in this class and will be considered a violation of the student code of responsibility that is subject to academic sanctions.
BOOKS MOST RELEVANT TO THE COURSE: One book has been ordered for CN 530:
Palmer, S. E. (1999). Vision science: From photons to phenomenology. Cambridge, MA: MIT Press. (Approx. $80.00). The bookstore lists it as ìrequired,î in the sense that many required readings can be found there. You may choose to use the CNS library edition of this book, or to purchase it. In the weekly listing of required readings, this book is designated as "SEP ." Note that all chapters of this book may be available for downloading with a subscription to Cognet: http://cognet.mit.edu/
You may wish to consider additional purchases, which have not been ordered for CN 530. Some comments are included below to help you make purchasing decisions. Books are interest in the order in which they are most likely to be useful for most students.
Strunk, W., Jr., and White, E.B. The Elements of style. 4th edition. Boston, Allyn & Bacon, 2000. At $6.95 (paperback) this may be the best book, on a price/performance basis, you ever purchase. One cannot overstate the importance of being able to communicate your ideas in forceful and direct English.
Wandell, B. A. (1995). Foundations of vision. Sunderland, Massachusetts: Sinauer Assoc., Inc. The bookstore lists this book as ìoptional.î This book contains an overview of current research issues in visual perception. Several chapters of this book were required reading in past editions of CN 530; in the weekly listing of supplementary readings, this book is designated as ìBAW.î This book is tutorial in organization, with a clear emphasis on ìvision science,î rather than visual perception. (Please ask me if you do not know the difference.) (Approx. $50.00).
Kandel (Editor) E. R., Schwartz, J. H., and Jessell, T. M. (Eds), Principles of Neural Science, 4th Edition. New York: McGraw-Hill. (hardcover $85.00) Do insist on the 4th edition of ìKSJ.î While only five chapters pertain to vision, many other chapters are assigned readings in other CNS courses.
Bruce, V. and Green, P. R. (1997). Visual perception: Physiology, psychology, and ecology. (3rd Ed.) Hillsdale, NJ: Lawrence Erlbaum Assoc. (paperback, $29.95, new; $22.50, used.) Earlier editions of this ìeasyî book were an undergraduate-level text. The "B & G" 3rd edition is a bit more advanced. It is recommended primarily as remedial reading for those with limited backgrounds in visual perception.
Kosslyn, S. M. & Anderson, R. A. (Eds.) (1992). Frontiers in cognitive neuroscience. Cambridge, MA: MIT Press. ($70.00, hardbound) This collection reprints ìclassicî papers in many in areas besides vision. Several chapters are required reading in CN 530; in the weekly listing of required readings, this book is designated as ìK & A.î
Grossberg, S. (Ed.) (1987). The adaptive brain II: Vision, speech, language, and motor control. Amsterdam: North Holland. ($61.50, paperback) This book includes core papers describing the work on vision done at the Center for Adaptive Systems. It also contains many other papers that will be used for other courses in CNS.
Answers to CN 530 FAQs:
Question 1: Why is there so much required reading? (In other words, what do I really have to do to get an A? What parts of this stuff can I skip?)
Answer: I have tried, through points raised in lectures, notes interspersed throughout the syllabus, the creation of a study guide, and the providing of ready access to prior examination questions to be as explicit as I know how to be about what you are expected to know. Further suggestions are welcomed. Thereís simply a lot to know about vision before you can even start to model it.
Question 2: Arenít you really making us read more than is really important? Couldnít you tell us more explicitly which sections, figures, equations, paragraphs, sentences, or phrases really matter?
Answer: Yes, itís true. There remain a few places where I could have been even more explicit than I have been about how you should separate wheat from chaff. By electing to take this course, however, you have embarked on study of an area of research so new, so unformed, and so controversial that -- for many topics -- consensual ìtextbookî knowledge can hardly be said to exist. Soon enough you will have to confront primary source material without any of the aids provided in this course! Part of my job is to train you to meet the challenge of transitioning from ìundergraduate modeî to ìresearcher mode.î I have done this in part by assigning -- in relatively few places -- entire chapters or articles for which I know that whole sections could be skipped without undue harm! Part of your task is to figure out which are those sections, and not to worry about them.
Question 3: Some of the contents of readings contradict parts of other readings. Whatís going on?
Answer: Welcome to the real world of science.
Question 4: Why do I have to bother with all this silly psychology and complicated physiology? How is this going to help me design real world vision applications?
Answer: Letís talk about this during our discussion periods.
Question 5: Why are so many of the course readings from primary sources (original research articles or research book chapters, as opposed to textbook chapters)? The authors use different terminology for the same concepts, and often contradict one another, and they take way more pages to explain things than a textbook does.
Answer: No single textbook appropriate for the entire course exists. The books that exist are either too elementary, too detailed, or too narrow in scope, and they barely cover much of the core material in the course. For certain ideas, there simply is no present substitute for primary sources. (Remember, thatís correlated with our department being involved in emerging, new, exciting, interdisciplinary, lemon-scented research!) Even in the case of psychophysics, and physiology, which the textbooks cover at least moderately well, I have asked you to read some primary sources. I believe that the extra effort required to read them will be rewarded by deeper understanding than can be gotten from textbooks. In any case, I encourage you to go back and forth, between the two types of readings, until you have satisfied yourself that you can master the material outlined in the study guide and presented in class.
Question 6: How will I know when Iím ìgetting it,î given how amorphous and confusing some of the readings are? How do I know which version of several descriptions of, for example, physiological functions of some visual area, is right?
Answer: Where experts disagree, you are entitled to make an informed choice among reasonable alternatives. You are, however, expected to understand the issues underlying the disagreement.
Question 7: How should I study for the in-class midterm?
Answer: During the mid-term, you will not be reading long articles, or reviewing lecture notes, so do not spend all of you preparation time in those activities! During the exam you will primarily be writing. You should practice writing. You should practice writing concise answers to short questions in limited time. I will do my best to make the exam less a measure of your rate of expression and more a matter of assessing your mastery of content. You can help avoid unpleasant surprises by giving yourself one or two ìpracticeî tests based on the study guide, without notes or readings, and in a realistically short amount of time. I would be happy to give you feedback on sample answers that you show me.
Question 8: Much of this course seems rigidly laid out; what if we want to do things differently (e.g. read unassigned articles, or do different simulations than required in assignments.)
Answer: Everything about this course is evolving, and everything is negotiable. Remember, though, that any proposed improvement has a cost (in human effort) that must be budgeted. If your suggestion involves the whole class, remember that the class motto is: ìTo suggest is to volunteer.î
WEEKLY TOPICS AND READINGS
The readings are listed along with a short synopsis of the theme of each weekís lecture. Readings for each week are designated under the headings Required Reading, Supplementary Reading, and BONUS Reading. You will be responsible (in the sense of possibly being tested) for material covered under the ìrequiredî heading only. Material listed as supplementary generally falls into one of two categories. The first includes ìenrichmentî or ìremedialî readings of relatively broad interest; these are generally followed by short parenthetical comments. The second category includes technical or scholarly citations. Those supplementary readings that are indicated by a bullet (ï) are likely to be the most useful to you, ask me about them if you have any trouble locating copies. BONUS readings are listed only because reading them might be fun; these will also be available in the CNS library. (Those contemplating careers as university professors can use these as a diagnostic; if you do not enjoy a significant portion of the bonus readings, you may wish to explore another line of work!) NOTE: Students will be expected to have read all of the required readings listed for a given week by the time that lecture is given, in the sense that the contents of the lecture will assume some familiarity with the readings. That is, the lectures will often comment upon the readings, rather than acting as a substitute for doing the readings.
Week 1: Fundamental problems of vision
1) Unit formation and grouping
2) Seeing and recognizing -- form/color interactions
3) Retinal veins and blind spot
4) Perceiving surface color: Constancy, contrast, and discounting the illuminant
5) Stabilized images: Boundaries and featural color and brightness
6) Complementary processing: Unoriented and oriented detectors
Required Reading:
There are no ìrequiredî readings for Week 1, insofar as you could not be expected to know what to read to prepare for the first lecture. However, three of the readings listed below are special in the sense that reading them is a ìrequirementî for saying that you know anything about current approaches to vision. Those readings are Marr (1982) and K–hler (1947), and Gibson (1966), listed below on this page. In the best of all possible worlds, you would have been exposed to these three authors before starting this course. In any case: (1) Grossbergís early career overlapped the abbreviated career of Marr; the two were intellectual rivals. (2) The intellectual underpinnings of the modeling of grouping and segmentation processes considered in the middle of the course are clarified by the K–hler reading, and (3) Gibson was a legitimate genius, whose views changed the course of 20th century research on vision. In particular, his views on the specification of environmental structure by information in the optic array were adapted by Marr and his colleagues into their tenets on the development of ìcomputational theory.î (Note that Gibson is the only intellectual rival attacked by name in Marrís first chapter.) If pressed for time, consider Marr the first priority. K–hler can wait for several weeks into the course, and Gibson can wait until later.
Supplementary Reading:
ï Marr, D. (1982). Vision. New York, Freeman. Read Chapters 1 and 2, pages 3 - 98. Download pdf (18.4 Mb)
Alternatively, read Marr and Nishiharaís paper in K & A, Ch. 14. If you do the latter, note that Marr and Nishihara speak of four ìlevelsî of analysis where Marr (1982) speaks of three. Marr argues clearly and persuasively for a point of view that has enjoyed considerable popularity in recent years. Much of the CAS/CNS work in vision can be cast in counterpoint -- explicit or implicit -- to Marrís views. If youíve never read any of Marrís writings, consider at least one of the Marr readings to be ìurgentî as well as ìrequired.î
ï K–hler, W. (1947). Gestalt Psychology. New York, New American Library. Chapter IV, ìDynamics as opposed to machine theoryî, 60-79. Download pdf.
Gibson, J. J.(1966) The senses considered as perceptual systems. New York: Houghton Mifflin. Read pages 1-15, 187-223, and 263. This is not ìurgent;î you can read it later in the course.
Week 2: Shunting competitive networks and representation in early vision
1) The noise--saturation dilemma
2) Reflectances and ratios; shunting and mass action
3) Brightness: Constancy and contrast
4) Shift property and Weber law
5) Retinal physiology
6) Hyperpolarization and featural noise suppression
7) Distance--dependent shunting networks
Required Readings: (To be read BEFORE class)
SEP. Read Chapters 1 and 2 for ìbackground.î Also read Ch. 4, Sec. 3.
Grossberg, S. (1982). Why do cells compete? UMAP Unit 484, The UMAP Journal, Vol. III, No. 1. (Education Development Center, 0197-3622/82/010101.) (This is by far the ìeasiestî introduction to shunting inhibition Grossberg has ever written.) Download pdf.
KSJ. Read Ch. 25 and Ch. 26. Also, skim Ch. 21 if you have had no undergraduate introduction to perception or cognition. Ch. 26 contains some details of the pharmacology of receptor phototransduction that will not be ìon the test for CN 530,î as clarified in class.
Supplementary Reading:
Note: While not ìrequired,î the first article listed below will be of particular interest to those of you concerned with computer vision (and is very short!)
ï Boyer, K. L., and Sakar, S. Computer Vision and Image Understanding. Perceptual Organization in Computer Vision: Status, Challenges, and Potential. Vol. 76, No. 1, October, pp. 1ñ5, 1999, Article ID IV990797. Download pdf. Also available online at http://www.idealibrary.com.
Cornsweet, T. (1970). Visual Perception. New York, Academic Press. Chapter XI, ìThe psychophysiology of brightness -- Iî, 268-310. (While this discussion is somewhat dated, it is lucid; also, Cornsweetís views about subtractive inhibition are illustrative of views that many of Grossbergís discussions of shunting inhibition are directed against.)
Borg-Graham LJ, Monier C, Fregnac Y (1998). Visual input evokes transient and strong shunting inhibition in visual cortical neurons, Nature, 393(6683), 369-373.
BONUS Reading:
Westheimer, G. (1986). Systems analysis of spatial vision: An essay in honor of Professor L. H. van der Tweel. Vision Research, 26, (1) 1-5. Compare the views of this meticulous psychophysicist to those of Grossberg (1983), cited above!
Week 3: Contrast sensitivity, linear systems approach, and spatial scales
1) Marr and Grossberg: Symbols, patterns, and the principle of least commitment
2) Recurrent networks
3) Structural scales: functional scales: kernels: receptive fields
4) Peak shifts and lateral inhibition
5) Detectors and filters
6) Contrast sensitivity and spatial scales
Required Reading: (To be read BEFORE class)
Kaufman, L., (1974). Sight and Mind. New York, Oxford University Press. Chapter 5, ìContrast and contourî, 128-152. (This is a good overview of some classic issues in spatial vision.)
Download pdf. (4.8 Mb)
SEP. Ch. 4.
Kiper, D. and Carandini, M. The neural basis of pattern vision. Encyclopedia of cognitive science , 2000, Macmillan Reference, Ltd. Download pdf.
Grossberg, S. (1983). The quantized geometry of visual space: The coherent computation of depth, form, and lightness. Behavioral and Brain Sciences, 6, 625-692. Reprinted as Chapter 1 of Grossberg, S. (Ed.) (1987). The adaptive brain II: Vision, speech, language, and motor control. Amsterdam: North Holland. Read Sections 1-3 and 21-25, 27, and 28, and the commentaries of Grimson and Stevens and Grossbergís reply to those commentaries. Note that the Commentary section appears only in the journal article and is not reprinted in the book. Sections 21-25 restate and extend the discussion of the ìUMAP Moduleî reading, and discuss some issues of relevance to Assignment 1. If you have not had CN 510, sections 27 and 28 may seem obscure -- if so, do not panic; subsequent work in CN 530 will clarify this material.
Download pdf. (23.6 Mb)
Supplementary Reading:
ï Levine, D. and Grossberg, S. (1976). Visual illusions in neural networks: Line neutralization, tilt after-effect, and angle expansion. Journal of Theoretical Biology, 61, 477-504. Levine was Grossbergís first Ph.D. student; this reading is also relevant to Simulation Assignment 1.
Download pdf. (7. Mb)
BAW: Read Chapter 5, but first, skim chapters 1, 2 and 3, noting the discussion of any words printed in boldface that have also appeared in class or in other readings.
Gaudiano P. (1994). A nonlinear model of spatiotemporal retinal processing: simulations of X and Y retinal ganglion cell behavior. Vision Research, 34, 1767--1784.
BONUS READING
Daugman, John. (1990) Brain metaphor and brain theory. Chapter 2 of Eric Schwartz (Ed.) Computational Neuroscience. Cambridge, MA: MIT Press. Reprinted as Chapter 2 in Philosophy and the Neurosciences, edited by W. Bechtel et al. Oxford: Blackwell Publishers. (Scanned .pdf file here). (This essay is a timely and entertaining polemic on the meaning of the word ìcomputational.î)
Week 4: Early visual pathways
1) Anatomical and physiological techniques
2) Retinal structure and function
3) ON and OFF channels
4) Anatomy and physiology of the early visual pathways
Required Reading: (To be read BEFORE class)
KSJ. Read Ch. 27.
Schiller, P. H. On the specificity of neurons and visual areas. Behavioural Brain Research, 1996, 76 (21-35). Download pdf.
Bullier J. (2001). Integrated model of visual processing. Brain Res Brain Res Rev.36(2-3):96-107. Note: This and the next paper may seem ìdense,î particularly if you are new to the physiology of visual perception. Such papers are intended to provoke thought (as for reading commentary assignments) rather than to add to the list of items that you will appear on the mid-term. Compare the study guide, course notes, and readings such as those from KSJ for those purposes.
Gilbert, C. D. Plasticity in visual perception and physiology. Current Opinion in Neurobiology 1996, 6:269-274. Download pdf.
Supplementary Reading:
Bullier, J. and Nowak, L. G. Parallel versus serial processing: new vistas on the distributed organization of the visual system. Current Opinion in Neurobiology, 1995, 5:497-503. Download pdf.
BAW: Read Chapter 7.
ï Callaway EM Local circuits in primary visual cortex of the macaque monkey Annual Review of Neuroscience 21, 47-74 1998. Comprehensive review.
ï Schiller, P. H. (1986). The central visual system. Vision Research, 26, (9), 1351-1386. This is a scholarly and entertaining review of 25 years of work in visual neurophysiology. Read Part A, pages 1351-1362.
Grossberg, S. (1973). Contour enhancement, short term memory, and constancies in reverberating neural networks. Studies in Applied Mathematics LII, 213-257. Reprinted in S. Grossberg, Studies of mind and brain (1982), Boston, Reidel.
Ellias, S. and Grossberg, S. (1975). Pattern formation, contrast control, and oscillations in the short term memory of shunting on-center off-surround networks. Biological Cybernetics, 20, 69-98.
Werblin, F. S. (1971). Adaptation in a vertebrate retina: Intracellular recordings in Necturus. Journal of Neurophysiology, 34, 228-241.
BONUS Reading:
Sacks, O. (1995). The case of the
colorblind painter. Pages 3-41 in O. Sacks, An anthropologist on Mars. New York: Alfred Knopf.
Week 5: Brightness and lightness, Part 1
1) Brightness perception: Quantifying percepts
2) Isomorphistic and nonisomorphistic theories
3) Craik-OíBrien-Cornsweet (COCE) effect
4) Retinex algorithm
5) Brightness assimilation
6) Grossberg and implementation of BCS/FCS
Required Reading: (To be read BEFORE class)
SEP. Skim Ch 3. Read Sec. 3.3 carefully.
Todorovi, D. (1987). The Craik--OíBrien--Cornsweet effect: New varieties and their theoretical implications. Perception & Psychophysics, 42, 545-560. Download pdf.
Concentrate on the distinction between isomorphistic and nonisomorphistic theories.
Check out Retinex-based commercial image processing at: http://dragon.larc.nasa.gov/retinex/
Compare Simon Hongís results by following ìprojectsî link at: http://cns-web.bu.edu/~yhong/
Adelson, E.H. (2000). Lightness Perception and Lightness Illusions, in M. Gazzaniga, M.S., ed., The New Cognitive Neurosciences, 2nd Ed.Cambridge, MA: MIT Press, pp. 339-351. Download pdf.
This is a clearly-written article that explains much of the important terminology used in the study of lightness perception. (Download of this article may be slow!)
Supplementary Reading:
For Retinex Matlab code, go to:http://www.cs.sfu.ca/~colour/publications/IST-2000/
Zaidi Q, Spehar B, Shy M.(1997). Induced effects of backgrounds and foregrounds in three-dimensional configurations: the role of T-junctions. Perception;26(4):395-408. Link. This paper rejects an important hypothesis about lightness perception, read the abstract first, if you are interested in the topic, then read the rest.
Gilchrist A, Kossyfidis C, Bonato F, Agostini T, Cataliotti J, Li X, Spehar B, Annan V, Economou E. An anchoring theory of lightness perception. Psychological Review, 1999 Oct;106(4):795-834.
Cornsweet, T. (1970). Visual Perception. New York, Academic Press. Chapter XII, ìPsychophysiology of brightness -- II, Modulation transfer functions,î 311-364.
Graham, N. (1980). Spatial frequency channels in human vision: Detecting edges without edge detectors. In C. S. Harris, Ed., Visual coding and adaptability. Hillsdale, NJ, Earlbaum, 215-262. The experiment described in Figure 6 (page 226) and the accompanying text is of fundamental importance to understanding issues related to ìspatial frequency channelsî in human vision.
BONUS Reading:
Barlow, H. B. (1972) Single units and sensation: A neuron doctrine for perceptual psychology? Perception 1:371-394. Reprinted as Chapter 14 of J. A. Anderson, A. Pellionisz, and E. Rosenfeld (Eds.) Neurocomputing 2, Directions for Research. Cambridge, MA, MIT Press, 1988. This is one of the foundational papers from the recent ìsingle unitî era in physiological psychology.
Week 6: Brightness and lightness, Part 2
1) Grossberg & simulations
2) Integration models
3) Challenges to brightness models
4) Symbolic models and energy models
5) Extensions of BCS/FCS-style brightness models
6) Diffusion and time
Required Reading: (To be read BEFORE class)
Marr, D. (1982). Vision. New York, W.H. Freeman. Pages 250-258. Download pdf.
Marr gives a lucid overview of Landís Retinex theory; read this before Land (1986).
Land, E. H. (1986). Recent advances in Retinex theory. Vision Research, 26(1), 7-21. Download pdf.
You will be expected to understand this approach, an example of an integration theory, in excruciating detail.
Grossberg, S. and Todorovi, D. (1988). Neural dynamics of 1-D and 2-D brightness perception: A unified model of classical and recent phenomena. Perception & Psychophysics, 43, 241-277. Reprinted in Grossberg, S. (Ed.) (1988). Neural Networks and Natural Intelligence. Cambridge, MA: MIT Press. Download pdf. (Concentrate on the role of boundaries and diffusion in the explanation of percepts.)
Supplementary Reading:
Pessoa, L., Thompson, E., & Noe, A. Finding out about filling-in: a guide to perceptual completion for visual science and the philosophy of perception. Behavioral and Brain Sciences, 1998 Dec;21(6):723-48.
ï Neumann H. (1996). Mechanisms of neural architecture for visual contrast and brightness perception. Neural Networks, 9(6), 921-936. NOTE: You may wish to consult this paper while doing Simulation Assignment 2. (Downloadable at: http://www.sciencedirect.com .)
Davey, M. P., Maddess, T., and Srinivasan M. V. (1998). The spatiotemporal properties of the Craik-OíBrien-Cornsweet effect are consistent with ìfilling-inî. Vision Research, 38(13), 2037-2046. The title speaks for itself.
BAW: Read Chapter 6.
Paradiso, M. A. and Nakayama, K. (1991). Brightness perception and filling-in. Vision Research, 31, 1221-1236.
Hung CP, Ramsden BM, Chen LM, Roe AW., Building surfaces from borders in Areas 17 and 18 of the cat., Vision Res. 2001;41(10-11):1389-407. Take a look for some electrophysiological evidence regarding the effect of boundary contrast on surface lightness.
Gerrits, H. J. M., and Vendrick, A. J. H. (1970) Simultaneous contrast, filling-in process and information processing in manís visual system. Experimental Brain Research, 11, 411-430.
BONUS Reading:
Westheimer, G. (1983). Herman Helmholtz and the origins of sensory physiology. Trends in Neurosciences, Jan., 5-9. (Did you know that Helmholtz is considered by many to be the greatest sensory psychologist who ever lived?)
Week 7: Parallel visual pathways and perceptual organization
1) Why edge detection?
2) How edge detection?
3) Front end of BCS
4) Neon color spreading
5) How thin is ìthinî
6) Spatial and orientational competition
7) Hyperacuity
Required Reading: (To be read BEFORE class)
SEP. Ch 6.
KSJ. Read Ch. 28.
Neumann, H. and Mingolla, E. (2003) Contour and surface perception. In M.A. Arbib, Ed., Handbook of brain theory and neural networks, II. Cambridge, MA: MIT Press. pp. 271-276. Download pdf.
Grossberg, S. and Mingolla, E. (1985). Neural dynamics of form perception: Boundary completion, illusory figures, and neon color spreading. Psychological Review, 92(2), 173-211. Reprinted as Chapter 2 of Grossberg, S. (Ed.) (1987). The adaptive brain II: Vision, speech, language, and motor control. Amsterdam: North Holland. Download pdf. Concentrate on the arguments for making the Boundary/Feature distinction in the first place; on the conditions that produce neon color spreading, and on the mechanisms of the theoryís explanation of neon color spreading. Skim lightly over the sections on cooperative completion of boundaries. In other words, concentrate on Sections 1-15. The Appendix of this article is subsumed by that of a subsequent article.
Supplementary Reading:
Spillmann, L., Werner, J.S. Long-range interactions in visual perception. TRENDS NEUROSCI 19: (10) 428-434 OCT 1996. Download pdf.
Fitzpatrick,D. Seeing beyond the receptive field in primary visual cortex. CURR OPIN NEUROBIOL 10: (4) 438-443 AUG 2000. Download pdf.
Bressan, P., Mingolla, E., Spillmann, L. and Watanabe, T. (1997). Neon color spreading: A review. Perception, 26(11), 1353-1366.
Takeichi H, Shimojo S, Watanabe T. (1992). Neon flank and illusory contour: interaction between the two processes leads to color filling-in. Perception 21(3):313-24. This shows how psychophysics can help to localize the areas responsible for boundary and surface formation.
Daugman, J. (1985). Uncertainty relation of resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. Journal of the Optical Society of America A, 2, 1160-1169. (This paper includes some key observations on uncertainty in filtering.)
BONUS Reading:
Bateson, G. (1979). Every schoolboy knows... Chapter 2 of Mind and Nature. Toronto, Bantam. Bateson was weird; do you know the definition of ìsacramentî as given in the Baltimore Catechism of the Roman Catholic Church? If not, how will you ever understand ìemergentî behavior of neural networks? For that matter, how will you ever know what remarks in a course syllabus to take seriously?
Week 8: Boundary detection, completion, and sharpening
1) Cooperative-Competitive (CC) Loop
2) Bipole cells, then and now
3) Spatial impenetrability
4) Boundary webs
5) von der Heydt, Peterhans, & Baumgartner, 1984
6) Free association
7) Autonomy of perception; cognitive impenetrability
Required Reading: (To be read BEFORE class)
Grossberg, S. and Mingolla, E. (1985). Neural dynamics of perceptual grouping:
Textures, boundaries, and emergent segmentations. Perception & Psychophysics, 38<