Combinatorial Calculus in Computer Vision: Formulating and Solving Continuous PDEs on Graphs

Note: Course materials were later expanded and collected into the book, "Discrete Calculus: Applied Analysis on Graphs for Computational Science" by Leo J. Grady and Jonathan R. Polimeni, Springer 2010.

Date: Sunday October 12, 2008
Time: Morning
Presenters: Leo Grady and Sébastien Bougleux
In association with ECCV 2008
Slides are also available



Description:
The widespread variational methods of computer vision follow the pattern of establishing an energy for which the (local or global) minimum solution represents the desired segmentation, registration, filtering, etc.  However, this energy is typically formulated in continuous space (with a continuous gradient), while the solving/testing operations are performed on a discrete computer.  Ideally, we could reformulate our energy in terms of combinatorial operators that may be efficiently solved using combinatorial optimization techniques.  The primary barrier to such a reformulation has not been the lack of established mathematics for translating between continuous and combinatorial operators, but rather a lack of familiarity in the computer vision community with the appropriate combinatorial operators corresponding to traditional continuous operators.  This tutorial will introduce these corresponding combinatorial operators to the computer vision community and show how combinatorial optimization techniques may be applied to solve PDEs formulated in the continuum.  The tutorial will consist of two divisions, presented by the two lecturers.


Outline:
Division 1 - Calculus on graphs.  This division will be given by Leo Grady and will introduce the combinatorial operators necessary to apply calculus to functions defined on graphs.  It will be shown through worked examples that these operators retain many (if not all) of the properties of the familiar continuous operators, such as Stokes' Theorem, the Helmholtz decomposition, the Laplacian, etc.  Additionally, with each of these familiar operators the combinatorial formulation usually offers new interpretations, often in terms of trees.  These techniques for performing calculus on graphs are then given application to computer vision by demonstrating how to reformulate select PDE methods in computer vision on a graph such that combinatorial optimization may be used to generate a solution.  More recent computer vision techniques that demonstrate the application of these calculus operators on graphs will also be presented, such as segmentation via random walkers and combinatorial minimal surfaces.


Outline of Division 1:
D1-1) Introduction
    A) Background history of development of continuous and combinatorial calculus
D1-2) Combinatorial and Continuous Space
    A) Calculus in the continuum revisited
        i)  Domain and boundary
        ii)  Integrals
        iii) Gradient, divergence and curl
        iv)  Exterior derivative
        v)  p-forms and p-vectors
    B) Combinatorial calculus
        i)  Domain and boundary
        ii)  Functions defined on a domain
        iii) Integrals
        iv)  Coboundary operator - Combinatorial gradient, divergence and curl
    C) Examples of combinatorial calculus
        i)  Stokes Theorem
        ii)  Helmholtz decomposition
        iii) Methods of solution (Euler-Lagrange equations)
        iv)  The Laplacian, shift-invariant domains and Fourier decomposition
D1-3) Applications
    A) Step-by-step example of how to reformulate and solve an existing PDE method on a graph (Mumford-Shah)
        i)  Formulation
        ii)  Optimization
        iii) Comparison with traditional methods of continuous optimization
    B) The Laplace equation, random walks and image segmentation
        i)  Seeded segmentation as a PDE with internal Dirichlet boundary conditions
        ii)  Laplace equation is equivalent to steady-state properties of random walks
        iii) Benefits of the technique for image segmentation
D1-4) Conclusion


Outline of Division 2:
Discrete regularization on weighted graphs. This division will be given by Sébastien Bougleux and will introduce recent discreteregularization frameworks to solve inverse problems on graphs.  Variational formulations, discrete operators and algorithms will be presented and compared to their continuous analogues.  The main advantages of the graph-based approach are its simplicity and its usability to process any discrete data or function defined on these data.  In particular, applications in image and mesh processing will be given (denoising, simplification, inpainting, ...).

D2-1) Introduction
    A) Motivations of a discrete framework
    B) Weighted graphs and functions on graphs
D2-2) Discrete operators on graphs and properties
    A) Second order operators
        i) Laplacian
        ii) Curvature
        iii) p-Laplacians
D2-3) Inverse problems and discrete regularization on graphs
    A) General variational formulation based on regularization
        i) Minimizers and regularization functionals: From convex to non-convex minimization
        ii) Link to local and non-local continuous regularizations
    B) Algorithms for smoothing, denoising and simplification
        i) Regularization equations based on p-Laplacians
        ii) Semi-discrete and discrete diffusions
        iii) Application to images, meshes and databases
    C) Solving general inverse problems in image processing
        i) Adapted energy for optimal graph recovering
        ii) Proximal resolution
        iii) Application to inpainting, super-resolution and compressive sampling
D2-3) Conclusion