# Variational PDE Models in Image Processing

←

**Page content transcription**

If your browser does not render page correctly, please read the page content below

Variational PDE Models in Image Processing Tony F. Chan, Jianhong (Jackie) Shen, and Luminita Vese I mage processing, traditionally an engineering as fingerprints and face identification). All these field, has attracted the attention of many math- highly diversified disciplines have made it neces- ematicians during the past two decades. From sary to develop common mathematical founda- the point of view of vision and cognitive sci- tions and frameworks for image analysis and ence, image processing is a basic tool used to processing. Mathematics at all levels must be in- reconstruct the relative order, geometry, topology, troduced to address the crucial criteria demanded patterns, and dynamics of the three-dimensional by this new era—genericity, well-posedness, accu- (3-D) world from two-dimensional (2-D) images. racy, and computational efficiency, just to name a Therefore, it cannot be merely a historical coinci- few. In return, image processing has created dence that mathematics must meet image process- tremendous opportunities for mathematical mod- ing in this era of digital technology. eling, analysis, and computation. The role of mathematics is determined also by This article gives a broad picture of mathemat- the broad range of applications of image process- ical image processing through one of the most re- ing in contemporary science and technology. These cent and very successful approaches—the varia- applications include astronomy and aerospace ex- tional PDE (partial differential equation) method. ploration, medical imaging, molecular imaging, We first discuss two crucial ingredients for image computer graphics, human and machine vision, processing: image modeling or representation, and telecommunication, autopiloting, surveillance processor modeling. We then focus on the varia- video, and biometric security identification (such tional PDE method. The backbone of the article consists of two major problems in image process- This paper is based on a plenary presentation given by ing that we personally have worked on: inpainting Tony F. Chan at the 2002 Joint Mathematics Meetings in and segmentation. By no means, however, do we San Diego. The work was supported in part by NSF grants intend to give a comprehensive review of the en- DMS-9973341 (Chan), DMS-0202565 (Shen), and ITR- tire field of image processing. Many of the authors’ 0113439 (Vese); by ONR grant N00014-02-1-0015 (Chan); and by NIH grant NIH-P20MH65166 (Chan and Vese). articles and preprints related to the subject of this paper can be found online at our group home- Tony F. Chan is professor of mathematics at the Univer- page [11], where an extended bibliography is also sity of California, Los Angeles. His email address is chan@math.ucla.edu. available. Jianhong (Jackie) Shen is assistant professor at the School Image Processing as an Input-Output System of Mathematics, University of Minnesota. His email address Directly connected to image processing are two is jhshen@math.umn.edu. dual fields in contemporary computer science: Luminita Vese is assistant professor of mathematics at the computer vision and computer graphics. Vision University of California, Los Angeles. Her email address (whether machine or human) tries to reconstruct is lvese@math.ucla.edu. the 3-D world from observed 2-D images, while 14 NOTICES OF THE AMS VOLUME 50, NUMBER 1

T Q0 Q graphics pursues the opposite direction by de- signing suitable 2-D scene images to simulate our denoising+ deblurring u0 = Ku + n clean & sharp u 3-D world. Image processing is the crucial middle inpainting u0 |Ω\D entire image u|Ω way connecting the two. Abstractly, image processing can be considered segmentation u0 “objects” as an input-output system [uk , Ωk ], k = 1, 2, . . . scale-space u0 multiscale images Q0 → Image Processor T → Q uλ1 , uλ2 , ... (1) (2) Here T denotes a typical image processor: for ex- motion estimation (u0 , u0 , ...) optical flows ample, denoising, deblurring, segmentation, com- v (1) , v ( (2) , ...) pression, or inpainting. The input data Q0 can rep- resent an observed or measured single image or image sequence, and the output Q = (q1 , q2 , · · · ) Table 1. Typical image processors and their inputs and contains all the targeted image features. outputs. The symbols represent (1) K: a blurring kernel, and n: For example, the human visual system can be an additive noise, both assumed in this paper to be linear for considered as a highly involved multilevel image simplicity; (2) u0 : the given noisy or blurred image; (3) Ω : the processor T . The input Q0 represents the image se- entire image domain, and D : a subset where image quence that is constantly projected onto the retina. information is missing or inaccessible; (4) [uk , Ωk ] : Ωk ’s are the The output vector Q contains all the major features segmented individual “objects”, while uk ’s are their intensity that are important to our daily life, from the low- values; (5) λk ’s are different scales, and uλ can be roughly level ones such as relative orders, shapes, and understood as the projection of the input image at scale λ; (6) (n) grouping rules to high-level feature parameters u0 ’s denote the discrete sampling of a continuous “movie” that help classify or identify various patterns and u0 (x, t) (with some small time step h) and v (n)’s are the objects. estimated optical flows (i.e., velocity fields) at each moment. Table 1 lists some typical image processing problems. Wavelet Representation. An image is often ac- The two main ingredients of image processing quired from the responses of a collection of are the input Q0 and the processor T . As a result, microsensors (or photo receptors), either digital or the two key issues that have been driving main- biological. During the past two decades, it has been stream mathematical research on image process- gradually realized (and experimentally supported) ing are (a) the modeling and representation of the that such local responses can be well approximated input visual data Q0 , and (b) the modeling of the by wavelets. This new representation tool has rev- processing operators T . Although the two are in- olutionized our notion of images and their multi- dependent, they are closely connected to each other scale structures [12]. The new JPEG2000 protocol by the universal rule in mathematics: the structure for image coding and the successful compression and performance of an operator T is greatly in- of the FBI database of fingerprints are its two most fluenced by how the input functions are modeled influential applications. The theory is still being ac- or represented. tively pushed forward by a new generation of geo- Image Modeling and Representation metric wavelets such as curvelets (Candés and To efficiently handle and process images, we need Donoho) and beamlets (Pennec and Mallat). first to understand what images really are mathe- Regularity Spaces. In the linear filtering theory of matically and how to represent them. For example, conventional digital image processing, an image u is it adequate to treat them as general L2 functions is considered to be in the Sobolev space H 1 (Ω) . The or as a subset of L2 with suitable regularity con- Sobolev model works well for homogeneous regions, straints? Here we briefly outline three major classes but it is insufficient as a global image model, since of image modeling and representation. it “smears” the most important visual cue, namely, Random fields modeling. An observed image u0 edges. Two well-known models have been intro- is modeled as the sampling of a random field. For duced to recognize the existence of edges. One is example, the Ising spin model in statistical me- the “object-edge” model of Mumford and Shah [13], chanics can be used to model binary images. More and the other is the BV image model of Rudin, Osher, generally, images are modeled by some Gibbs/Mar- and Fatemi [15]. The object-edge model assumes that kovian random fields [10]. The statistical proper- an ideal image u consists of disjoint homogeneous ties of fields are often established through a fil- object patches [uk , Ωk ] with uk ∈ H 1 (Ωk ) and regu- tering technique and learning theory. Random field lar boundaries ∂Ωk (characterized by one-dimen- modeling is the ideal approach for describing nat- sional Hausdorff measure). The BV image model as- ural images with rich texture patterns such as trees sumes that an ideal image has bounded total and mountains. variation Ω |Du|. Regularity-based image models JANUARY 2003 NOTICES OF THE AMS 15

are generally applicable to images with low texture Variational PDE Method patterns and without rapidly oscillatory compo- Having briefly introduced the general picture of nents. mathematical image processing, we now focus on Modeling of Image Processors the variational PDE method through two processors: How images are modeled and represented very inpainting and segmentation. much determines the way we model image proces- For the history and a detailed description of current developments of the variational and PDE sors. We shall illustrate this viewpoint through the method in image and vision analysis, see two spe- example of denoising u = T u0 : u0 = u + n , as- cial issues in IEEE Trans. Image Processing [7 (3), suming for simplicity that the white noise n is ad- 1998] and J. Visual Comm. Image Rep. [13 (1/2), ditive and homogeneous, and there is no blurring 2002] and also two recent monographs [1], [18]. involved. In the variational or “energy”-based models, When images are represented by wavelets, the nonlinear PDEs emerge as one derives formal Euler- denoising processor T is in some sense “diago- Lagrange equations or tries to locate local or global nalized” and is equivalent to a simple engineering minima by the gradient descent method. Some on the individual wavelet components. This is a cel- PDEs can be studied by the viscosity solution ebrated result of Donoho and Johnstone on approach [8], while many others still remain open threshold-based denoising schemes. to further theoretical investigation. Under the statistical/random field modeling of Compared with other approaches, the varia- images, the denoising processor T becomes MAP tional PDE method has remarkable advantages in (Maximum A Posteriori) estimation. By Bayes’s for- both theory and computation. First, it allows one mula, the posterior probability given an observa- to directly handle and process visually important tion u0 is geometric features such as gradients, tangents, p(u|u0 ) = p(u0 |u)p(u)/p(u0 ). curvatures, and level sets. It can also effectively sim- ulate several visually meaningful dynamic The denoising processor T is achieved by solving processes, such as linear and nonlinear diffusions the MAP problem max p(u|u0 ) . Therefore, it is im- and the information transport mechanism. Sec- u portant to know not only the random field image ond, in terms of computation, it can profoundly model p(u) but also the mechanism by which u0 is benefit from the existing wealth of literature on generated from the ideal image u (the so-called numerical analysis and computational PDEs. For generative data model). The two are crucial for example, various well-designed shock-capturing successfully carrying out Bayesian denoising. schemes in Computational Fluid Dynamics (CFD) can be conveniently adapted to edge computation Finally, if the ideal image u is modeled as an el- in images. ement in a regular function space such as H 1 (Ω) or BV(Ω) , then the denoising processor T can be Variational Image Inpainting and realized by a variational optimization. For instance, Interpolation in the BV image model, T is achieved by The word inpainting is an artistic synonym for image interpolation; initially it circulated among 1 min |Du| subject to (u − u0 )2 dx ≤ σ 2 , museum restoration artists who manually restore u Ω |Ω| Ω cracked ancient paintings. The concept of digital where the white noise is assumed to be well ap- inpainting was recently introduced into digital image processing in a paper by Bertalmio, Sapiro, proximated by the standard Gaussian N(0, σ 2 ) . Caselles, and Ballester. Currently, digital inpaint- This well-known denoising model, first proposed ing techniques are finding broad applications in by Rudin, Osher, and Fatemi, belongs to the more image processing, vision analysis, and digital tech- general class of regularized data-fitting models. nologies such as image restoration, disocclusion, Just as different coordinate systems that de- perceptual image coding, zooming and image super- scribe a single physical object are related, different resolution, error concealment in wireless image formulations of the same image processor are transmission, and so on [2], [4], [9]. Figure 1 shows closely interconnected. Again, take denoising for an example of error concealment. example. It has been shown that the wavelet tech- We now discuss the mathematical ideas and nique is equivalent to an approximate optimal reg- methodologies behind variational inpainting tech- ularization in certain Besov spaces (Cohen, Dahmen, niques. Throughout this section, u denotes the Daubechies, and DeVore). On the other hand, original complete image on a 2-D domain Ω , and Bayesian processing and the regularity-based u0 denotes the observed or measured portion of variational approach can also be connected (at u, which can be either noisy or blurry, on a sub- least formally) by Gibbs’s formula in statistical domain or general subset D . The goal of inpaint- mechanics (see (3) in the next section). ing is to recover u on the entire image domain Ω 16 NOTICES OF THE AMS VOLUME 50, NUMBER 1

as faithfully as possible from the available data u0 on D . From Shannon’s Theorem to Variational Inpainting Interpolation is a classical topic in approximation theory, numerical analysis, and signal and image processing. Successful interpolants include poly- nomials, harmonic waves, radially symmetric func- tions, finite elements, splines, and wavelets. Despite the diversity of the literature, there exists one most widely recognized result due to Shannon, known as Shannon’s Sampling Theorem. Theorem (Shannon’s Theorem). If a signal u(t) is Figure 1. TV inpainting for the error concealment of a blurry band-limited within (−ω, ω), then image. ∞ π ω to specify the conditional probability p(u0 |u) . u(t) = u n sinc t −n . n=−∞ ω π Finally, in the Bayesian framework, Shannon’s “then” statement is replaced, as indicated earlier, That is, if an analog signal u(t) (with finite en- by the Maximum A Posteriori (MAP) optimization ergy or, equivalently, in L2 (R ) ) does not contain any given by Bayes’s formula: high frequencies, then it can be perfectly interpo- lated from its properly sampled discrete sequence (1) max p(u|u0 ) = p(u0 |u)p(u)/p(u0 ). u u0 [n] = u(nπ /ω) (where ω/π is known as the Nyquist frequency). (It is equivalent to maximizing the product of the All interpolation problems share this “if-then” prior model and the data model, since the de- structure. “If” specifies the space where the target nominator is a fixed normalization constant once signal u is sought, while “then” gives the recon- u0 is given.) To summarize, Bayesian inpainting struction or interpolation procedure based on the means finding the most probable image given its discrete samples (or, more generally, any partial incomplete and possibly distorted observation. information about the signal). The variational approach resembles the Bayesian Unfortunately, for most real applications in sig- methodology, but now everything is expressed nal and image processing, one cannot expect a deterministically. The Bayesian prior model p(u) closed-form formula as clean as Shannon’s. This is becomes the specification of the regularity of due to at least two factors. First, in vision analysis an image u , while the data model p(u0 |u) now and communication, signals like images are in- measures how well the observation u0 fits if the trinsically not band-limited because of the presence original image is indeed u. Regularity is enforced of edges (or Heaviside-type singularities). Second, through “energy” functionals: for example, the for most real applications, the given incomplete Sobolev norm E[u] = Ω |∇u|2 dx , the total variation data are often noisy and become blurred during the (TV) model E[u] = Ω |Du| of Rudin, Osher, and imaging and transmission processes. Therefore, Fatemi, and the Mumford-Shah free-boundary in the situation of Shannon’s Theorem, we are deal- model E[u, Γ ] = Ω\Γ |∇u|2 dx + βH 1 (Γ ) , where H 1 ing with a class of “bad” signals u with “unreliable” denotes the one-dimensional Hausdorff measure. samples u0 . The quality of data fitting u → u0 is often judged Naturally, for image inpainting, both the “if” and “then” statements in Shannon’s Theorem need by an error measure E[u0 |u] . For instance, the least to be modeled carefully. It turns out that there are square measure prevails in the literature due to the two powerful and interdependent frameworks that genericity of Gaussian-type noise and the Central 1 can carry out this task: one is the variational Limit Theorem: E[u0 |u] = |D| D (T u − u0 )2 dx, method, and the other is the Bayesian frame- where D is the domain on which u0 has been work [10]. sampled or measured, |D| is its area (or cardinal- In the Bayesian approach the “if” statement ity for the discrete case), and T denotes any linear specifies both the so-called prior model and the or nonlinear image processor (such as blurring and data model. The prior model specifies how images diffusion). In this variational setting, Shannon’s are distributed a priori or, equivalently, which “then” statement becomes a constrained opti- images occur more frequently than others. Proba- mization problem: bilistically, it specifies the prior probability p(u) . min E[u] over all u such that E[u0 |u] ≤ σ 2 . Let u0 denote the incomplete data that are ob- served, measured, or sampled. Then the second part Here σ 2 denotes the variance of the white noise, of “if” is to model how u0 is generated from u or which is assumed to be known by proper statistical JANUARY 2003 NOTICES OF THE AMS 17

Figure 2. Mumford-Shah inpainting for text removal. estimators. Equivalently, the model solves the fol- The data model is explicitly given by lowing unconstrained problem using Lagrange mul- 1 tipliers (e.g., Chambolle and Lions): (4) E[u0 |u, D] = (Ku − u0 )2 dx. |D| D (2) min E[u] + λE[u0 |u]. u Therefore, from the variational point of view, the Generally, λ expresses the balance between regu- quality of an inpainting model crucially depends larity and fitting. In summary, variational inpaint- on the prior model or the regularity energy E[u] . ing searches for the most “regular” image that best The TV prior model E[u] = Ω |Du| was first in- fits the given observation. troduced into image processing by Rudin, Osher, The Bayesian approach is more universal in the Fatemi in [15]. Unlike the Sobolev image model sense of allowing general statistical prior and data E2 [u] = Ω |∇u|2 dx , the TV model recognizes one models, and it is powerful for restoring both arti- of the most important vision features, the “edges”. ficial images and natural images (or textures). But For example, for a cartoon image u showing the to learn the prior model and the data model is night sky (u = 0 ) with a full bright moon (u = 1 ), usually quite expensive. The variational approach the Sobolev energy blows up, while the TV energy is ideal for dealing with regularity and geometry Ω |Du| equals the perimeter of the moon, which and tends to work best for man-made indoor and is finite. Therefore, in combination with the data outdoor scenes and images with low textures. The model (4), the variational TV inpainting model two approaches (1) and (2) can be at least formally minimizes unified under Gibbs’s formula in statistical me- chanics: (5) Etv [u|u0 , D] = α |Du| + λ (Ku − u0 )2 dx. Ω D (3) E[·] ∝ −β log p(·), or p(·) ∝ e−E[·]/β , The admissible space is BV(Ω) , the Banach space where β = kT is the product of the Boltzmann con- of all functions with bounded variation. It is very stant and temperature, and ∝ means equality up similar to the celebrated TV restoration model of to a multiplicative or additive constant. (However, Rudin, Osher, and Fatemi [15]. In fact, the beauty the definability of a rigorous probability measure and power of the model exactly lie in the provision over “all” images is highly nontrivial because of the of a unified framework for denoising, deblurring, multiscale nature of images. Recent efforts can be and image reconstruction from incomplete data. found in the work of Mumford and Gidas.) Figure 1 displays the computational output of the Variational Inpainting Based on Geometric Image model applied to a blurry image with simulated ran- Models dom packet loss due to the transmission failure of In a typical image inpainting problem, u0 denotes a network. the observed or measured incomplete portion of The second well-known prior model is the ob- a clean “good” image u on the entire image domain ject-edge model of Mumford and Shah [13]. The Ω . A simplified but already very powerful data edge set Γ is now explicitly singled out, unlike in model in various digital applications is blurring fol- the TV model, and an image u is understood as a lowed by noise degradation and spatial restriction: combination of both the geometric feature Γ and u0 = (Ku + n)D , the piecewise smooth “objects” ui on all the con- D nected components Ωi of Ω \ Γ . Thus in both the where K is a continuous blurring kernel, often Bayesian and the variational languages, the prior assumed to be linear or even shift-invariant, and n model consists of two parts (applying (3) for the is an additive white noise field assumed to be close transition between probability and “energy”): to Gaussian for simplicity. The information u0 |Ω\D is missing or inaccessible. The goal of inpainting is p(u, Γ ) = p(u|Γ )p(Γ ) and to reconstruct u as faithfully as possible from u0 D . E[u, Γ ] = E[u|Γ ] + E[Γ ]. 18 NOTICES OF THE AMS VOLUME 50, NUMBER 1

In the Mumford-Shah model the edge regularity is specified by E[Γ ] = H 1 (Γ ) , the one-dimensional Hausdorff measure, or in most computational ap- plications, E[Γ ] = length(Γ ) , assuming that Γ is Lip- schitz. The smoothness of the “objects” is naturally characterized by the ordinary Sobolev norm: E[u|Γ ] = Ω\Γ |∇u|2 dx . Therefore, in combination with the data model (4), the variational inpainting model based on the Mumford-Shah prior is given by (6) inf Ems [u, Γ |u0 , D] = α |∇u|2 dx u,Γ Ω\Γ + βH 1 (Γ ) + λ (Ku − u0 )2 dx. Figure 3. Smooth inpainting by the Mumford- D Shah-Euler model. Figure 2 shows one application of this model for and Shen [2], and as expected it improves the TV text removal. Notice that edges are preserved and inpainting model. smooth regions remain smooth. Similarly, the Mumford-Shah image model Ems Numerous applications have demonstrated that, can be improved by replacing the length energy by for classical applications in denoising, deblurring, Euler’s elastica energy: and segmentation, both the TV and the Mumford- Shah models perform sufficiently well even by the Emse [u, Γ ] = α |∇u|2 dx + e[Γ ]. high standard of human vision. But inpainting does Ω\Γ have special characteristics. We have demonstrated in [2], [4], [9] that for large-scale inpainting prob- This was first applied to image inpainting by Ese- lems, high-order image models which incorporate doglu and Shen [9]. Figure 3 shows one example of the curvature information become necessary for applying this image prior model to the inpainting more faithful visual effects. of an occluded disk. Both the TV and Mumford-Shah The key to high-order geometric image models inpainting models would complete the interpola- is Euler’s elastica curve model: tion with a straight-line edge and introduce visible corners as a result. The elastica model restores the smooth boundary. e[γ] = (a + bκ 2 ) ds, a, b > 0, γ The improved performance of curvature-based models comes at a price in terms of both theory where κ denotes the scalar curvature. Birkhoff and and computation. The existence and uniqueness of de Boor called it the “nonlinear spline” model in the TV and Mumford-Shah inpainting models can approximation theory. It was first introduced into be studied in a fashion similar to the classical computer vision by Mumford. Unlike straight restoration and segmentation problems. But theo- lines (for which b = 0 ), the elastica model allows retical study on high-order models is only begin- smooth curves because of the curvature term, ning. The difficulty lies in the involvement of the which is important for computer vision and com- second-order geometric feature of curvature and puter graphics. in the identification of a proper function space to By imposing the elastica energy on each indi- study the models. Secondly, in terms of computa- vidual level line of u (at least symbolically or by tion, the calculus of variation on the curvature assuming that u is regular enough), we obtain the term leads to fourth-order highly nonlinear PDEs, so-called elastica image model: whose fast and efficient numerical solution im- ∞ poses a tremendous challenge. Eel [u] = e[u ≡ λ] dλ We conclude this section with a brief discussion −∞ ∞ of computation, especially for the TV and Mumford- (7) = (a + bκ 2 ) ds dλ Shah inpaintings. −∞ u≡λ For the TV inpainting model Etv , the Euler- = (a + bκ 2 )|∇u| dx. Lagrange equation is formally (or assuming that u Ω is in the Sobolev space W 1,1 ) given by In the last integrand the curvature is given by ∇u κ = ∇ · [∇u/|∇u|] . (Notice that in the absence of (8) −∇ · + µK ∗ χD (Ku − u0 ) = 0. |∇u| the curvature term, the above formula is exactly the co-area formula for smooth functions (e.g., Giusti). Here K ∗ denotes the adjoint of the linear blurring This elastica prior model was first studied for in- kernel K, the multiplier χD (x) is the indicator of D , painting by Masnou and Morel, and by Chan, Kang, and µ = 2λ/α . The boundary condition along ∂Ω JANUARY 2003 NOTICES OF THE AMS 19

is Neumann adiabatic to eliminate any boundary Caselles, and Ballester, and by Bertalmio, Bertozzi, contribution during the integration-by-parts process. and Sapiro. In particular, it has been interestingly This nonlinear PDE can be solved iteratively by found in the latter paper that the earlier PDE model the freezing technique: if u(n) denotes the current by Ballester, Bertalmio, Caselles, Sapiro, and Verdera inpainting at step n, then the updated inpainting is closely related to the stream function–vorticity u(n+1) solves the linearized PDE equation in fluid dynamics. ∇u(n+1) Variational Level Set Image Segmentation −∇ · + µK ∗ χD (Ku(n+1) − u0 ) = 0. |∇u(n) | Images are the proper 2-D projections of the 3-D world containing various objects. To successfully In practice the intermediate diffusivity coeffi- reconstruct the 3-D world, at least approximately, cient 1/|∇u(n) | is often modified to the first crucial step is to identify the regions in im- 1/ |∇u(n) |2 + *2 for some small conditioning pa- ages that correspond to individual objects. This is rameter * or by the mandatory ceiling and floor- the well-known problem of image segmentation. It ing between * and 1/* . The convergence of such has broad applications in a variety of important algorithms has been well studied in the literature fields such as computer vision and medical image (e.g., Chambolle and Lions, and Dobson and Vogel). processing. There are also many other possible techniques in Denote by u0 an observed image on a 2-D Lipschitz the literature for solving (8) (e.g., Vogel and Oman, open and bounded domain Ω . Segmentation means and Chan, Mulet, and Golub). We need only to re- finding a visually meaningful edge set Γ that leads late (8) to the conventional TV restoration case. to a complete partition of Ω . Each connected com- The computation of the Mumford-Shah inpaint- ponent Ωi of Ω \ Γ should correspond to at most one ing model is also very interesting. For inpainting, real physical object or pattern in our 3-D world, unlike segmentation, one’s direct interest is only in for example, the white matter in brain images or the u, not in Γ. Such understanding makes the Γ-conver- abnormal tissues in organs. In some applications, gence approximation theory perfect for inpainting. one is interested also in the clean image patches According to Ambrosio and Tortorelli, by introduc- ui on each Ωi of the segmentation, since u0 is often ing an edge signature function z(x) ∈ [0, 1] , x ∈ Ω, noisy. and having E[u|Γ ] = α Ω\Γ |∇u|2 dx replaced by Therefore, there are two crucial ingredients in E[u|z] = α Ω z 2 |∇u|2 dx , one can approximate the the mathematical modeling and computation of length energy in the Mumford-Shah model by a qua- the segmentation problem. The first is how to dratic integral in z (up to a constant multiplier): formulate a model that appropriately combines the effects of both the edge set Γ and its segmented *|∇z|2 (z − 1)2 regions {Ωi , i = 1, 2, · · · } . The other is to find E* [z] = β + dx, * 1. Ω 2 2* the most efficient way to represent the geometry of both the edge set and the regions and to repre- Thus the Mumford-Shah inpainting model is ap- sent the segmentation model as a result. This of proximated by course reflects the general philosophy in the introduction. E* [u, z|u0 , D] = E[u|z] + E* [z] + λE[u0 |u, D], In the variational PDE approach, these two issues have found good answers in the literature: for which is a quadratic integral in both u and z ! It leads the first, the celebrated segmentation model of Mum- to a coupled system of linear elliptic-type PDEs in ford and Shah [13] and for the second, the level-set both u and the edge signature z , which can be representation technology of Osher and Sethian [14]. solved efficiently using any numerical elliptic solver. In what follows we detail our recent efforts in ad- The example in Figure 2 was computed by this vancing the application of the level-set technology to scheme. various Mumford-Shah-related image segmentation Finally, we mention some of the major applica- models. Much of the work can be found in our pa- tions of the inpainting and geometric image inter- pers (e.g., [3], [5], [17], [19] and many more on our polation models developed above. These include group homepage [11]) and also in related works by digital zooming, primal-sketch-based perceptual Yezzi, Tsai, and Willsky; Paragios and Deriche; Zhu image coding, error concealment for wireless image and Yuille; and Cohen, Bardinet, and Ayache [6], [7]. transmission, and progressive disocclusion in com- We start with a novel active-contour model whose puter vision [2], [4], [9]. Extensions to color or more formulation is independent of intensity edges de- general hyperspectral images and nonflat image fined by the gradients, in contrast to most con- features (i.e., ones that live on Riemannian mani- ventional ones in the literature. We then explain how folds) are also currently being studied in the liter- this model can be efficiently computed based on the ature. Other approaches to the inpainting problem multiphase level-set method. In the second part we can be found in the papers by Bertalmio, Sapiro, extend these results to the level-set formulation and 20 NOTICES OF THE AMS VOLUME 50, NUMBER 1

computation of the general Mumford-Shah segmentation model for piecewise-smooth images. In the last part we present our re- cent work on extending the previous mod- els to logical operations on multichannel image objects. Active Contours without Edges and Multiphase Level Sets The active contour is a powerful tool in image and vision analysis for boundary de- tection and object segmentation. The key idea is to evolve a curve so that it eventu- ally stops along the object edges of the given image u0 . The curve evolution is controlled Figure 4. Top: Detection of a simulated minefield by our new active- by two sorts of energies: the internal energy contour model. Bottom: Segmentation of an MRI brain image. Notice defining the regularity of the curve and the that the interior boundaries are automatically detected. external energy determined by the given image u0 . The latter is often called the feature-driven energy. In almost all classical active-contour models, the feature-driven energies rely heavily on the gradient feature |∇u0 | or on its smoothed version |∇Gσ ∗ u0 | , where Gσ denotes a Gaussian kernel with a small variance σ . They work well for detecting gradient-defined edges but fail for more general classes of edges such as the bound- ary of a nebula in some astronomical images or the top image in Figure 4. Our new model, active contours without edges, first introduced in [5], is independent Figure 5. Left: Two curves given by φ1 = 0 and φ2 = 0 partition the of the gradient information and therefore can domain into four regions based on indicator vector (sign(φ1 ),sign(φ2 )) . handle more general types of edges. The Right: Three curves given by φ1 = 0 , φ2 = 0 , and φ3 = 0 partition the model is to minimize the energy domain into eight regions based on the triple (sign(φ1 ),sign(φ2 ), sign(φ3 )) . E2 [c1 , c2 , Γ |u0 ] = |u0 (x) − c1 |2 dx int(Γ ) E2 [c1 , c2 , φ|u0 ] = |u0 (x) − c1 |2 H(φ) dx Ω (9) + |u0 (x) − c2 |2 dx + ν|Γ |, ext(Γ ) + |u0 (x)−c2 |2 (1−H(φ)) dx+ν |∇H(φ)| dx. Ω Ω where ν denotes a given positive weight, the c ’s are unknown constants, int(Γ ) and ext(Γ ) denote the Minimizing E2 [c1 , c2 , φ|u0 ] with respect to c1 , c2 , interior and exterior of Γ, and |Γ | is its length. The and φ leads to the Euler-Lagrange equation: subscript 2 in E2 indicates that it deals with two- phase images, i.e., ones whose “objects” can be ∂φ ∇φ completely indexed by the interior and exterior = δ(φ) νdiv ∂t |∇φ| of Γ. In the level-set formulation of Osher and − |u0 − c1 |2 + |u0 − c2 |2 , Sethian [14], Γ is embedded as the zero-level set u0 (x)H(φ(x)) dx {φ = 0} of a Lipschitz continuous function c1 (t) = Ω , φ : Ω → R . Consequently, {φ > 0} and {φ < 0} de- Ω H(φ(x)) dx fine the interior Ω+ and exterior Ω− of the curve. u0 (x)(1 − H(φ(x))) dx c2 (t) = Ω , (The level-set approach is computationally superior Ω (1 − H(φ(x))) dx to other curve representations, because it lets one directly work on a fixed rectangular grid and it with a suitable initial guess φ(0, x) = φ0 (x) . In nu- allows automatic topological changes such as merg- merical implementations the Heaviside function ing and breaking.) Denote by H the 1-dimensional H(z) is often regularized by some Hε (z) in C 1 (R) Heaviside function: H(z) = 1 if z ≥ 0 and 0 if z < 0 . that converges as ε → 0 to H(z) in some suitable Then the energy in our model becomes sense. As a result, the Dirac function δ(z) in the last JANUARY 2003 NOTICES OF THE AMS 21

active-contour model is insufficient, and we need to introduce multiple level-set functions. Therefore, we have generalized the above framework to multi- phase active contours or, equivalently, the piece- wise-constant Mumford-Shah segmentation with multiphase regions: inf Ems [u, Γ |u0 ] u,Γ (10) = |u0 − ci |2 dx + ν|Γ |. i Ωi Here the Ωi ’s denote the connected components of Ω \ Γ , and u = ci on Ωi . Notice that Γ can now be a general set of edge curves, including for example the T-junction class. Generally, consider m level-set functions φi : Ω → R . The union of the zero-level sets of the φi represents the edges in the segmented image. Using these m level-set functions, one can define up to n = 2m phases, which form a disjoint and complete partitioning of Ω . Therefore, each point x ∈ Ω belongs to one and only one phase. In par- ticular, there is no vacuum or overlap among the phases. This is an important advantage compared with the classical multiphase representation, where a level-set function is associated to each phase and more level-set functions are needed as a result. Figure 6. The original and segmented images Figure 5 shows two typical examples of multiphase (top row), and the final four segments (the rest). partitioning corresponding to m = 2 and m = 3 . equation is regularized to δε (z) = Hε (z) . We have We now illustrate the multiphase level-set ap- discovered in [5] that a carefully designed approx- proach through the example of n = 4 and m = 2 . imation scheme can even allow interior contours to Let c = (c11 , c10 , c01 , c00 ) denote a constant vector emerge, a challenging task for most conventional and Φ = (φ1 , φ2 ) the two-phase level-set vector. algorithms. Also notice that the length term in the Then we are looking for an ideal image u in the form energy has led to the mean-curvature motion. of The model performs as an active contour in the class of piecewise-constant images taking only two u = c11 H(φ1 )H(φ2 ) + c10 H(φ1 )(1−H(φ2 )) values; it looks for a two-phase segmentation of a + c01 (1−H(φ1 ))H(φ2 ) + c00 (1−H(φ1 ))(1−H(φ2 )). given image. The internal energy is defined by the length, while the external energy is independent of The Mumford-Shah segmentation energy be- the gradient |∇u0 | . Defining the segmented image comes by u(x) = c1 H(φ(x)) + c2 (1 − H(φ(x))) , we realize that the energy model is exactly the Mumford-Shah E4 [c, Φ|u0 ] = |u0 (x) − c11 |2 H(φ1 )H(φ2 ) dx Ω segmentation model [13] restricted to the class of piecewise-constant images. However, our model + |u0 (x) − c10 |2 H(φ1 )(1 − H(φ2 )) dx Ω was initially developed from the active-contour point of view. (11) + |u0 (x) − c01 |2 (1 − H(φ1 ))H(φ2 ) dx Two typical numerical outputs of the model are Ω displayed in Figure 4. The top row shows that our + |u0 (x) − c00 |2 (1 − H(φ1 ))(1 − H(φ2 )) dx model can segment and detect objects without Ω clear gradient edges. The bottom one shows that +ν |∇H(φ1 )| dx + ν |∇H(φ2 )| dx. it can also capture complicated boundaries and Ω Ω interior contours. For more complicated situations where multiple Its minimization leads to the Euler-Lagrange equa- objects occlude each other and multiphase edges tions. First, with Φ fixed, the c minimizer can be such as T-junctions emerge, the above two-phase explicitly worked out as before: 22 NOTICES OF THE AMS VOLUME 50, NUMBER 1

cij (t) = average of u0 on {(2i − 1)φ1 > 0, (2j − 1)φ2 > 0}, i, j = 0, 1. In turn, this new c information leads to the Euler- Lagrange equations for Φ : ∂φ1 ∇φ 1 = δ(φ1 ) νdiv ∂t |∇φ1 | − (u0 − c11 )2 − (u0 − c01 )2 H(φ2 ) − (u0 − c10 )2 − (u0 − c00 )2 (1 − H(φ2 )) , Figure 7. An example of texture segmentation (at increasing times). ∂φ2 ∇φ = δ(φ2 ) νdiv 2 now show how to carry out the model based on the ∂t |∇φ2 | multiphase level-set approach [5]. As before, we − (u0 − c11 )2 − (u0 − c01 )2 H(φ1 ) start with the two-phase situation where a single level-set function φ is sufficient, followed by the − (u0 − c10 )2 − (u0 − c00 )2 (1 − H(φ1 )) . more general multiphase case. Notice that the equations are governed both by the In the two-phase situation, the ideal image u is mean curvatures and by jumps of the data-energy segmented to u± by the level-set function φ: terms across the boundary. Figure 6 shows an application of the model to u(x) = u+ (x)H(φ(x)) + u− (x)(1 − H(φ(x))). the medical analysis of a brain image. Displayed are the final segmented image and its associated four We assume that both u+ and u− are C 1 functions phases. Our model successfully identifies and seg- up to the boundary {φ = 0} . Substituting this ex- ments the white and the gray matters. Recently the above models and algorithms have pression into (12), we obtain been extended to multichannel, volumetric, and E[u+ , u− , φ|u0 ] = |u+ − u0 |2 H(φ) dx texture images (e.g., Chan, Sandberg, and Vese [3]). Ω Let us give a little more detail about texture segmentation from our work. Texture images are + |u− u0 |2 (1 − H(φ)) dx Ω general images of natural scenes, such as grass- (13) lands, beaches, rocks, mountains, and human body tissues. They typically carry certain coherent struc- +µ |∇u+ |2 H(φ) dx Ω tures in scales, orientations, and local frequencies. To segment texture images using the above mod- +µ |∇u− |2 (1 − H(φ)) dx + ν |∇H(φ)|. Ω Ω els, we first apply Gabor’s filters to extract these coherent structures. The filter responses create a First, with φ fixed, the variation on new vectorial (or multichannel) feature image in the form of U(x) = (uα (x), uβ (x), · · · , uγ (x)) , where E[u+ , u− , φ|u0 ] leads to the two Euler-Lagrange the Greek letters stand for the filter signatures, and equations for u± separately: typically each takes a value of (scale, orientation, local frequency). We then apply the vectorial active- u± − u0 = µu± on ± φ > 0, contour-without-edges model to the segmentation (14) ± ∂u of U. Figure 7 shows one typical example. =0 on {φ = 0}. ∂n Piecewise-Smooth Mumford-Shah Segmentation The most general Mumford-Shah piecewise-smooth (Here ± takes either of the values + and − , but uni- segmentation [13] is defined by formly across the formula.) They act as denoising operators on the homogeneous regions only. No- (12) inf Ems [u, Γ |u0 ] = |u − u0 |2 dx u,Γ Ω tice that no smoothing is done across the bound- ary {φ = 0} , which is very important in image +µ |∇u|2 dx + ν|Γ |, Ω\Γ analysis. Next, keeping the functions u+ and u− fixed and where µ and ν are positive parameters. It allows the segmented “objects” to have smoothly varying minimizing E[u+ , u− , φ|u0 ] with respect to φ, we intensities instead of being strictly constant. We obtain the motion of the zero-level set: JANUARY 2003 NOTICES OF THE AMS 23

As in the previous section, there are cases where the boundaries forming a complete partition of the image cannot be represented by a single level-set function. Then one has to turn to the multi-phase approach. In our papers, thanks to the planar Four- Color Theorem, we have been able to conclude that two level-set functions are sufficient for all multi- phase partition problems. By the Four-Color Theorem one can color all the regions in a partition using only four colors, so that any two adjacent regions are color distinguishable. Identifying a phase with one color, we see that two level-set functions φ1 and φ2 are sufficient to pro- duce four “colors”: {±φ1 > 0 , ±φ2 > 0} . There- Figure 8. Numerical result from the piecewise-smooth fore, they can completely segment a general image Mumford-Shah level-set algorithm with one with a multiphase boundary set Γ given by {φ1 = 0} level-set function. or {φ2 = 0}. As before, we do not have the prob- lems of “overlapping” or “vacuum” as in the works A A 1 1 A2 A2 by Zhao, Chan, Merriman, and Osher. Note that in this formulation, generally each “color” can still have many isolated components. Therefore, the segmentation is complete only after one applies an extra step of the well-known topological processor for finding the connected components of an open set. In this four-phase formulation, the ideal image Figure 9. A synthetic example of an object in u is segmented into four disjoint but complete two different channels. Notice that the lower parts u±± , each defined by one of the four phases: left corner of A1 and the upper corner of A2 are missing. {±φ1 > 0, ±φ2 > 0}. Λ1∪Λ2 Λ1∩Λ2 Λ1∪¬Λ2 Overall, by using the Heaviside function, we obtain the following synthesis formula: u =u++ H(φ1 )H(φ2 ) + u+− H(φ1 )(1 − H(φ2 )) + u−+ (1 − H(φ1 ))H(φ2 ) + u−− (1 − H(φ1 ))(1 − H(φ2 )), for all x ∈ Ω. We can express the energy function Figure 10. Different logical combinations for of u and Φ = (φ1 , φ2 ) in a similar way and derive the sample image: the union, the intersection, the corresponding Euler-Lagrange equations. and the differentiation. Notice the remarkable feature of this single model, which includes both the original energy ∂φ ∇φ = δ(φ) ν∇ − (|u+ − u0 |2 formulation and the elliptic and evolutionary PDEs: ∂t |∇φ| it naturally combines all three image processors— active contour, segmentation, and denoising. + µ|∇u+ |2 − |u− u0 |2 − µ|∇u− |2 ) , Logic Operators for Multichannel Image Segmentation with some initial guess φ(t = 0, x) . The above equa- In a multichannel image u(x) = (u1 (x), u2 (x), tion is actually computed at least near a narrow · · · , un (x)) , a single physical object can leave band of the zero-level set. As a result, computa- different traces in different channels. For example, tionally we have to continuously extend both u+ and Figure 9 shows a two-channel image containing a triangle that is, however, incomplete in each u− from their original domains {±φ > 0} to a suit- individual channel. For this example, most con- able neighborhood of the zero-level set {φ = 0} . ventional segmentation models for multichannel Figure 8 displays an application of the model in as- images (e.g., Guichard, Sapiro, Zhu and Yuille) tronomical image analysis. Although the nebula would output the complete triangle, i.e., the union itself does not seem to be a smooth object, the of both channels. The union is just one of the piecewise-smooth model can still correctly cap- several possible logical operations for multichan- ture the main features. nel images. For example, the intersection and the 24 NOTICES OF THE AMS VOLUME 50, NUMBER 1

Truth table for the two-channel case z1in z2in z1out z2out A1 ∪ A2 A1 ∩ A2 A1 ∩ ¬A2 differentiation are also very common in applica- x inside Γ 1 1 0 0 1 1 1 tions, as illustrated in Figure 10. (or x ∈ Ω+ ) 1 0 0 0 0 1 1 In this section we outline our recent efforts in 0 1 0 0 0 1 0 developing logical segmentation schemes for multi- 0 0 0 0 0 0 1 channel images based on the active-contour- x outside Γ 0 0 1 1 1 1 0 (or x ∈ Ω− ) without-edges model [16]. 0 0 1 0 1 0 1 0 0 0 1 1 0 0 First, we define two logical variables to encode 0 0 0 0 0 0 0 the information inside and outside the contour Γ separately for each channel i: Table 2. The truth table for two channels. Notice that inside Γ 1, if x is inside Γ and not on “true” is represented by 0 . It is designed so as to encourage ziin (u0i , x, Γ ) = the object, the contour to enclose the targeted logical object at a lower 0, otherwise; energy cost. is because in practice, as mentioned above, the z ’s 1 if x is outside Γ and on are approximated and take continuous values. For zi (u0 , x, Γ ) = out i the object, 0 otherwise. example, possible interpolants for the union and intersection are Such different treatments are motivated by the en- fA1 ∪A2 (x) = z1in (x)z2in (x) ergy minimization formulation. Intuitively speak- ing, in order for the active contour Γ to evolve and + 1 − (1 − z1out (x))(1 − z2out (x)) , eventually capture the exact boundary of the tar- geted logical object, the energy should be designed fA1 ∩A2 (x) =1 − (1 − z1in (x))(1 − z2in (x)) so that both partial capture and overcapture lead to high energies (corresponding to ziout = 1 and + z1out (x)z2out (x). ziin = 1 separately). Imagine that the target object is tumor tissue: then in terms of decision theory, The square roots are taken to keep the functions over and partial captures correspond respectively of the same order as the original scalar models. It to false alarms and misses. Both are to be penal- is straightforward to extend the two-channel case ized. to more general n-channel ones. The energy functional E for the logical objective In practice we do not have precise information function f can be expressed by the level set func- of “the object” to be segmented. One possible way tion φ. Generally, as just shown above, the objec- to approximate ziin and ziout is based on the inte- tive function can be separated into two parts, rior (Ω+ ) and exterior (Ω− ) averages ci± in channel i: f = f (z1in , z1out , · · · , znin , znout ) |u0i (x) − ci+ |2 ziin (u0i , x, Γ ) = , = fin (z1in , · · · , znin ) + fout (z1out , · · · , znout ). maxy∈Ω+ |u0i (y) − ci+ |2 for x ∈ Ω+, and The energy functional is then defined by |u0i (x) − ci− |2 E[φ|c + , c − ] = µlength(φ = 0) ziout (u0i , x, Γ ) = , maxy∈Ω− |u0i (y) − ci− |2 + λ [fin (z1in , · · · , znin )H(φ) Ω for x ∈ Ω−. + fout (z1out , · · · , znout )(1 − H(φ))] dx. The desired truth table can then be described using the ziin ’s and ziout ’s. Table 2 shows three ex- Here each c ± = (c1± , · · · , cn± ) is in fact a multi- amples of logical operations for the two-channel channel vector. The associated Euler-Lagrange equa- case. Notice that “true” is represented by 0 inside tion is similar to the scalar model: Γ. The method is designed so as to encourage en- ∇φ ergy minimization when the contour tries to cap- ∂φ = δ(φ) µdiv ture the targeted object inside. Also note that the ∂t |∇φ| “ziin ” terms and the “ziout ” terms play asymmetric but complementary roles. For example, the union − λ fin (z1in , · · · , znin ) − fout (z1out , · · · , znout ) , A1 ∪ A2 corresponds to the union of the “in” terms and the intersection of the “out” terms. Similarly, with suitable boundary conditions as before. Even the intersection A1 ∩ A2 corresponds to the inter- though the form often looks complicated for a typ- section of the “in” terms and the union of the “out” ical application, its implementation is very similar terms. to that of the scalar model. We then design continuous objective functions Numerical results support our above efforts. Fig- to smoothly interpolate the binary truth table. This ure 9 shows two different occlusions of a triangle. JANUARY 2003 NOTICES OF THE AMS 25

We are able to success- of Variations, Appl. Math. Sci., vol. 147, Springer-Ver- fully recover the union, lag, 2001. the intersection, and the [2] T. F. CHAN, S.-H. KANG, and J. SHEN, Euler’s elastica and differentiation of the ob- curvature based inpaintings, SIAM J. Appl. Math. (2002), jects in Figure 10 using in press. our model. In Figure 11 [3] T. F. CHAN, B. SANDBERG, and L. VESE, Active contours with- out edges for vector-valued images, J. Visual Comm. we have a two-channel Image Rep. 11 (1999), 130–41. image of the brain. In [4] T. F. CHAN and J. SHEN, Mathematical models for local one channel we have a nontexture inpaintings, SIAM J. Appl. Math. 62 (2001), “tumor” with some 1019–43. noise, while the other [5] T. F. CHAN and L. A. VESE, Active contours without channel is clear. The im- edges, IEEE Trans. Image Process. 10 (2001), 266–77. ages are not registered. [6] L. COHEN, Avoiding local minima for deformable curves We want to find in image analysis, Proc. 3rd. Internat. Conf. on Curves A1 ∩ ¬A2 so that the and Surfaces (Chamonix-Mont Blanc, 1996), Vol. 1, (A. tumor can be observed. Le Méhauté, C. Rabut, and L. L. Schumaker, eds.), Van- Figure 11. Region-based logical This happens to be a derbilt Univ. Press, Nashville, TN, 1997, pp. 77–84. model on a medical image. In the first [7] L. COHEN, E. BARDINET, and N. AYACHE, Surface recon- very complicated exam- channel, A1, the noisy image has a struction using active contour models, Proc. SPIE Conf. ple, as there are a lot of “brain tumor”, while channel A2 does on Geometric Methods in Computer Vision, SPIE, Belling- features and textures. not. The goal is to spot the tumor ham, WA, 1993. However, the model that is in channel A1 but not in A2, i.e., [8] M. G. CRANDALL, H. ISHII, and P. L. LIONS, User’s guide to finds the tumor suc- the differentiation A1 ∩ ¬A2 . In the viscosity solutions of second order partial linear dif- cessfully. right-hand column we observe that ferential equations, Bull. Amer. Math. Soc. (N.S.) 27 the tumor has been successfully (1992), 1–67. captured. [9] S. ESEDOGLU and J. SHEN, Digital inpainting based on the Mumford-Shah-Euler image model, European J. Appl. Math. 13 (2002), 353–70. Conclusion [10] S. GEMAN and D. GEMAN, Stochastic relaxation, Gibbs In this article we have discussed some recent develop- distributions, and the Bayesian restoration of images, ments in one successful approach to mathematical im- IEEE Trans. Pattern Anal. Machine Intell. 6 (1984), age and vision analysis, the variational PDE method. 721–41. Besides the inpainting and segmentation problems dis- [11] UCLA Imagers homepage, http://www.math.ucla. cussed here, some other problems for which this method edu/~imagers/. is well suited are adaptive image enhancement and scale- [12] S. MALLAT, A Wavelet Tour of Signal Processing, Aca- space theory, geometric processing of curves and sur- demic Press, 1998. faces, optical flows of motion pictures, and dynamic ob- [13] D. MUMFORD and J. SHAH, Optimal approximations by ject tracking. Advantages of the method include faithful piecewise smooth functions and associated variational modeling and processing of vision geometry and its re- problems, Comm. Pure Applied. Math. 42 (1989), lated visual optimization, effective simulation of dy- 577–685. [14] S. OSHER and J. SETHIAN, Fronts propagating with namic visual processes such as selective diffusion and curvature-dependent speed: Algorithms based on information transport, and close interaction with the Hamilton-Jacobi formulation, J. Comput. Phys. 79 rich literature of numerical analysis and computational (1988), 12–49. PDEs. This subject shows that mathematics has a key role [15] L. RUDIN, S. OSHER, and E. FATEMI, Nonlinear total vari- to play in addressing real-world problems in science ation based noise removal algorithms, Physica D 60 and technology. Some challenges for the future are fur- (1992), 259–68. ther theoretical study on the variational and PDE mod- [16] B. SANDBERG and T. F. CHAN, Logic operations for els developed in recent years, more intrinsic integration active contours on multi-channel images, UCLA with stochastic modeling and applied harmonic analy- Department of Mathematics CAM Report 02-12, 2002. sis such as geometric wavelets, and more systematic in- [17] B. SANDBERG, T. F. CHAN, and L. VESE, A level-set and vestigation on the computation and numerical analysis Gabor-based active contour algorithm for segmenting of geometry-based variational optimizations and PDEs. textured images, UCLA Department of Mathematics CAM report 02-39, 2002. Acknowledgments [18] G. SAPIRO, Geometric Partial Differential Equations Jackie Shen thanks his former advisors Gilbert Strang and and Image Processing, Cambridge University Press, Stan Osher for their constant support and inspiration. 2001. [19] L. A. VESE and T. F. CHAN, A multiphase level set framework for image segmentation using the Mumford References and Shah model, UCLA Department of Mathematics [1] G. AUBERT and P. KORNPROBST, Mathematical Problems in Image CAM Report 01-25, to appear in Internat. J. Comp. Vi- Processing: Partial Differential Equations and the Calculus sion (2001). 26 NOTICES OF THE AMS VOLUME 50, NUMBER 1

You can also read