本书是数字视频处理领域的权威图书，深入阐述数字图像与视频处理技术。第2版更新了近80%的体系知识和内容，全书章节调整为全新的8章，包括图像滤波、运动估计、视频分割与跟踪、视频滤波、图像压缩和视频压缩等，尤其体现了近年在信号处理和计算机视觉方面的重要技术进展，以及3D、超高分辨率的视频和数字电影的最新应用。

Contents1 Multi-Dimensional Signals and Systems 11.1 Multi-Dimensional Signals 21.1.1 Finite-Extent Signals and Periodic Signals 21.1.2 Symmetric Signals 51.1.3 Special Multi-Dimensional Signals 51.2 Multi-Dimensional Transforms 81.2.1 Fourier Transform of Continuous Signals 81.2.2 Fourier Transform of Discrete Signals 121.2.3 Discrete Fourier Transform (DFT) 141.2.4 Discrete Cosine Transform (DCT) 181.3 Multi-Dimensional Systems 201.3.1 Impulse Response and 2D Convolution 201.3.2 Frequency Response 231.3.3 FIR Filters and Symmetry 251.3.4 IIR Filters and Partial Difference Equations 271.4 Multi-Dimensional Sampling Theory 301.4.1 Sampling on a Lattice 301.4.2 Spectrum of Signals Sampled on a Lattice 341.4.3 Nyquist Criterion for Sampling on a Lattice 361.4.4 Reconstruction from Samples on a Lattice 411.5 Sampling Structure Conversion 42References 47Exercises 48Problem Set 1 48MATLAB Exercises 502 Digital Images and Video 532.1 Human Visual System and Color 542.1.1 Color Vision and Models 542.1.2 Contrast Sensitivity 572.1.3 Spatio-Temporal Frequency Response 592.1.4 Stereo/Depth Perception 622.2 Digital Video 632.2.1 Spatial Resolution and Frame Rate 642.2.2 Color, Dynamic Range, and Bit-Depth 652.2.3 Color Image Processing 672.2.4 Digital-Video Standards 702.3 3D Video 752.3.1 3D-Display Technologies 752.3.2 Stereoscopic Video 792.3.3 Multi-View Video 792.4 Digital-Video Applications 812.4.1 Digital TV 812.4.2 Digital Cinema 852.4.3 Video Streaming over the Internet 882.4.4 Computer Vision and Scene/Activity Understanding 912.5 Image and Video Quality 922.5.1 Visual Artifacts 922.5.2 Subjective Quality Assessment 932.5.3 Objective Quality Assessment 94References 96Image Filtering 1013.1 Image Smoothing 1023.1.1 Linear Shift-Invariant Low-Pass Filtering 1023.1.2 Bi-Lateral Filtering 1053.2 Image Re-Sampling and Multi-Resolution Representations 1063.2.1 Image Decimation 1073.2.2 Interpolation 1093.2.3 Multi-Resolution Pyramid Representations 1163.2.4 Wavelet Representations 1173.3 Image-Gradient Estimation, Edge and Feature Detection 1233.3.1 Estimation of the Image Gradient 1243.3.2 Estimation of the Laplacian 1283.3.3 Canny Edge Detection 1303.3.4 Harris Corner Detection 1313.4 Image Enhancement 1333.4.1 Pixel-Based Contrast Enhancement 1333.4.2 Spatial Filtering for Tone Mapping and Image Sharpening 1383.5 Image Denoising 1433.5.1 Image and Noise Models 1443.5.2 Linear Space-Invariant Filters in the DFT Domain 1463.5.3 Local Adaptive Filtering 1493.5.4 Nonlinear Filtering: Order-Statistics, Wavelet Shrinkage, and Bi-Lateral Filtering 1543.5.5 Non-Local Filtering: NL-Means and BM3D 1583.6 Image Restoration 1603.6.1 Blur Models 1613.6.2 Restoration of Images Degraded by Linear Space-Invariant Blurs 1653.6.3 Blind Restoration – Blur Identification 1713.6.4 Restoration of Images Degraded by Space-Varying Blurs 1733.6.5 Image In-Painting 176References 177Exercises 182Problem Set 3 182MATLAB Exercises 185MATLAB Resources 1894 Motion Estimation 1914.1 Image Formation 1924.1.1 Camera Models 1924.1.2 Photometric Effects of 3D Motion 1974.2 Motion Models 1984.2.1 Projected Motion vs. Apparent Motion 1994.2.2 Projected 3D Rigid-Motion Models 2034.2.3 2D Apparent-Motion Models 2064.3 2D Apparent-Motion Estimation 2104.3.1 Sparse Correspondence, Optical-Flow Estimation, and Image-Registration Problems 2104.3.2 Optical-Flow Equation and Normal Flow 2134.3.3 Displaced-Frame Difference 2154.3.4 Motion Estimation is Ill-Posed: Occlusion and Aperture Problems 2164.3.5 Hierarchical Motion Estimation 2194.3.6 Performance Measures for Motion Estimation 2204.4 Differential Methods 2214.4.1 Lukas–Kanade Method 2214.4.2 Horn–Schunk Motion Estimation 2264.5 Matching Methods 2294.5.1 Basic Block-Matching 2304.5.2 Variable-Size Block-Matching 2344.5.3 Hierarchical Block-Matching 2364.5.4 Generalized Block-Matching – Local Deformable Motion 2374.5.5 Homography Estimation from Feature Correspondences 2394.6 Nonlinear Optimization Methods 2414.6.1 Pel-Recursive Motion Estimation 2414.6.2 Bayesian Motion Estimation 2434.7 Transform-Domain Methods 2454.7.1 Phase-Correlation Method 2454.7.2 Space-Frequency Spectral Methods 2474.8 3D Motion and Structure Estimation 2474.8.1 Camera Calibration 2484.8.2 Affine Reconstruction 2494.8.3 Projective Reconstruction 2514.8.4 Euclidean Reconstruction 2564.8.5 Planar-Parallax and Relative Affine Structure Reconstruction 2574.8.6 Dense Structure from Stereo 259References 259Exercises 264Problem Set 4 264MATLAB Exercises 266MATLAB Resources 2685 Video Segmentation and Tracking 2695.1 Image Segmentation 2715.1.1 Thresholding 2715.1.2 Clustering 2735.1.3 Bayesian Methods 2775.1.4 Graph-Based Methods 2815.1.5 Active-Contour Models 2835.2 Change Detection 2855.2.1 Shot-Boundary Detection 2855.2.2 Background Subtraction 2875.3 Motion Segmentation 2945.3.1 Dominant-Motion Segmentation 2955.3.2 Multiple-Motion Segmentation 2985.3.3 Region-Based Motion Segmentation: Fusion of Color and Motion 3075.3.4 Simultaneous Motion Estimation and Segmentation 3095.4 Motion Tracking 3135.4.1 Graph-Based Spatio-Temporal Segmentation and Tracking 3155.4.2 Kanade–Lucas–Tomasi Tracking 3155.4.3 Mean-Shift Tracking 3175.4.4 Particle-Filter Tracking 3195.4.5 Active-Contour Tracking 3215.4.6 2D-Mesh Tracking 3235.5 Image and Video Matting 3245.6 Performance Evaluation 326References 327MATLAB Exercises 334Internet Resources 3356 Video Filtering 3376.1 Theory of Spatio-Temporal Filtering 3386.1.1 Frequency Spectrum of Video 3386.1.2 Motion-Adaptive Filtering 3416.1.3 Motion-Compensated Filtering 3416.2 Video-Format Conversion 3456.2.1 Down-Conversion 3476.2.2 De-Interlacing 3516.2.3 Frame-Rate Conversion 3576.3 Multi-Frame Noise Filtering 3636.3.1 Motion-Adaptive Noise Filtering 3636.3.2 Motion-Compensated Noise Filtering 3656.4 Multi-Frame Restoration 3706.4.1 Multi-Frame Modeling 3716.4.2 Multi-Frame Wiener Restoration 3716.5 Multi-Frame Super-Resolution 3736.5.1 What Is Super-Resolution? 3746.5.2 Modeling Low-Resolution Sampling 3776.5.3 Super-Resolution in the Frequency Domain 3826.5.4 Multi-Frame Spatial-Domain Methods 385References 390Exercises 395Problem Set 6 395MATLAB Exercises 3967 Image Compression 3977.1 Basics of Image Compression 3987.1.1 Information Theoretic Concepts 3987.1.2 Elements of Image-Compression Systems 4017.1.3 Quantization 4027.1.4 Symbol Coding 4057.1.5 Huffman Coding 4067.1.6 Arithmetic Coding 4107.2 Discrete-Cosine Transform Coding and JPEG 4137.2.1 Discrete-Cosine Transform 4147.2.2 ISO JPEG Standard 4167.2.3 Encoder Control and Compression Artifacts 4237.3 Wavelet-Transform Coding and JPEG 2000 4247.3.1 Wavelet Transform and Choice of Filters 4257.3.2 ISO JPEG 2000 Standard 429References 435Exercises 437Internet Resources 4408 Video Compression 4418.1 Video-Compression Approaches 4428.1.1 Intra-Frame Compression, Motion JPEG 2000, and Digital Cinema 4428.1.2 3D-Transform Coding 4438.1.3 Motion-Compensated Transform Coding 4468.2 Early Video-Compression Standards 4478.2.1 ISO and ITU Standards 4478.2.2 MPEG-1 Standard 4488.2.3 MPEG-2 Standard 4568.3 MPEG-4 AVC/ITU-T H.264 Standard 4638.3.1 Input-Video Formats and Data Structure 4648.3.2 Intra-Prediction 4658.3.3 Motion Compensation 4668.3.4 Transform 4688.3.5 Other Tools and Improvements 4698.4 High-Efficiency Video-Coding (HEVC) Standard 4718.4.1 Video-Input Format and Data Structure 4718.4.2 Coding-Tree Units 4728.4.3 Tools for Parallel Encoding/Decoding 4738.4.4 Other Tools and Improvements 4758.5 Scalable-Video Compression 4778.5.1 Temporal Scalability 4788.5.2 Spatial Scalability 4798.5.3 Quality (SNR) Scalability 4808.5.4 Hybrid Scalability 4828.6 Stereo and Multi-View Video Compression 4828.6.1 Frame-Compatible Stereo-Video Compression 4838.6.2 Stereo and Multi-View Video-Coding Extensions of the H.264/AVC Standard 4848.6.3 Multi-View Video Plus Depth Compression 487References 492Exercises 494Internet Resources 495A Ill-Posed Problems in Image and Video Processing 497A.1 Image Representations 497A.1.1 Deterministic Framework – Function/Vector Spaces 497A.1.2 Bayesian Framework – Random Fields 498A.2 Overview of Image Models 498A.3 Basics of Sparse-Image Modeling 500A.4 Well-Posed Formulations of Ill-Posed Problems 501A.4.1 Constrained-Optimization Problem 501A.4.2 Bayesian-Estimation Problem 502References 502B Markov and Gibbs Random Fields 503B.1 Equivalence of Markov Random Fields and Gibbs Random Fields 503B.1.1 Markov Random Fields 504B.1.2 Gibbs Random Fields 505B.1.3 Equivalence of MRF and GRF 506B.2 Gibbs Distribution as an a priori PDF Model 507B.3 Computation of Local Conditional Probabilities from a Gibbs Distribution 508References 509C Optimization Methods 511C.1 Gradient-Based Optimization 512C.1.1 Steepest-Descent Method 512C.1.2 Newton–Raphson Method 513C.2 Simulated Annealing 514C.2.1 Metropolis Algorithm 515C.2.2 Gibbs Sampler 516C.3 Greedy Methods 517C.3.1 Iterated Conditional Modes 517C.3.2 Mean-Field Annealing 518C.3.3 Highest Confidence First 518References 519D Model Fitting 521D.1 Least-Squares Fitting 521D.2 Least-Squares Solution of Homogeneous Linear Equations 522D.2.1 Alternate Derivation 523D.3 Total Least-Squares Fitting 524D.4 Random-Sample Consensus (RANSAC) 526References 526Glossary527目　　录第1章　多维信号与系统11.1　多维信号21.1.1　有限域信号和周期信号21.1.2　对称信号51.1.3　特殊的多维信号51.2　多维变换81.2.1　连续信号的傅里叶变换81.2.2　离散信号的傅里叶变换121.2.3　离散傅里叶变换（DFT）141.2.4　离散余弦变换（DCT）181.3　多维系统201.3.1　脉冲响应和2D卷积201.3.2　频率响应231.3.3　FIR滤波器及对称性251.3.4　IIR滤波器及偏微分方程271.4　多维采样理论301.4.1　格上采样301.4.2　格上采样信号的谱341.4.3　格上采样中的奈奎斯特准则361.4.4　格上采样信号重建411.5　采样格式转换42参考文献47习题48问题集148MATLAB习题50第2章　数字图像和视频532.1　人类视觉系统和色彩542.1.1　色觉及彩色模型542.1.2　对比敏感度572.1.3　时空频率响应592.1.4　立体/深度感知622.2　数字视频632.2.1　空间分辨率和帧速642.2.2　颜色、动态范围和位深652.2.3　彩色图像处理672.2.4　数字视频标准702.3　3D视频752.3.1　3D显示技术752.3.2　立体视频792.3.3　多视角视频792.4　数字视频应用812.4.1　数字电视812.4.2　数字影院852.4.3　互联网中的视频流882.4.4　计算机视觉和场景/行为理解912.5　图像和视频的质量922.5.1　视觉效果损伤922.5.2　主观质量评估932.5.3　客观质量评估94参考文献96第3章　图像滤波1013.1　图像平滑1023.1.1　线性移不变低通滤波1023.1.2　双边滤波1053.2　图像重采样和多分辨率表示1063.2.1　图像抽取1073.2.2　图像内插1093.2.3　多分辨率金字塔表示1163.2.4　小波表示1173.3　图像梯度估计、边缘和特征检测1233.3.1　图像梯度估计1243.3.2　拉普拉斯估计1283.3.3　Canny边缘检测1303.3.4　Harris角检测1313.4　图像增强1333.4.1　基于像素的对比度增强1333.4.2　用于色调映射和图像锐化的空间滤波1383.5　图像去噪1433.5.1　图像和噪声模型1443.5.2　DFT域的线性空间不变滤波器1463.5.3　局部自适应滤波1493.5.4　非线性滤波：排序统计、小波收缩和双边滤波1543.5.5　非局部滤波：NL-Means和BM3D1583.6　图像复原1603.6.1　模糊模型1613.6.2　线性空间不变模糊图像的复原1653.6.3　盲复原—模糊识别1713.6.4　空间变化模糊图像的复原1733.6.5　图像修复176参考文献187习题182问题集3182MATLAB习题185MATLAB资源189第4章　运动估计1914.1　图像的形成1914.1.1　相机模型1924.1.2　3D运动的光学效果1974.2　运动模型1984.2.1　投射运动与表观运动1994.2.2　3D刚体运动投射模型2034.2.3　2D表观运动模型2064.3　2D表观运动估计2104.3.1　稀疏对应性、光流估计和图像配准问题2104.3.2　光流方程和法向流2134.3.3　帧间差2154.3.4　运动估计的病态性：遮挡与孔洞问题2164.3.5　分层运动估计2194.3.6　运动估计的性能衡量2204.4　差分法2214.4.1　Lukas-Kanade法2214.4.2　Horn-Schunk运动估计2264.5　匹配法2294.5.1　基本的块匹配2304.5.2　变尺寸块匹配2344.5.3　分层块匹配2364.5.4　扩展的块匹配—局部变形运动2374.5.5　特征对应的单应性估计2394.6　非线性优化法2414.6.1　像素递归运动估计2414.6.2　贝叶斯运动估计2434.7　变换域方法2454.7.1　相位相关法2454.7.2　空间-频率谱法2474.8　3D运动估计和结构估计2474.8.1　相机标定2484.8.2　仿射重建2494.8.3　投影重建2514.8.4　欧氏重建2564.8.5　平面视差和相关仿射结构重建2574.8.6　立体中的致密结构259参考文献259习题264问题集4264MATLAB习题266MATLAB资源268第5章　视频分割与跟踪2695.1　图像分割2715.1.1　阈值法2715.1.2　聚类法2735.1.3　贝叶斯法2775.1.4　图形法2815.1.5　主动轮廓模型2835.2　变化检测2855.2.1　镜头边界检测2855.2.2　背景差法2875.3　运动分割2945.3.1　主要运动分割2955.3.2　复杂运动分割2985.3.3　基于区域的运动分割：彩色与运动的融合3075.3.4　运动估计与分割的同时实现3095.4　运动跟踪3135.4.1　基于图形的空-时分割与跟踪3155.4.2　Kanade-Lucas-Tomasi跟踪3155.4.3　Mean-Shift跟踪3175.4.4　粒子滤波跟踪3195.4.5　主动轮廓跟踪3215.4.6　2D-Mesh跟踪3235.5　图像抠图和视频抠像3245.6　性能评估326参考文献327MATLAB习题334互联网资源335第6章　视频滤波3376.1　空–时滤波理论3386.1.1　视频的频谱3396.1.2　运动自适应滤波3416.1.3　运动补偿滤波3416.2　视频格式转换3456.2.1　降采样3476.2.2　去隔行3516.2.3　帧率转换3576.3　多帧联合噪声滤除3636.3.1　运动自适应噪声滤除3636.3.2　运动补偿噪声滤除3656.4　多帧联合复原3706.4.1　多帧联合建模3716.4.2　多帧联合维纳复原3716.5　多帧联合超分辨率重建3736.5.1　什么是超分辨率重建3746.5.2　低分辨率采样建模3776.5.3　频域超分辨率重建3826.5.4　空域多帧法385参考文献390习题395问题集6395MATLAB习题396第7章　图像压缩3977.1　图像压缩的基础3987.1.1　信息论概念3987.1.2　图像压缩系统的组成4017.1.3　量化4027.1.4　符号编码4057.1.5　Huffman编码4067.1.6　算术编码4107.2　离散余弦变换编码和JPEG4137.2.1　离散余弦变换4147.2.2　ISO JPEG标准4167.2.3　编码控制与压缩损伤4237.3　小波变换编码和JPEG 20004247.3.1　小波变换和滤波器选择4257.3.2　ISO JPEG 2000标准429参考文献435习题437互联网资源440第8章　视频压缩4418.1　视频压缩方法4428.1.1　帧内压缩、运动JPEG 2000和数字影院4428.1.2　3D变换编码4438.1.3　运动压缩变换编码4468.2　早期的视频压缩标准4478.2.1　ISO和ITU标准4478.2.2　MPEG-1标准4488.2.3　MPEG-2标准4568.3　MPEG-4 AVC/ITU-T H.264标准4638.3.1　视频输入格式和数据结构4648.3.2　帧内预测4658.3.3　运动补偿4668.3.4　变换4688.3.5　其他工具和改进4698.4　高效视频编码（HEVC）标准4718.4.1　视频输入格式和数据结构4718.4.2　编码树单元4728.4.3　并行编码/解码工具4738.4.4　其他工具与改进4758.5　可伸缩视频压缩4778.5.1　时间可伸缩性4788.5.2　空间可伸缩性4798.5.3　质量（SNR）分级4808.5.4　混合可伸缩4828.6　立体视频和多视角视频压缩4828.6.1　帧兼容的立体视频压缩4838.6.2　H.264/AVC标准中关于立体和多视角视频编码的扩展4848.6.3　多视角加深度信息的视频压缩487参考文献492习题494互联网资源495附录A　图像和视频处理中的病态问题497A.1　图像表示497A.1.1　确定性框架—函数/矢量空间497A.1.2　贝叶斯框架—随机场498A.2　图像模型概览498A.3　图像稀疏建模基础500A.4　病态问题的适定公式501A.4.1　条件优化问题501A.4.2　贝叶斯估计问题502参考文献502附录B　Markov和Gibbs随机场503B.1　Markov随机场与Gibbs随机场的等价性503B.1.1　Markov随机场504B.1.2　Gibbs随机场505B.1.3　MRF和GRF的等价性506B.2　先验PDF模型的Gibbs分布507B.3　Gibbs分布中局部条件概率的计算508参考文献509附录C　优化方法511C.1　基于梯度的优化512C.1.1　最速下降法512C.1.2　Newton-Raphson法513C.2　模拟退火法514C.2.1　Metropolis算法515C.2.2　Gibbs抽样516C.3　贪婪法517C.3.1　条件递归法517C.3.2　平均场退火法518C.3.3　最高信任优先法518参考文献519附录D　模型拟合521D.1　最小均方拟合法522D.2　齐次线性方程组的LS解522D.2.1　交替推导法523D.3　总体最小均方拟合法524D.4　随机采样一致性（RANSAC）526参考文献526术语表527