Computational Tutorial of Steepest Descent Method and Its Implementation in Digital Image Processing

Abstract

In the last decade, optimization techniques have extensively come up as one of principal signal processing techniques, which are used for solving many previous intractable problems in both digital signal processing (DSP) problems and digital image processing (DIP) problem. Due to its low computational complexity and uncomplicated implementation, the Gradient Descent (GD) method [1] is one of the most popular optimization methods for problems, which can be formulated as a differentiable multivariable functions. The GD method is ubiquitously used from basic to advanced researches. First, this paper presents the concept of GD method and its implementations for general mathematical problems. Next, the computation of GD processes is shown step by step with the aim to understand the effect of important parameters (such as its initial value and step size) to the performance of GD. Later, the computational concept of GD method for DIP problems [2-5] is formulated and the computation of GD is demonstrated step by step. The effect of the initial value and the step size to the performance of GD method in DIP is also presented.