This algorithm works well on the ideal cases as well as many actual photos. Instead of doing onion peel order (just the confidence term mentioned above), the heuristic of propagating edge information works quite well. It works best filling small holes that have very strong edges into the hole.

Priority is calcuated from a confidence and a data term. The confidence term is based on the confidence of surrounding pixels. In the case of the initial contour of the mask, it is the number of already known pixels. As the algorithm continues, confidence is calculated from the confidence of newly filled surrounding pixels. An onion peel order would calculate the confidence every time, filling in the patches centered at pixels surrounded by the most known pixels.

An improvement to that is to propagate confidence values. If we are confident in our choice for a pixel, then we are just as confident in our choice for pixels in the newly filled region.

The confidence term does not take into consideration structural information from the image. This necessitates a data term. From the Object Removal by Exemplar-based Inpainting, the suggested data term is an isophote dotted with the normal of the pixel; this measures the strength of a linear structure as it hits the contour of the hole. An isophote is essentially the gradient rotated by 90 degrees and it represents the strength of flow of an edge.

Usually, taking larger patches worked better than smaller patches. But by taking larger patches, less of the image becomes available to use as the algorithm is forced to choose patches far enough from the hole in the image. Unfortunately it is up to the user to determine the appropriate size.