An image processing system comprises an image processor configured to identify a plurality of candidate boundaries in an image, to obtain corresponding modified images for respective ones of the candidate boundaries, to apply a mapping function to each of the modified images to generate a corresponding vector, to determine sets of estimates for respective ones of the vectors relative to designated class parameters, and to select a particular one of the candidate boundaries based on the sets of estimates. The designated class parameters may include sets of class parameters for respective ones of a plurality of classes each corresponding to a different gesture to be recognized. The candidate boundaries may comprise candidate palm boundaries associated with a hand in the image. The image processor may be further configured to select a particular one of the plurality of classes to recognize the corresponding gesture based on the sets of estimates.