An image processing system comprises an image processor having image processing circuitry and an associated memory. The image processor is configured to implement a gesture recognition system. The gesture recognition system comprises a cursor detector, a dynamic gesture detector, a static pose recognition module, and a finite state machine configured to control selectively enabling of the cursor detector, the dynamic gesture detector and the static pose recognition module. By way of example, the finite state machine includes a cursor detected state in which cursor location and tracking are applied responsive to detection of a cursor in a current frame, a dynamic gesture detected state in which dynamic gesture recognition is applied responsive to detection of a dynamic gesture in the current frame, and a static pose recognition state in which static pose recognition is applied responsive to failure to detect a cursor or a dynamic gesture in the current frame.