A method and apparatus for tracking a color-based object in video sequences are provided. According to the method, an initial object area in one frame of video sequences desired to be tracked is assigned, and an initial object effective window containing the initial object area is assigned. A frame following the frame containing the assigned initial object area is received as a newly input image, and an object search window containing the initial object area for tracking and the initial object effective window in the newly input image is assigned. Then, the model histogram of the initial object area corresponding to a predetermined bin resolution value and the input histogram of the image in the object search window are calculated. From the calculated object probability image, using a predetermined method, a new object area to which the initial object area moved is obtained in the next frame in which the initial object area of the frame desired to be tracked is given as a previous (tracked) object area. By doing so, the object in video sequences is tracked. Accordingly, using the continuously extracted video object region information, an object-based interactive-type additional information service function in movies, TV programs, and CFs can be implemented effectively.
