Video subtitle processing system - Patent Review 5684542

Description Submit all comments and votes

FIELD OF THE INVENTION

The present invention relates to a video subtitle data encoder and decoder suitable for use in transmitting subtitle data together with video data so that subtitle information may be displayed in a superimposed manner on a video image with reduced degradation of video image quality.

BACKGROUND OF THE INVENTION

Typically, when one watches, for example, a foreign movie, a textual subtitle is often superimposed on the display screen along with video imagery. In video, disks or ordinary TV broadcasting, video signals are transmitted in a state in which subtitles have been superimposed beforehand on the video images. In currently known systems, such as CAPTAIN and CD-G, subtitles may be transmitted as character codes or dot patterns.

CD-G is adapted to be able to record graphics utilizing subcodes and utilizing that, it is also possible to record subtitles in a CD. In the CD-G, one frame of data is composed of one byte subcode and 32 byte data, as shown in FIG. 22. In the 32 byte data, six samples of two byte data per sample are allocated to L and R channels, respectively. Accordingly, the total is 24 bytes. Then, an eight byte error correction code is added to the 24 byte audio data; resulting in the data of 32 bytes in total.

On the other hand, 98 frames of subcodes are assembled to compose one block. Among the 98 frames of subcodes, the first two frames of subcodes are allocated for sync patterns SO and S1. Various subcode data may be recorded in the remaining 96 frames of subcodes. However, among one byte subcodes (each bit is represented by P through W) data for searching a track has been already allocated for data in P and Q channel. Then, graphic data may be allocated with 6 bits in the remaining R Channel through W channel. That is, a substantial range in which the graphic data can be allocated is 6.times.96 bits.

Because one block data is transmitted with a frequency of 75 Hz, an amount of one frame of data transferred is 75.times.98 Hz. Accordingly, the bit rate for transmitting the subcodes is 7.35 kbyte/s.

FIG. 23 shows a transmission format of such graphic data. As shown in the figure, a packet is composed of 96 symbols of data, with one symbol being 6 bit data from R channel through W channel, and each packet is composed of four packs. Each pack is composed of 24 symbols from symbol 0 to symbol 23. Mode information is allocated to three bits of R, S and T in the symbol 0 and item information is allocated to three bits of U, V and W, respectively. The following modes are defined by combining such modes and items:

MODE ITEM

000 000 Zero Mode

001 000 Graphic Mode

001 001 TV-graphic Mode

111 000 User Mode

Then, an instruction is allocated to the symbol 1 and the mode and parity bits for the item and instruction are allocated to the symbols 2 and 3, so that a substantial range to which graphic data can be allocated is 12 symbols, for example, as shown in FIG. 23 among the symbols from symbol 4 through symbol 19. Parity bits for 20 symbols from the symbol 0 through symbol 19 are allocated to four symbols from symbol 20 through symbol 23.

Graphic data can be thus allocated as binary data in the range of 6.times.12 pixels of each pack in the CD-G. Because the rate of the pack is 75.times.4=300 packs/s, 300 characters may be transmitted per one second if one character is allocated in this range of 6.times.12 pixels.

Further, because one screen defined in the CD-G is 288 horizontal pixels.times.192 lines, it takes 2.56 seconds to transmit this one screen of characters as shown in the following expression:

(288/6).times.(192/12)/300=2.56

Still more, because different patterns have to be transmitted four times per one character pattern if a hexadecimal representation is made in each pixel, it takes 10.24 seconds which is four times of the above time.

By the way, among such conventional methods, there has been a disadvantage that an user cannot turn on or off a displayed subtitle arbitrary in the method by which the subtitle superimposed on a video image is transmitted, like the video disk and normal TV broadcasting. Further, there has been another disadvantage in them that they do not allow to prepare subtitles in a plurality of languages and to let the user select certain one.

Contrary to that, although the method in the CAPTAIN system and CD-G allows to arbitrary turn on off the display of the subtitle, it has a disadvantage that its resolution is not good enough.

That is, whereas a displayable area of one screen is 248 horizontal pixels.times.192 lines in the CAPTAIN system, component digital TV signal has a resolution of 720 horizontal pixels.times.480 lines, and it can be seen that the resolution of the former is not as good as compared to the resolution of the latter.

Furthermore, because only one bit data can be accommodated per one pixel in the CD-G, data is represented by binarizing it. Accordingly, there has been a disadvantage that such phenomena as an aliasing phenomenon by which slanted portions of a character appears in zig-zag and a flicker phenomenon by which characters flicker become significant, giving an unpleasant feeling to the user.

Although it is conceivable to convert the binary information to multi-valued information by means of a filter for example, it requires a high precision filter and hence increases the cost. The use of such filter may also degrade the background image.

If one pixel is represented in hexadecimal in the CD-G, it takes about four times the amount of time when it is displayed in binary and it becomes difficult to switch the display of subtitles quickly. Further, because the data rate is low in the method in the CAPTAIN system and CD-G, it has been difficult to display subtitles which dynamically change timewise.

Accordingly, it is an object of the present invention to solve the aforementioned problems by providing means which allows to display a high quality subtitle in accordance to a preference of an user, to switch the display of the subtitle quickly without degrading the background video image and to display the subtitle which dynamically changes timewise with less data amount.

SUMMARY OF THE INVENTION

The subtitle data encoding unit of the present invention for encoding a subtitle to be superimposed on a video image and to be displayed comprises a color lookup table (CLUT) 71 as memory means for storing color data and a color quantization circuit 70 as detecting means for detecting an address in the CLUT 71 for storing color data which corresponds to a color of the subtitle, and is characterized in that the address detected by the color quantization circuit 70 is transmitted as color information concerning on the color of the subtitle.

The subtitle data encoding unit is characterized in that it transmits, in addition to the color information, pattern information concerning on a pattern of the subtitle and a repeat time which is a number of vertical synchronous signals in the video image on which the subtitle is continuously superimposed.

The subtitle data encoding unit for encoding a subtitle to be superimposed on a video image and to be displayed comprises range specifying means (e.g., Step S53 in a program in FIG. 16) for specifying a range in which the subtitle is to be displayed, bit number determining means (e.g., Step S55 in the program in FIG. 16) for determining a bit number per one pixel in quantizing the subtitle, encoding means (e.g., Step S58 in the program in FIG. 16) for encoding in correspondence with values determined by the range specifying means and bit number determining means, and correcting means (e.g., Step S60 in the program in FIG. 16) for correcting the values determined by the range specifying means and bit number determining means in correspondence with an amount of codes encoded by the encoding means.

This subtitle data encoding unit may be further provided with adjusting means (e.g., Step S54 in the program in FIG. 16) for adjusting a horizontal resolution of the subtitle.

The recording medium according to one aspect of the invention is characterized in that it records the data of subtitle encoded by the subtitle data encoding unit described above.

According to another aspect of the invention, a subtitle data decoding unit for decoding data of the subtitle encoded by the subtitle data encoding unit and containing, in addition to the color information, the pattern information concerning on a pattern of the subtitle comprises a pattern decoder 100 as pattern information decoding means for decoding the pattern information and a color decoder 101 as color information decoding means for decoding the color information, and is characterized in that the pattern decoder 100 or color decoder 101 performs a process for decoding the pattern information or color information respectively in parallel.

The subtitle data decoding unit according to a further aspect of the invention is characterized in that the color decoder 101 further comprises a color lookup table (CLUT) 32 for storing color data as output means and that the CLUT 32 outputs data of a color stored at an address when the address is specified, that the color information is the address for outputting a color rectangular area (frame) composed of color data from the CLUT 32 and that the address which is the color information may be modified corresponding to the pattern information.

The subtitle data decoding unit according to yet another aspect of the invention is characterized in that the pattern information contains at least data of brightness of the subtitle and further comprises an arithmetic unit 33 as multiplication means for multiplying the output of the CLUT 32 and the brightness data.

The subtitle data decoding unit according to another aspect of the invention is characterized in that the pattern information is composed of either of the subtitle brightness data or key data which corresponds to a mute factor of the video signal in superimposing the subtitle and discrimination information for discriminating either of them and that the address which is the color information is modified corresponding to the discrimination information.

According to another aspect of the invention, the subtitle data decoding unit for decoding the data encoded by the subtitle data encoding unit comprises a code buffer 22 or 28 as data storage means for storing the data and is characterized in that the data stored in the code buffer 22 or 28 is repeatedly decoded by a number of times of the repeat time.

The subtitle data decoding unit according to another aspect of the invention is characterized in that the repeat time is decremented, when a n-time speed reproduction is made, by one with a timing of n times of a timing of the vertical synchronous signal of the video image and that the data stored in the code buffer 22 or 28 is repeatedly decoded until the repeat time becomes zero.

According to the subtitle data encoding unit, an address in the CLUT 71 storing data of colors which correspond to colors of a subtitle is detected and the address is transmitted as color information concerning on the color of the subtitle, so that the colored subtitle may be transmitted with less information amount.

According to the subtitle data encoding unit according to another aspect of the invention, pattern information concerning on a pattern of a subtitle and a repeat time which is a number of vertical synchronous signals in a video image on which the subtitle is continuously superimposed are transmitted, in addition to the color information, so that the subtitle superimposed on the video image across a plurality of frames or fields may be transmitted with less data amount.

According to the subtitle data encoding unit according to a further aspect of the invention, the coding of the subtitle is carried out in Step S58 in the program in FIG. 16 corresponding to the display range and a number of bits specified in Steps S53 and S55, respectively. Then, the code amount is compared with the criterion value in Step S60 and the range and the number of bits are corrected corresponding to the comparison result. Accordingly, the subtitle display range may be freely changed to any range without modifying the basic structure of the subtitle data encoding unit.

According to the recording medium according to yet another aspect of the invention, it records the data of the subtitle encoded by the subtitle data encoding unit described in claim 1, so that it can store many more information other than the subtitle data.

According to the subtitle data decoding unit according to another aspect of the invention, the pattern decoder 100 or the color decoder 101 performs the process for decoding the pattern information or color information respectively in parallel, so that the decoding can be completed quickly. Further, the color information may be modified by the pattern information, so that a subtitle whose color changes timewise can be displayed with less data amount. According to the subtitle data decoding unit according to a further aspect of the invention, the color frame composed of color data is output from the CLUT 32 by giving an address which is the color information to the CLUT 32 by modifying it corresponding to the pattern information, so that the subtitle whose color changes timewise may be displayed with less data amount.

According to the Subtitle data decoding unit according to an aspect of the invention, the output of the CLUT 32 and brightness data are multiplied, so that a subtitle having a smooth edge may be displayed.

According to the subtitle data decoding unit according to another aspect of the invention, the color frame composed of color data is output from the CLUT 32 by giving an address which is color information to the CLUT 32 by modifying it corresponding to discrimination information, so that the subtitle whose color changes timewise may be displayed with less data amount.

According to the subtitle data decoding unit according to a further aspect of the invention, stored data is repeatedly decoded by a number of times of the repeat time, so that the subtitle may be superimposed on the video image across a plurality of frames or fields.

According to the subtitle data decoding unit according to another aspect of the invention, the repeat time is decremented, when a n-time speed reproduction is made, by one with a timing of n times of a timing of the vertical synchronous signals of the video image and the data is repeatedly decoded until the repeat time becomes zero, so that the subtitle may be superimposed on the video image across a predetermined plurality of frames or fields corresponding to the reproduction speed.

The above and other related objects and features of the present invention will be apparent from a reading of the following description of the disclosure found in the accompanying drawings and the novelty thereof pointed out in the appended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a construction of one embodiment of an encoding unit of the present invention;

FIGS. 2A and 2B are diagrams for explaining a relation of subtitle data and key data;

FIG. 3 is a diagram for explaining a principle for encoding the subtitle data and key data;

FIG. 4 is a flowchart for explaining a key data quantization operation in a quantization circuit 64 in FIG. 1;

FIG. 5 is a flowchart for explaining a subtitle data quantization operation of the present invention;

FIGS. 6A and 6B are drawings for explaining a squeezing method;

FIG. 7 is a block diagram showing a detailed arrangement of a processing circuit of the present invention;

FIG. 8 is a variable length encoding (variable length decoding) table;

FIG. 9 is a diagram showing an encoded data format;

FIG. 10 is a block diagram showing a construction of one embodiment of a decoding unit to which the present invention is applied;

FIG. 11 is a block diagram showing a construction of a subtitle decoding unit of the present invention;

FIG. 12 is a diagram for explaining an operation of a code buffer 22 (and a code buffer 28) in FIG. 11;

FIG. 13 is a diagram showing an exemplary construction of a color lockup table in FIG. 11;

FIGS. 14A through 14D are drawings for explaining an operation of the CLUT 32 in FIG. 14;

FIG. 15 is a variable length encoding (variable length decoding) table;

FIG. 16 is a flowchart for explaining the processing operation of the quantization circuit 64 in FIG. 1;

FIGS. 17A through 17C are drawings for explaining an example of the processing of the present invention;

FIG. 18A and 18B are diagrams for explaining a horizontal resolution;

FIGS. 19A through 19C are drawings for explaining another example of the process of the present invention;

FIGS. 20A through 20C are drawings for explaining still another example of the processing in FIG. 16;

FIGS. 21A through 21C are drawings for explaining another processing in FIG. 16;

FIG. 22 is a diagram for explaining a conventional subcode format; and

FIG. 23 is a diagram showing a conventional subcode transmission format.

DETAILED DESCRIPTION OF THE INVENTION

Referring now to the drawings, preferred embodiments of the present invention will be explained. FIG. 1 is a block diagram showing a construction of one embodiment of a subtitle data encoding unit to which the present invention is applied. In the present embodiment, a video signal output from a video camera 51 is supplied to a video encoding unit 52, is converted from analog to digital, is compressed and then packeted. A video disk player or video tape recorder may be used of course instead of the video camera and a video signal reproduced by them may be supplied to the video encoding unit 52.

The video encoding unit 52 contains a rate controller 52a for controlling a compression ratio of the video data corresponding to a bit rate control signal output by a packing circuit 68. That is, while subtitle data is encoded in a subtitle encoding unit 57 as described later, if an amount of the encoded data is small, the overall data amount will not increase even if an amount of encoded video data is increased by that amount of the subtitle. That is, the quality of the video image may be improved to that degree. Contrary to that, when the subtitle data amount is large, a data amount allocated to the video data is decreased.

The video data (e.g., component signals of 4:2:2) thus compressed, encoded and packeted by the video encoding unit 52 is supplied a multiplexer 58. Similarly, an audio signal collected by a microphone 53 is supplied to an audio encoding unit 54 to be converted from analog to digital. Then it is compressed, encoded and packeted. A tape recorder (CD player or the like) for example may be used instead of the microphone 53 also in this case to supply an audio signal reproduced by that to the audio encoding unit 54. The audio data encoded by the audio encoding unit 54 is also supplied to the multiplexer 58.

On the other hand, subtitle data generated by a character generator 55 or that output from a flying spot scanner 56 is supplied to a subtitle data encoding unit 57 to be compressed, encoded and packeted and supplied to the multiplexer 58.

The multiplexer 58 multiplexes (e.g., time-division multiplexing) the packeted data output respectively from the subtitle encoding unit 57, video encoding unit 52 and audio encoding unit 54. After performing a process for correcting errors such as ECC and a modulation process such as EFM on the data, the multiplexer 58 records them to a recording medium such as a disk 91 and transmits to the receiver side via a channel.

Next, the subtitle encoding unit 57 will be explained further. The character generator 55 generates subtitle data which corresponds to a video image encoded by the video encoding unit 52 and supplies it to a contact (a) of a switch 61, in the subtitle encoding unit 57. The switch 61 is switched to the contact (a) or contact (b) with a predetermined timing to select subtitle data or key data as necessary to supply it, to a quantization circuit 64 via a digital filter circuit 72 and contact (b) of a switch 62.

Referring now to FIGS. 2A and 2B, a relation of the key data and subtitle data (fill data) will be explained. Here assume that a character A exists as a character to be displayed in a subtitle as shown in FIG. 2A. Subtitle data of a line represented by one horizontal line in FIG. 2A is shown in FIG. 2B. As shown in FIG. 2B, the subtitle data has a level which corresponds to a brightness of the character to be displayed in a period T3. Then, in periods T1 and T2 and periods T4 and T5 before and after that, the level of the subtitle data stays at the lowest level. Accordingly, it can be said that the subtitle data is composed of a pattern of the character to be displayed and a brightness level within the pattern (pattern information concerning on subtitle pattern).

The key data, on the other hand, stays at the lowest level in the period T3 in which the character is to be displayed and becomes the highest level in the periods T1 and T5 separated before and after the period T3. Then the level in the period T2 between the periods T1 and T3 and the period T4 between the T3 and T5 is set at a predetermined intermediate level between the lowest and highest level. In the period T2, the level changes gradually from the highest, level to the lowest level and in the period T4, the level changes gradually from the lowest level to the highest level.

That is, in the period T3, the level of a video signal of the background video image is substantially muted to a black level. On the other hand, the level of a subtitle signal which corresponds to the subtitle is muted to a predetermined level (although to a predetermined gray level in the present embodiment, it may be a black level) in the periods T1 and T5. Then, in the periods T2 and T4, the background video image is muted with a ratio corresponding to the values of the key data. In the present embodiment, the greater the value of the key data, the smaller the mutation of the background video image is (the greater a mute factor of the subtitle is) and the smaller the value of the key data, the greater the ratio of mutation of the background video image is (the smaller the mute factor of the subtitle is). Thus, the background video image is substantially completely muted during the period in which the character is displayed and is gradually muted near the character, so that the subtitle (character) is kept to be easily Seen.

The quantization circuit 64 consolidates the levels of subtitle data and key data based on a predetermined quantization table included therein and represents as four bits data. FIG. 3 shows a principle for consolidating and representing the levels of subtitle data (fill data) and key data. As shown in the figure, among 16 stages of levels which can be represented by four bits, 8 stages of levels from 0 to 7 are allocated to the level of the key data and 8 stages of levels from 8 to 15 are allocated to the subtitle data. That is, the levels of the key data are represented by four bit data from 0000 to 0111 and the levels of the subtitle data are represented by four bit data from 1000 to 1111.

As a result, the MSB of the four bit data represents, so to speak, a transmission level, and when the MSB is 1, the subtitle data is displayed and when it is 0, the background video image is muted and the subtitle is displayed in the level of 0 (level of black frame). When the key data level is 7 (0111), the mute factor of the background video image becomes 0 and the background video image is represented in the level as it is.

By thus consolidating the key data and subtitle data and representing them by four bit data (since the MSB indicates the type, each level substantially represents three bit data), the four bit data may be disposed within a bit stream as substantially the same data and hence the circuit structure may be simplified. By the way, the bit allocation to the subtitle data may be any number of bits from one to four bits (when four bits are allocated to the subtitle data, the key data is ignored since there is no bit to be allocated to the key data). That is, the allocation of the levels to the subtitle data and key data may be appropriately changed.

FIGS. 4 and 5 show a principle for quantizing the levels of the key data and subtitle data. As shown in FIG. 4, when the key data has levels of 256 stages from 0 to 255, the quantization circuit 64 classifies the levels of 256 stages into eight ranges and discriminates into which range a value of inputted key data (y) falls in Step S21 through S28. That is, the eight ranges are: from 0 to 6, 7 to 10, 11 to 26, 27 to 50, 51 to 95, 96 to 130, 131 to 204 and 205 to 255. When it is discriminated that the value falls into either one of those ranges in Step S21 through S28, the process advances to Step 829 through S36 to output either one value of 7 (0111) through 0 (0000) as four bit quantized data (z).

Similarly, when the subtitle data has levels of 256 stages from 0 stage through 255 stage, it is discriminated into which range of 255 to 234, 233 to 215, 214 to 150, 149 to 140, 139 to 128, 127 to 86, 85 to 55 and 54 to 0 the subtitle data (fill data).times.falls. When it is discriminated that it belongs to either one of those ranges, the process advances from Step S9 through S16 respectively to set either value of 15 (1111) through 8 (1000) as four bit quantization data (z).

The quantization circuit 64 is supplied with blanking data by the character generator 55 via a contact (a) of the switch 62. Various data may be inserted to the blanking data as necessary.

Furthermore, the quantization circuit 64 is supplied With subtitle data outputted by a processing circuit 63 via a contact (c) of the switch 62. The processing circuit 63 processes an analog subtitle signal outputted by the flying spot scanner 56 and outputs it as digital subtitle data.

When the video signal supplied to the video encoding unit 52 is what a movie is converted into the video signal, its aspect ratio is oblong as shown in FIG. 6A for example. Some recent TV receivers have a screen having an aspect ratio of 16:9 as typified by a so-called high definition television receiver and such TV receiver having such aspect ratio can display pictures having the aspect ratio of movie on its screen as it is.

Contrary to that, the aspect ratio of screen of the conventional NTSC scheme is defined to be 4:3 as shown in FIG. 6B. Although the picture having the oblong aspect ratio can be viewed by the screen having the aspect ratio of the normal NTSC scheme by converting the aspect ratio into 4:3, the picture becomes longitudinal by doing so, as shown in FIG. 6B.

However, the TV receiver having the oblong aspect ratio allows to view the picture having the correct ratio as shown in FIG. 6A by returning the video signal converted into the aspect ratio of 4:3 into the original aspect ratio. Many of the receivers having the oblong aspect ratio contain a converter for returning a video signal converted into the aspect ratio of 4:3 by a squeezing method thus into the original aspect ratio. Then, when a picture having the oblong aspect ratio is input, the video encoding unit 52 is adapted to encode it by converting into the aspect ratio of 4:3 by the squeezing method as shown in FIG. 6B.

When the aspect ratio of the picture is thus converted by the squeezing method, a subtitle having the oblong aspect ratio also have to be converted by the squeezing method. The processing circuit 63 performs such a function.

FIG. 7 shows an exemplary construction of the processing circuit 63. The flying spot scanner 56 supplies an analog subtitle signal Vin which corresponds to the video image input to the video encoding unit 52 to a comparison circuit 135 in the processing circuit 63. The comparison circuit 135 is also supplied with reference voltages output by a voltage dividing circuit 134 having n resistances 1341 through 134n.

The comparison circuit 135 has n comparators 1351 through 135n to Which each reference voltage output by the voltage dividing circuit 134 is supplied and compares the subtitle signal supplied from the flying spot scanner 56 with each reference value. The comparators 1351 through 135n output a logical H signal for example when the subtitle signal is larger than the reference value and output a logical L signal when the reference value is larger.

An encoder 136 monitors the outputs of the comparators 1351 through 135n and decodes into signals of m bits satisfying n=2 m for n levels. Whether the value of 0 after the quantization is made to correspond with the maximum Value or minimum value output by the comparison circuit 135 is specified by input signals NMINV and NLINV. A latch circuit 137 latches the m bit data output by the encoder 136 and outputs it to the quantization circuit 64 via a contact (c) of a switch 10.

The comparison circuit 135, encoder 136 and latch circuit 137 are supplied with clocks output by an oscillation circuit 131 or 132 via a switch 133. Frequency of the clock output by the oscillation circuit 131 is 13.5 MHz and that output by the oscillation circuit 132 is 10.125 MHz. That is, the ratio of the both is set to be 4:3.

When the video signal processed in the video encoding unit 52 is that of the NTSC scheme having the normal aspect ratio of 4:3, the switch 133 is switched upward (to normal) in the figure to output the clock output by the oscillation circuit 131. Contrary to that, when the video encoding unit 52 encodes a video signal by the squeezing method, the switch 133 is switched downward (to squeeze) in the figure to output the clock output by the oscillation circuit 132. The frequency of the clock at this time is 10.125 MHz which is 3/4 of the frequency of the normal case, so that the subtitle signal input to the comparison circuit 135 is also processed by the squeezing method.

On the other hand, because the subtitle data and key data supplied from the character generator 55 are digital data, they are converted from the normal data to squeezed data by the digital filter circuit 72 to be sent to the quantization circuit 64 via the contact (b) of the switch 62.

The subtitle data quantized by the quantization circuit 64 (hereinafter, the subtitle data (fill data) in a narrow sense and the key data are combined and referred to as the subtitle data in a wide sense) is input to a DPCM circuit 65 via a switch 69 for alternately selecting the output of the quantization circuit 64 or a color quantization circuit 70 to be DPCM-ed (Differentially Pulse Code Modulated). The output of the DPCM circuit 65 is supplied to a run length coding circuit 66. The run length coding circuit 66 encodes the input DPCM subtitle data into a data pair of level data and run data. A variable length encoding circuit 67 executes a variable length encoding process on the run supplied from the run length coding circuit 66 in accordance to variable length encoding data stored in a look up table in memory. This encoding data is shown in FIG. 8. A packing circuit 68 combines the variable length encoded data and level.

When the output of the run length coding circuit 66 is data of 1, 3, 3, 5, 5, 5, . . . for example, combined output data of the packing circuit 68 becomes as follows:

00010011000000101000010. . .

The first four bits of data 0001 represents that the first data (level) is 1. The next four bit data 0011 represents that the next data (level) is 3. Because the following data (level) is also 3, the run in this case is 1. Then, in addition to a four bit data 0000 which represents a start of zero run, a VLC code 0 for the case when the run length is 1 is added. Because the next data (level) is 5, four bit data 0101 is disposed and because two 5s continue following to that, the run length is 2 and its VLC code is 10. Then, this VLC code 10 is disposed next to a start code of run 0000.

When the run length exceeds a predetermined number (27 in the case of the present embodiment), the data amount may be reduced by fixing the length of the VLC code. When the run length exceeds 27, the VLC code is fixed to 10 bits in the present embodiment. When the fixed length VLC code is used, an escape (ESC) code 11111 for indicating that it is a fixed length Code is inserted just before the VLC code. A VLC code 1111111 indicating the end is also disposed in the end of one line.

The data variable length coded by the variable length coding circuit 67 as described above is packed with the level value extracted from the run length coding circuit 66 in the packing circuit 68.

Incidentally, the packing circuit 68 adds (multiplexes) a time code (PTSS described later), position information (disp start pos and disp end pos, described later), EOP and subtitle encoding information (subtitle header described later, all of them will be described later in detail) to the subtitle data and packets and outputs them to the multiplexer 58.

The packing circuit 68 computes a generated amount of the subtitle data at constant intervals and supplies its computed result to the rate controller 52a of the video encoding unit 52. Since the rate controller 52a can recognize a bit amount of the first pass of the video coding in the processing of the first pass and when it can recognize the data amount of the subtitle data from the signal from the packing circuit 68, it sets a bit rate in the video encoding unit 52 so that it becomes a variable rate fully utilizing the capacity of the channel or the recording medium such as the disk 91.

On the other hand, when the subtitle data encoded as described above is colored, i.e. chromatic color subtitle data (fill data) is superimpo