A fine granularity scalability encoding/decoding apparatus includes a first quantizer for performing a DCT on a motion-compensated image and quantizing a resulting value, a second quantizer for re-quantizing the value obtained by the first quantizer, an inverse-quantizer for re-quantizing a value re-quantized by the second quantizer, and a number of subtracters. A first subtracter obtains a difference between a value of N times the re-quantized value and the inverse-quantized value. A second subtracter obtains a difference between the value quantized by the first quantizer and the value quantized by the inverse-quantizer. And, a third subtracter subtracts the output value of the first subtracter from the output value of the second subtracter. Coding efficiency is increased and a bit-rate reduction of an entire coding stream is achieved by making an enhancement layer signal and a base layer signal have the same code.
