Reconfiguring a watermark detector - Patent Review 6522769

Description Submit all comments and votes

FIELD OF THE INVENTION

The present invention relates to applications of digital watermarking in conjunction with audio, video, imagery, and other media content.

BACKGROUND AND SUMMARY

Watermarking (or "digital watermarking") is a quickly growing field of endeavor, with several different approaches. The present assignee's work is reflected in U.S. Pat. No. 5,862,260; in copending applications Ser. No. 09/503,881 and 09/452,023; and in published specifications WO 9953428 and WO0007356 (corresponding to U.S. Ser. Nos. 09/074,034 and 09/127,502). A great many other approaches are familiar to those skilled in the art. The artisan is presumed to be familiar with the full range of watermarking literature.

In the present disclosure it should be understood that references to watermarking encompass not only the assignee's watermarking technology, but can likewise be practiced with any other watermarking technology.

The invention relates to methods, devices and systems for reconfiguring a watermark detector. In many applications, it is useful to be able to change the operation of a watermark detector. Such changes may include changing how the watermark detector decodes or interprets a watermark embedded in a signal of a given media type, such as audio, video or still images.

The ability to reconfigure a watermark detector is advantageous in many applications. When watermark detectors are deployed widely in consumer electronic devices and software, the capability to reconfigure the watermark detector reduces or prevents the devices and software from becoming obsolete when changes to the detector are needed. In content management schemes, where the watermark is used as a security feature, the security mechanism may be hacked. In other applications, the watermark technology provider may want to incorporate new features into the watermark detector, expand the message payload of the watermark, or change how the watermark payload is interpreted. In these circumstances, it is advantageous to be able to reconfigure a watermark detector, and particularly, to reconfigure the detector remotely.

One aspect of the invention is a method for reconfiguring a watermark detector. The method decodes a watermark embedded in a signal of a given media type, such as an image or audio signal (e.g., still images, motion pictures, audio, video, etc.). The watermark includes a command signal used to trigger a change in operation of the watermark detector. Based on the command signal, the method changes the operation of the watermark detector. This change may include changing how the watermark detector decodes or interprets a watermark in a signal of the media type.

The scope of this method encompasses a variety of implementations. The command signal may be represented as one or more bits of a watermark payload carried by the watermark. Changing the operation of the detector may include re-programming it or altering how it interprets watermark data embedded in a media object. For example, the method may transfer firmware instructions to the detector to replace instructions stored earlier. As another example, the command may change the operation of the detector according to a preprogrammed rule. The rule may define a change in watermark key, for instance. Yet another example is changing the operation of the detector by changing how a device responds to the watermark signal extracted from a media object. The behavior of a hardware or software media player, for instance, may be updated to respond differently to the watermark signal, and specifically, to the message carried in a watermark payload.

The method may use watermark payload data to change the operation of the detector. For example, the payload may specify instructions or watermark key data. In response to this payload data, the detector may install and execute new instructions or use the new watermark key to decode watermarks in media objects.

Another aspect of the invention is an alternative method for reconfiguring a watermark detector. This method receives a media object and a command associated with the media object signaling that the watermark detector requires an upgrade to decode a watermark from the media object. In response to the command, the method updates the watermark detector to create an updated watermark detector. It decodes the watermark from the media object with the updated watermark detector. The command may be encoded in a watermark in the media object, or may be conveyed in a channel different than the watermark channel yet transmitted along with the media object (e.g., out-of-band channels like a file header or footer, sub-titling data channel, SCA channel, etc.). The scope of this method encompasses a variety of implementations. In addition, aspects of this method may be used in combination with aspects of the method summarized earlier.

Yet another aspect of the invention is a re-programmable watermark detector. The detector comprises a watermark decoder for detecting a command to upgrade the detector. It also includes instructions that are replaceable in response to detecting the command to upgrade the detector. The instructions may be conveyed to the watermark detector along with a media object, either in a watermark payload, or a channel other than the watermark channel.

Another aspect of the invention is a method of an encoding an upgrade trigger in a watermark. The method receives a media object of a given media type, and encodes a watermark into the media object. The watermark includes a command signal used to trigger a change in operation of a watermark detector. When received in a detector, this change operates to alter how the watermark detector decodes or interprets a watermark in a signal of the media type.

Further features and aspects of the invention will become apparent with reference to the following detailed description and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing the participants, and channels, involved in the distribution of music.

FIG. 2 shows a conceptual model of how music artists, record labels, and E-Music distributors can all interact with a Media Asset Management System, of which several are detailed in the following specification.

DETAILED DESCRIPTION

For expository convenience, much of the following discussion focuses on music, but the same principles and techniques are largely or wholly applicable to other source data, whether non-music audio, video, still imagery, printed materials, etc.

Music Asset Management

Referring to the figures, the music distribution process begins with a creative artist 10. The artist's music has traditionally been distributed by a record label 12. (While the following discussion refers to distribution through such a label, it should be understood that such distribution can just as well be effected directed under the artist's control, without a record label intermediary.)

In traditional distribution 14, the record label produces tangible media, such as records, tapes, videos (e.g. music videos), and CDs 16. These media are physically distributed to end-consumers 18. Additionally, the label 12 distributes the music media to outlets 20, such as radio and TV stations, cable and satellite systems, etc., which broadcast (or narrowcast) the artist's work to an audience. Distribution through such media outlets may be monitored by playout tracking services. Playout tracking data, collected by firms including Arbitron, Nielsen, ASCAP, BMI, etc., can be used to compute royalty payments, to verify broadcast (e.g. for advertising), etc.

Increasingly, the distribution of the music to the media outlets is performed electronically. Such distribution first took the form of analog audio over high quality landlines or satellite channels. Digital audio quickly supplanted analog audio in such distribution channels due to higher fidelity.

More recently, distribution of the music from the record labels to the media outlets has occurred over secure links, now including the internet. Such security was first provided simply by scrambling the audio signal or data. More sophisticated "container"-based systems are now coming into vogue, in which the audio is "packaged" (often in encrypted form) with ancillary data.

Electronic distribution of music to the consumer is also gaining popularity, presently in the MP3 format primarily. The music providers may deal directly with the public, but more commonly effect such consumer distribution through a newly emerging tier of digital media outlets, such as internet sites that specialize in music. From such sites, consumers can download digital audio files into personal digital audio players. (The Diamond Rio, and the Audible MobilePlayer devices are some of the first of what will doubtless be a large number of entrants into this personal internet audio appliance market.) Or the downloaded data can be stored by the consumer-recipient onto any other writeable media (e.g. hard disk, CD, DVD, tape, videotape, etc.). Typically a personal computer is used for such downloading, but this intermediary may be dispensed with by coupling next generation of personal audio appliances to an internet-like link.

The data downloaded by the consumer can be stored either in the native digital format, translated into another digital format (which translation may include decryption), converted into analog and recorded in analog form, etc.

Unauthorized copying or use of the music can occur anywhere in the foregoing channels. However, one of the greatest risks occurs once the music has been delivered to the consumer (whether by tangible media, by traditional broadcast media outlets, by emerging digital distribution, or otherwise).

The general idea of embedding auxiliary data into music (i.e. watermarking) has been widely proposed, but so far has been of limited applicability.

For example, GoodNoise is planning to embed a digital signature--termed a multimedia identifier, or MMI--in its MP3 music. MMI will register the song and its author with a licensing number. In addition to providing information about the songwriter and distributor, this digital encoding may also include lyrics, liner notes, and other information. But all of the proposed uses serve only to convey information from the distributor to the consumer; use for "tracking" is actively disclaimed. (Wired News, "GoodNoise Tags MP3 Files," Feb. 3, 1999.)

The Genuine Music Coalition--a partnership of various companies in the music distribution business--likewise has announced plans to employ watermarking of MP3 music. The watermarking technology, to be provided by Liquid Audio, will convey data specifying the artist or producer contact, copyright data, and a number to track ownership. The Coalition hopes that the provision of this embedded information will help thwart piracy. Industry observers believe Liquid Audio will next introduce playback technology only plays audio in which its watermark is detected. (Wired News, "Liquefying MP3," Jan. 23, 1999.)

A similar initiative has been announced by the Recording Industry Association of America (RIAA). Termed the Secure Digital Music Initiative (SDMI), the program seeks to define a voluntary specification that will assure proper compensation to those who produce and distribute music. One element of the system will likely be a watermarking component. (Dow Jones Newswire, "Spurred By Maverick Technology, Music Industry Eyes Web," Dec. 31, 1998.)

Yet another initiative has been announced by Solana and ASCAP. Other companies promoting watermarking for music include Aris Technology, MCY.com, and AudioSoft.

The watermark payload can represent various types of data. An exemplary payload includes data relating to the artist, distribution entity, title, and copyright date/proprietor. Additionally, the payload can include a digital object identifier--an ISBN-like number issued by a central organization (e.g. a rights management organization) to uniquely identify the work.

Such payload data can be encoded literally (e.g. the title by a series of ASCII characters, etc.). In other embodiments, codes or abbreviations can be employed--with each code having a known meaning. In still other embodiments, the data can be meaningless by itself, but may serve as a key (e.g., a Unique Identifier, or UID) into a remote data database or repository. An example of such a remote data repository is a web site at a Master Global Address (MGA) associated with content, as detailed below.

An exemplary data payload may, for example, have the following format: A B C D E F G H I

Where A is a six-byte (8-bits to a byte) ASCII string serving as a digital object identifier (which may serve as a link to a Master Global Address through a default name server, as discussed below), B is a two-byte ASCII field serving as a key into an "artist" field of the remote database, C is a three-byte ASCII field serving as a key into a "title" field of the remote database; D is a 14-bit field serving as a key into a "label" field of the remote database, E is an 8-bit integer representing the work's year of first publication (with 0 representing the year 2000); F is a 10-bit field serving as a key into a "price" field of the remote database, G is a two-byte usage control string (detailed below), H is a streaming data channel, and I is a string of bits serving as a cyclic redundancy checksum for the foregoing. (More sophisticated error correcting checksums can, of course, be employed.) This payload format totals 136 bits, exclusive of the CRC coding and the streaming data channel.

This payload is encoded repeatedly, or redundantly through the music, so that the fill payload can be decoded from partial excerpts of the music.

The encoding is also desirably perceptually adaptive, so that higher energy encoding is employed where the listener is less likely to perceive the additional "noise" introduced by the encoding, and vice versa. Various techniques for perceptually adaptive encoding are known. For example, some tie the amplitude of the encoded signal to the instantaneous amplitude of the music. Others exploit psychoacoustic "masking" of one signal by a spectrally- or temporally-adjoining signal of higher energy. Still other approaches fill gaps in the music's spectrum with watermark energy.

In other embodiments, perceptually adaptive encoding is not used. In some such embodiments, no tailoring of the temporal or spectral characteristics of the watermark signal is employed. In others, the watermark signal is spectrally filtered to emphasize low frequency audio components (e.g. less than 500 hz), high frequency audio components (e.g. higher than 2500 hz), or mid-frequency audio components (500-2500 hz).

The streaming data field channel (H) is a medium by which data can be conveyed from a distribution site (or other site) to the end user. Such data may be entirely unrelated to the underlying work. For example, it may serve a utilitarian purpose, such as conveying data to a memory in the consumer device to replace previously-stored data that is out-of-date. It may be a commercial channel on which bandwidth is sold for access to the consumer or the consumer's device. Essentially any purpose can be served by this streaming data field. Unlike most of the other fields, the streaming data field may not endlessly repeat the same data, but can convey data that changes with time.

Desirably, the encoding is performed in a manner permitting recovery of the watermark data even if the audio is corrupted, e.g. by format conversion, re-sampling, tape wow and flutter, compression, coding, or various forms of audio processing (e.g. filtering, pre-emphasis, re-scaling, etc.). One way to provide for such robustness is to encode a signal of known character that can be recognized through all such corruption. By identifying such known signal, the watermark signal can then be decoded. (The known signal can take various forms, e.g. a synchronization signal, a marker signal, calibration signal, a universal code signal as described in applicant's patents, etc.)

In some embodiments, a watermark "dial-tone" signal is provided. This dial-tone signal is a low amplitude, relatively wideband, repetitive signal that commonly conveys only limited information (e.g. a single bit of information). Its presence in an audio signal can serve as a "do not record," or similar instruction signal. Alternatively, or in addition, the dial-tone signal can serve as an aid in "locking" to a plural-bit digital watermark signal that is also encoded in the audio. For example, the cyclical repetition of the signal can serve to identify the start of the plural-bit digital watermark signal. Or the spectrum or repetition rate of the signal can identify any temporal corruption of the audio. An exemplary such signal is detailed as a "simple universal code" in U.S. Pat. No. 5,636,292.

A track of music can be pre-authorized for specified types of use. For example, the usage control string of the watermark payload may include a six-bit field detailing the classes of devices for which the audio is authorized. Each bit would correspond to a different class of device. Class 1 devices may be personal playback devices with only analog-audio output. Class 2 devices may be personal entertainment devices capable of outputting music in digital (e.g. MP3, redbook, *.WAV) format, as well as analog audio. Class 3 devices may be personal computer systems (i.e. with essentially unlimited ability for processing and outputting digital audio). Etc., etc. A device to which such MP3 audio is provided would check the usage control string data to determine whether it is authorized to utilize the audio. A personal playback device with analog-only output, for example, would examine the first bit of the usage control string. If it was "1," the device would be authorized to use (i.e. playback) the MP3 data; if it was a "0," the device would refuse to play the music.

In addition to pre-authorization for certain classes of devices, the usage control string can also include bits indicating the number of permitted playbacks. This data can be encoded in bits seven through nine, representing eight possibilities:

0--no playback permitted

1--single playback permitted

2--two playbacks permitted

3--three playbacks permitted

4--four playbacks permitted

5--five playbacks permitted

6--10 playbacks permitted

7--unlimited playbacks permitted

8--refer to associated data (within the watermark, or stored at a remote site) which specifies number of permitted playbacks.

The playback device may include a non-volatile store in which the number of permitted playbacks is stored for each track of music. The device would decrement this number at the beginning of each playback.

The usage control string can also include a two-bit field (bits ten and eleven) indicating re