Media storage and retrieval system including determination of media data associated with requests based on source identifiers and ranges within

Description Submit all comments and votes

This application includes a microfiche appendix pursuant to 37 C.F.R. 1.96(b) containing one microfiche with seventy-six frames.

BACKGROUND OF THE INVENTION

The invention relates to non-linear editing systems and the storage and retrieval of the media associated with the system, i.e., video and audio data.

Non-linear editing on computer oriented systems involves digitizing media data recorded from a linear source, e.g., a video tape cassette, and storing the digitized media data on a storage device, e.g., a hard disk drive. Once digitized, the media data can be accessed quickly at any point in the linear sequence in which it was recorded so that various portions of the data can be accessed and edited in a non-linear way.

Editing in either a linear or non-linear system involves a similar principle. Source material from some source (video tape, audio recording, film etc.) is broken down into a series of separate "clips" representing the material desired for the final master, and then reassembling these "clips" into a final sequence achieving the desire of the editor and producer. "Clips" can be either video or audio material or both (synchronous audio and video.) In a non-linear system the typical approach involved alloting to each clip an associated digitized section of the original source in storage on the system in a "media file." The system would allow the user to manipulate the clips in order to produce the final sequence. The clips referred to the media files when certain specific information about the source media was needed, such as the original source name or nature of the media (video or audio), or when the need arose to actually view or hear (i.e., play) the media associated with the clip.

For example, a user editing on a non-linear system had the ability to manipulate clips into any order, use audio clips with other video clips, and create new clips by using smaller pieces of other clips. Tools existed to allow the user to combine clips of similar material for other effects. Video clips were used in combination to create dissolve effects, and audio clips to create various audio effects.

Typically, the output of an edit, i.e., an editing procedure such as the one described above, is an "Edit Decision List" (EDL) which can be used either by a conventional on-line editing system such as the CMX300 or a non-linear system to create or assemble a new linear sequence from other existing linear source material, e.g., video tape. The EDL is used to direct the on-line system to locate or "cue" the first frame of a desired clip which is recorded on a source video tape and loaded into a video tape recorder (VTR). The editing system then records the cued clip onto a target or destination medium, e.g., video tape, and cues the first frame of the next desired clip. (Note that the next desired clip may be recorded on the same or a different physical source medium as the first clip). Once cued, the editing system records the next desired clip onto the target medium. This process is repeated until the EDL is exhausted and the target medium represents the selected original material reorganized into the sequence described by the EDL.

The standard or conventional method when establishing a system of media archival is as follows: As each clip of source material is captured for storage in the system, the information about the clip and its actual digitized data is either coresident or linked directly at the time of the capture. Whenever the clip is referenced by the user of the system, the media associated with it is always the same particular one that was associated with it at the time of the capture (whether the media was digitized or actually was still intact on the original source). Any manipulation or editing concerning the clip or segment would directly use the media data tied to it for viewing or playback. Any information about the source that it came from or equivalent sources would need to be stored with each clip or segment. As such, the whole collection of clips or segments would be needed at any time in order to determine the breadth of any source relationships. And as new source relationships were developed it would be difficult if not impossible to inform all clips or segments of the new information. Additionally, tying the media data directly to a clip or segment would make it necessary to duplicate media data if certain clips or segments overlapped or were contained entirely within one another.

The invention solves these and other difficulties and problems.

SUMMARY OF THE INVENTION

The invention involves dynamically linking or binding a digitized representation of the media with a specific reference to the media at the time the information is needed at run time and being able to change the binding as certain facets in the system change. To that end the invention is a system for determining the media needed at the time a clip is requested to be played, viewed or information retrieved concerning the media associated with the clip. Specifically, each clip is dynamically connected to the specific media at the time that it needs access to the media associated with it.

The invention also involves the separation of information concerning the specifics of a piece of digitized media, information specific about the source material the media was derived from, and information concerning the connection of media data to those requesting or needing access to it. Specifically, the three groups of information that are distinctly separate from each other are:

(1) the information concerning physical source mediums may indicate which sets (or subsets) of physical source material are equivalent, or make correlations in the labeling of certain segments of the source material (example: film edge numbers equivalenced (i.e., correlated with time code);

(2) the information about the specific digitized media as to the type of media, the length of the data, the range on the source the media represents and the locations of such media resources; and

(3) the information concerning the binding of the media data to the requesters of media. Included in the invention is the concept that the binding of media resources to those in need of the media is not made until the request for the media is made, and the fulfillment of the request may change depending on the media available at the time of the request.

The invention also involves the method of storage and retrieval of the necessary source relational information from one invocation of the application to the next, such that it is coresident with the clips and/or media that it is specific for. This makes knowledge of the form of information storage inperceptable to the user of the system.

Advantages of such a system are described below:

Media need only be digitized once. Clips referring in part or in whole to the same media result in references to the same physical data in the system. Duplicate copies of the media are not needed or created.

Deletion and recapturing of segments of the original source results in all clips referring to the specific new source material entered into the system.

Clips requesting media from one physical source may receive media from a distinctly different physical source if the sources have been identified as equivalent.

Actual location of the media in storage is free to move to any location on disk, without notification necessary to clips requiring reference to the media.

BRIEF DESCRIPTION OF THE DRAWING

FIG. 1 is a block diagram showing the control and media data flow among the media file manager, the source manager, media storage, and media capture and editing facilities.

FIG. 2 is a block diagram showing the control and media data flow between the media database and a table containing media equivalency relationships.

DESCRIPTION OF PREFERRED EMBODIMENT

FIG. 1 is a block diagram illustrating the overall functional relationships of the media storage and retrieval system according to the invention, with media hardware, disk storage and the user interface.

As shown in FIG. 1, media sources such as VTRs containing linear media data are controlled via editing and capture systems under the user's control. Digitized media data from the capture system is transferred and stored on a mass storage volume such as a hard disk drive as digitized media files and selectively retrieved under the control of a media storage and retrieval system which is the subject of the present application. The media storage and retrieval system is implemented preferably as a software application having two distinct components: the media file manager (MFM) and the source manager (SM) along with their respective databases, namely, the media database and the SM database. A user accesses and operates on the digitized media files via calls placed by the editing system to the MFM which both creates and manages the media files. The MFM also interacts with the SM which inter alia maintains a table of relations between the linear media data, recorded, for example, on source tapes, and the digitized media files. MFM exists in modular software form and consists of a procedural interface that accepts requests for specific pieces of media from specific source material. SM exists in modular form and consists of a procedural interface that accepts requests for source material relational information, and requests to read or write source relational and specific information to an area of storage.

The source code appendix provides specific code for implementing both MFM and SM. The system makes use of two other procedural subsystems: one being a linked data list manager, and the other being a sectional file writing and reading manager. These subsystems are conventional utilities sometimes offered as "toolboxes".

Linked data list management involves the functions of linking together records of information in some order. Other procedural interactions with this tool make it possible to sort the records in a variety of orders depending on certain key fields in the records. The list manager is also able to "sift" certain records from the entire pool of records based on requested values of key fields in the records.

Sectional file writing utility provides the ability for multiple clients to write data to the same file and retrieve it without knowledge of either the format of the file or the identity of other clients using the file.

Media File Manager

Media File Manager (MFM) is responsible for the management of all media that is present at any time and available to the system for use. This media may be media recorded earlier and stored in some medium, or available on demand via some link to mechanical devices. The media may be duplicately represented in a variety of resolutions or qualities. MFM's purpose is to locate media (or information pertaining thereto) specified by a user request consisting of a range of time from some specific source. The range in time may be specified by the now common SMPTE time code, film edge numbers, real time or some other standard of range identification of source material. The request does consist of a specific source from which the media is desired. This specific source request is unique and the system works on the concept that identifiers exist that separately identify any source medium.

At any time after the system is initialized the MFM has an internal abbreviation (i.e., a log or set of records) of all the media that is known to be accessible to it, and where and how that material may be retrieved. The internal abbreviation is contained in the media database built by the MFM. When a client of the MFM requests information or access to actual media, MFM uses its internal abbreviation of the media available to determine a media source that will serve as a satisfactory response to the client's request. A MFM identifier is returned to the client. In any other requests for the specific media the client may use the identifier for quick access to the material. This identifier represents the dynamic link or binding of a client's need for media and the actual source of media to be used. This identifier remains constant to the client, and any media deletions, changes or additions are changed internally by the MFM, i.e., transparently to the user, and have corresponding effects on any identifiers already released. As described in the procedural interface, two different types of identifiers can be released by the MFM. For the sake of differentiation, one type is designated a MFM.sub.-- CRUX and the other a MFM.sub.-- GIST, the main difference between these being a level of privilege available to the client holding one and/or the other. The holder of a MFM.sub.-- CRUX is able to make any number of procedural calls to MFM requesting information about the media, but the privilege to read or write the actual media is withheld. Using a MFM.sub.-- CRUX, the client can call a routine mfm.sub.-- open which will give the client a MFM.sub.-- GIST identifier. This identifier is accepted by mfm.sub.-- read and mfm.sub.-- close to be able to read the actual media from the file. The reason for this is to give the MFM some control over which physical connections to actual media are opened or closed. On some systems this is desired as the number of channels to existing media may be limited, and MFM needs a method of managing access.

Media File Procedural Interface

The media file manager (MFM) is a procedural interface between a media requester and the media files themselves. The interface consists of a set of software modules which are described below.

Mfm Init

Mfm.sub.-- init is called at the time the application is invoked. It is a one-time operation. Its basic functionality is to initialize the linked list data structures that will be needed and determine all media available on the system at the current time. Functionally it scans all the disk drives on the system and determines if the short-hand version of the media file database which it has previously placed on the drive is valid. This is determined by comparing the time stamp on the file with the time stamp on the directory in which it was stored. If they are equal (they are made equal when the database was written) then the database is valid and MFM proceeds to read in the information pertaining to it into RAM using the sectional file routines and then passes the file to the SM (sm.sub.-- readNtable) so that it can read in the SM information stored there. Of course, the file itself is not transferred; only its address in memory. If it is invalid then the file is passed to the SM for processing the SM information contained in it (see mfm.sub.-- quit), and then all media files on the volume are scanned individually. In the reading of the media databases and the scanning of the other drives the run time media database (FIG. 1) is initialized with its original media.

Mfm.sub.-- Handle

Mfm.sub.-- handle is the call a client, e.g., the user interface (FIG. 1), uses to receive an identifier (MFM.sub.-- CRUX) giving inquiry rights on the media and for determining a binding between the request and an actual media source. The request is comprised of a source unique identifier or "id", a range on the source, type of media (video or audio,) and the physical channel requested if any that the source media was recorded from (for instance the type of media may be audio, and the physical channel may be two, indicating audio2media). To handle the request MFM sifts through its existing linked list of records based on the values of the request. (This is actually done with search support procedures within the linked list utility.) If a match is found then the handle to that record is returned in the form of a MFM.sub.-- CRUX. If no media is found, then MFM calls SM.sub.-- relate to determine if any other source material has equivalent material contained in it, equal to that being requested by the client. If so, MFM then again sifts its database, looking for the appropriate media on these other sources. If any is found a MFM.sub.-- CRUX handle is returned to the client. If no media is obtained via any of these methods, mfm.sub.-- handle returns a MFM.sub.-- CRUX for BLACK or SILENCE depending on the media type originally requested, and flags an error message that no media as requested was available.

Mfm.sub.-- Open

Mfm.sub.-- open is called by the client when it will be necessary to actually obtain media data from the media source. Mfm.sub.-- open accepts a MFM.sub.-- CRUX identifier, which the client/requester must have already obtained, and proceeds to establish a connection with the media source (open the file). Once the connection is established the client is given a MFM.sub.-- GIST identifier. This identifier can be used with mfm.sub.-- read calls to obtain the actual raw media data.

Mfm.sub.-- Read

Mfm.sub.-- read is the procedural interface used to pass actual media data from the media source into a buffer specified by the caller. The parameters to the call are designed such that the caller asks for a frame of information using a range identifier to identify the frame offset from zero of the range scale. For example in the time code ranging method, the caller asks for the absolute time code of the frame desired. The call analyzes the type of media being requested, the size of the buffer the caller has provided and the number of frames the caller has requested. Based on this information a best fit is made for the caller's buffer and the actual number of frames passed to the buffer is returned.

Mfm.sub.-- Close

Mfm.sub.-- close is used to allow MFM.sub.-- to close the channel to the media source. The call accepts a MFM.sub.-- GIST identifier and from this identifier MFM.sub.-- is able to distinguish if the media channel is open for write (a previous mfm.sub.-- create call), or open for read (a previous mfm.sub.-- open call).

If the media channel is open for write, the call examines a parameter which indicates caller specific information about the nature of the write to the channel. Length, range identifier, media identifier and data rate over time are all specified. MFM includes this information in the media channel in a header and then closes the channel. This media channel (source) is now available in the MFM database as a possible candidate for fulfilling a mfm.sub.-- handle request.

If on the other hand the channel was open for read (via a previous mfm.sub.-- open) the channel is simply noted as being closed for that particular client, and if no other clients are using the channel then the channel is closed.

Regardless of the type of closure, the MFM.sub.-- GIST identifier passed in is no longer valid, and a MFM.sub.-- CRUX identifier is passed back to the caller. This identifier would be used in further calls to mfm.sub.-- open if the client again desired an open channel to the media data.

The call mfm.sub.-- close also makes decisions on the type of channel being created. Two types are possible, temporary and disk resident. A temporary media channel exists in memory only for the duration of the application run, disk resident files are placed on disk and will be available at the time of the next application invocation. For example, an experimental dissolve effect on two other video media channels might become a temporary file, while actual video from an external source might be captured, digitized and stored in a disk resident file.

Mfm.sub.-- Create

Mfm.sub.-- create is the procedural interface used by the client who wishes to permanently store media data on file with the MFM. The call grants the caller a MFM.sub.-- GIST identifier allowing calls to mfm.sub.-- write to actually write the media data to a open channel. At the time of the call MFM checks its available space for recording of such information and sets up to receive media data into such space. Specifically mfm.sub.-- create creates files on disk and pre-allocates their size as large as possible. The initial header on the file is created and it is cast a media channel in the stage of creation. In this way it can be identified later if the caller fails to write data to the channel, or fails to close the channel (via mfm.sub.-- close.)

Mfm.sub.-- Write

Mfm.sub.-- write is the procedural interface used by the caller to actually transfer media data obtained from source into a media channel of the MFM. It is in this way that MFM is able to store media data for use later as a media source in response to mfm.sub.-- handle requests.

Specifically the call takes in a pointer to a buffer containing the data and a length on this particular buffer. The data is copied from the buffer for the specified length into the media channel identified by the MFM.sub.-- GIST identifier handled in via a previous call to mfm.sub.-- create. The buffer information is simply copied onto the end of any data already written to the channel. The channel may be a temporary channel (main memory) or a disk resident channel (disk file). The two types of records are structured according to the following formats.

Runtime MFM Record Structure

One of these is present in memory at runtime for each known media file out on disk.

See C typedef for MFM.sub.-- Crux.sub.-- t in Mfm.sub.-- Pvt.h

Channel Identifier

This is the physical channel associated with the media type from the physical source. Ie: Two tracks of audio from the same source would be differentiated by different channel identifiers.

File.sub.-- Use

An internal identifier indicating whether the media file is open for access and if so the nature of the open, read or write.

Media Type

This is an internal identifier indication what type of media data is stored in the file. Ie: video or audio or some other.

File Type

This is an internal identifier as to the format of the media stored in the file.

Volume ID

Dir ID

Filename

These three fields together indicate the exact position on disk such that the file can be opened, read