WikiPatents - Community Patent Review
Create Free Account  |  License or Sell Your Patent  |  WikiPatents Marketplace  |  WikiPatents Blog
Username:  Password:  
    
Advanced Search
Methods and apparatus for imaging volume data with shading    
United States Patent4835712   
Link to this pagehttp://www.wikipatents.com/4835712.html
Inventor(s)Drebin; Robert A. (Corte Madera, CA); Carpenter; Loren C. (Novato, CA)
AbstractMethod and apparatus for shading an image volume so that surfaces and boundaries may be rendered to subvoxel accuracy. A gradient vector is generated for each voxel of an image volume by calculating the change in opacity across that voxel in relation to its immediate neighbors. The gradient in the X, Y and Z direction of a three-dimensional voxel array is used to define a gradient length. By multiplying the RGBA values of individual voxels by their gradient length, a image volume may be shaded so that surfaces remain, but the interiors of solids are rendered more transparent revealing additional detail. Surfaces are shaded by multiplying the RGBA values of each voxel by a shading function. A reference point for a light source illuminating the displayed image volume is defined. A light vector is chosen for each voxel, a shading function is then generated based on the angle between the gradient vector and the light vector. The shading function allows for inputs for backlighting and side lighting.
   














 Title Information Submit all comments and votes
 
Patent Text Patent PDF Print Page Summary File History
Plain text PDF images Print Summary File History
Drawing from US Patent 4835712
Methods and apparatus for imaging volume data with shading - US Patent 4835712 Drawing
Methods and apparatus for imaging volume data with shading
Inventor     Drebin; Robert A. (Corte Madera, CA); Carpenter; Loren C. (Novato, CA)
Owner/Assignee     Pixar (San Rafael, CA)
Patent assignment
All assignments
Publication Date     May 30, 1989
Application Number     06/896,944
PAIR File History     Application Data   Transaction History
Image File Wrapper   Patent Term   Fees
Litigation
Filing Date     August 15, 1986
US Classification     345/423 345/424 345/426
Int'l Classification     G06F 003/14
Examiner     Shaw; Gareth D.
Assistant Examiner     Mills; John G.
Attorney/Law Firm     Hecker & Harriman
Address
Parent Case     This application is a continuation-in-part of application Ser. No. 851,776 filed on Apr. 14, 1986 now abandoned.
Priority Data    
USPTO Field of Search     364/200 364/900 364/568 364/521
Patent Tags     methods imaging volume data shading
   
Enter a comma (,) or semicolon (;) between multiple tag words/phrases.
Describe this patent:
 Amusing   
 Clever   
 Complex   
 Efficient   
 Historic   
 Important   
 Innovative   
 Interesting   
 Practical   
 Simple   
[no votes]
Patent WIKI

Share information and news about this patent, including information and news about the technology, inventors, company, ligation and licensing.

 References Submit all comments and votes
 
*references marked with an asterisk below are user-added references
 U.S. References
 
Add a new US reference:  
ReferenceRelevancyCommentsReferenceRelevancyComments
4730261
Smith
345/422
Mar,1988

[0 after 0 votes]
4720803
Ishii
345/22
Jan,1988

[0 after 0 votes]
4706205
Akai
345/648
Nov,1987

[0 after 0 votes]
4698778
Ito
345/667
Oct,1987

[0 after 0 votes]
4692878
Ciongoli
345/419
Sep,1987

[0 after 0 votes]
4685070
Flinchbaugh
345/424
Aug,1987

[0 after 0 votes]
4649499
Sutton
345/419
Mar,1987

[0 after 0 votes]
 Foreign References
 Other References
 Market Review Submit all comments and votes
   
Market Size
Estimate the gross annual revenues of the relevant market sector:
> $10B
$5B - $10B
$2B - $5B
$500M - $2B
$100M - $500M
$10M - $100M
$1M - $10M
$500K - $1M
$100K - $500K
< $100K
[No votes]
$0
 
$0   $2.5B   $5B   $7.5B   $10B
Market Share
Estimate the percentage of the relevant market sector this invention will capture:
75% - 100%
50% - 74.99%
25% - 49.99%
10 - 24.99%
5 - 9.99%
2 - 4.99%
1 - 1.99%
< 1%
[No votes]
0.0%
 
0%   25%   50%   75%   100%
Reasonable Royalty
What percentage of gross sales should the inventor or assignee be paid?
75% - 100%
50% - 74.99%
25% - 49.99%
10 - 24.99%
5 - 9.99%
2 - 4.99%
1 - 1.99%
< 1%
[No votes]
0.0%
 
0%   25%   50%   75%   100%
Public's "Guesstimation" of Royalty Value
Market SizeN/A[No votes]
xMarket ShareN/A[No votes]
xReasonable RoyaltyN/A[No votes]

N/A

License Availablity
If you are NOT the owner or assignee, answer here:
Yes, license is available for purchase

No, license is not currently available



[No votes]
License Availablity
If you ARE the owner or assignee, answer here:
Yes, license is available for purchase

No, license is not currently available



[No votes]
Competitive Advantage
Does this invention have a significant competitive advantage over similar technologies?
Yes

No



[No votes]
Most helpful competitive advantage comment
[No comments]

Commercial Alternatives
Are there viable commercial alternatives for this invention?
Yes

No



[No votes]
Most helpful commercial alternative comment
[No comments]

 Technical Review Submit all comments and votes
 Claims Submit all comments and votes
 


We claim:

1. A method of generating a two dimensional rendering from three dimensional volume data comprising the steps of:

providing a first image volume, said first image volume representing said three dimensional data set;

storing said first image volume in a first image memory as a plurality of homogeneous storage arrays, each array comprising a plurality of planes having a plurality of volume elements (voxels);

generating a second image volume from said first image volume by combining one of said planes of voxels with the next consecutive of said planes of voxels along a selected path;

repeating the prior step for each consecutive of said planes of voxels with each generated combination image volume through the nth plane of voxels where n represents the total number of said planes of voxels;

said combination image volume having voxels whose color and opacity values are based on the percentage composition of the color and opacity values of voxels contained in combined planes of voxels;

defining a gradient vector G for each said voxel, said gradient vector representative of the change in opacity across each voxel relative to neighboring voxels disposed adjacent said voxel, said gradient vector being normal to an inferred surface at said voxel, and having a greater magnitude for a voxel identifying a surface than for a voxel identifying a solid;

multiplying the color and opacity values of each voxel by the magnitude of said gradient of each voxel;

providing said voxels of said combination image volume to a display means whereby a surface of an image volume is determined and is capable of display on said display means.

2. The method of claim 1 wherein said first image volume is converted to a plurality of digital signals and stored in a digital memory.

3. The method of claim 1 wherein said color values comprise a red component, a green component and a blue component.

4. The method of claim 3 wherein said combination image volume is generated in a channel processor wherein said red, blue and green components and said opacity values are combined in first, second, third and fourth channels respectively.

5. The method of claim 1 wherein the color and opacity value of each voxel of a combination image volume is given by:

FG+(1-(FG)(A))(BG)

where;

FG=the color and opacity components of a voxel of one of said scan lines;

A=the opacity of said voxel of one of said scan lines;

BG=the composite color and opacity components of the corresponding voxel in previously combined scan lines.

6. In a computer system, a method of deriving a surface of a volume element (voxel) having color and opacity values, said voxel having a plurality of neighboring voxels disposed adjacent to said voxel, said method comprising the steps of:

defining a gradient vector G for said voxel representative of the amount of change in opacity across said voxel relative to said neighboring voxels, said gradient vector being normal to an inferred surface at said voxel, said gradient vector having a magnitude, said magnitude being greater for a voxel identifying a surface than for a voxel identifying a solid;

deriving a surface of said voxel by multiplying said color and opacity values of said voxel by said magnitude of said gradient vector of said voxel;

whereby a non-binary surface contribution of a voxel is determined to subvoxel accuracy and is capable of display on a display means.

7. The method of claim 1 wherein the magnitude of said gradient is found by:

Gl=.sqroot.[Gx]2+][Gy]2

where Gx=the change in opacity across said voxel in the x direction and Gy represents the change in opacity across said voxel in the y direction of an xy coordinate system.

8. The method of claim 7 wherein Gx is defined by:

Gx=(A(f)-A(d))+[(A(c)-A(g))+(A(i)-A(a))]/.sqroot.2

where A(d), A(f) represent the opacity of said neighboring voxels to said voxel in the X direction; and

A(c), A(g), A(i), A(a) represent the opacity of the nearest coplanar neighboring voxels diagonal from said voxel.

9. The method of claim 8 wherein the y component of said gradient vector is given by:

Gy=(A(h)-A(b))+[(A(g)-A(c))+(A(i)-A(a))]/.sqroot.2

where A(h), A(b) represent the opacity of the nearest voxels to said voxel in the Y direction.

10. The method of claim 1 wherein the length of said gradient vector is given by:

Gl=.sqroot.[Gx]2+[Gy]2+[Gz]2

where G(x) represents the change in opacity across said voxel in the X direction, G(y) represents the change in opacity across said voxel in the Y direction and G(z) represents the change in opacity across said voxel in the Z direction of an XYZ coordinate system.

11. The method of claim 10 wherein the x component of said gradient vector is given by: ##EQU5## where A(fl), A(dl) represent the nearest coplanar voxels to said voxel in the X direction;

A(il), A(al), A(cl), A(gl) represent the nearest coplanar voxels diagonal to said voxel in the X plane;

A(d0), A(f0), A(d2), A(f2) represent the nearest coplanar diagonal voxels to said voxel in the Y plane;

A(i0), A(a2), A(c2), A(g0), A(i2), A(a0), A(c0), A(g2) represent the opacities of the nearest neighboring voxels diagonal to said voxel.

12. The method of claim 11 wherein said y component of said gradient is given by: ##EQU6## where A(hl), A(bl) represent the opacity of the nearest neighboring coplanar voxels in the Y direction to said voxel;

A(h0), A(b2), A(bO) A(h2) represent the opacity of the nearest coplanar diagonal voxels to said voxel in the Z direction.

13. The method of claim 12 wherein the z component of said gradient vector is given by: ##EQU7## where A(e2), A(e0) represent the opacity of the nearest coplanar voxels in the Z direction.

14. The method of claim 1 further including the step of surface shading said voxel, said surface shading accomplished by the steps of:

defining a location of a light source;

defining a light vector L for said voxel;

generating a shading function by combining said light vector and said gradient vector;

multiplying the color and opacity values of said voxel by said shading function;

said shading function defined by:

((((q)(G.multidot.L)+r)(G.multidot.L)+s)(G.multidot.L)+t)(G.multidot.L)+u

where

q=SL-(BL+1)/2

r=(BL-1)/4

s=2q

t=3r

u=SL

SL=the amount of side lining utilized to shade said voxel

BL=the percentage of back lining used to shade said voxel.
 Description Submit all comments and votes
 


BACKGROUND OF THE INVENTION

In a wide variety of modern applications, it is desirable to observe the three dimensional coherence of a volume or object of interest. In the case of imaging real three dimensional ("3D") solids, ordinarily it is only possible to view discrete planar cross sections of the 3D volume and its contents. It is not possible typically to view 3D volumes such that internal and object surfaces, boundaries, interfaces, and spatial relationships within the volume can be separated and identified visually.

In medical imaging, for example, it is highly desirable to be able to visualize anatomical structures by three dimensional representations on computer graphic displays. The ability to produce accurate, volumetric, anatomical models from computerized tomographic (CT) scans is extremely valuable as a surgical aid, (such as for displaying structures to plan surgical intervention, or to represent details of the anatomical structure without the need for exploratory surgery). Thus, the 3D shape, size and relative position of pathologic structures provides important data for surgical planning, diagnosis and treatment.

Computer simulation of real 3D volumes depends on the ability to reconstruct 3D structures from planar section data, such as CT scans. These and other scans provide data from which a 3D density image volume consisting of volume elements (voxels) is available as input data for image processing. Unfortunately, such input image volumes are typically of low resolution as compared to the level of detail desired to represent accurately the sampled volume.

For example, in CT scan image volumes, voxels represent x-ray attenuation or other image volume data throughout the volume, including across surface boundaries. Although a voxel is assigned only a single "homogenous" value, in fact there exists a boundary and discrete materials on either side of the boundary within the object under consideration. Thus, a voxel along an edge is a sample extending over both sides of the edge. Further, if a material (such as a bone) is less that one voxel wide, the voxel that provides boundary information about that bone is very low resolution. Thus, the boundary shape within a voxel is not readily determined.

Various approaches have been used in an effort to approximate surface boundaries within volumes. One well known method is "thresholding". In thresholding, voxels that cross a boundary are classified as being composed of one or the other material type on either side of the boundary. The projected visible boundary thus becomes the binary classified voxel border.

The larger the voxels, the greater the error that is introduced by thresholding. Further, for coarse images or images with high density and closely spaced surface boundaries, thresholding provides an even further degraded result, such that the resultant images become less and less accurate. Subsequent approximation techniques are sometimes used in an attempt to render a more accurate approximation from the thresholding result. However, attempts to approximate the unknown surface gives rise to a grossly inaccurate result since these approximations rely on the initial binary classification of the voxels.

It is also highly desirable to be able to view all the data within the volume simultaneously from selected stationary or rotating views; that is, to view into the center of a volume, and to detect objects and hidden surfaces within the volume (and therefore internal boundaries and surfaces). To do so, it is necessary to be able to see partially through interfering objects when desired (e.g., for bone surrounded by muscle, to be able to observe the bone as well as the muscle surrounding the bone). Prior art techniques for rendering volume elements force a binary decision to be made as to whether a pixel is made of a given material or not. A binary classification introduces aliasing (or sampling) errors as the continuous image function is not preserved. The error introduced by binary classification is introduced upon any attempt to reconstruct the original image volume from the classified volume. Because the reconstructed image can only have as many intensity levels as there are materials, material interfaces will be jagged and the intensities will not represent the original image function.

Further, it may be desirable to shade the volume data so that boundaries and surfaces may be determined more accurately. In addition, it may be desired to shade the surface of a projected image to give the impression that the image is illuminated by one or more sources of light. Such surface shading can also be used to emphasize and highlight selected areas of the displayed image volume.

SUMMARY OF THE INVENTION

The imaging system of the present invention provides apparatus and methods for projecting a two dimensional (2D) representation of three dimensional (3D) volumes where surface boundaries and objects internal to the volumes are readily shown, and hidden surfaces and the surface boundaries themselves are accurately rendered. Also, the two dimensional image produced is capable of having the same resolution as the sampling resolution of the input image volume of interest. This is accomplished through the implementation methods for determining "partial volumes" of voxels. Partial voluming provides for the assignment of selected colors and opacities to different materials (or data components) represented in an image data volume based on the percentage composition of materials represented in each voxel of the image volume. Unlike prior art systems, such as those that use thresholding techniques, the imaging of the present invention has a high degree of precision and definition that is independent of image volume on a per voxel basis.

An image volume representing a volume object or data structure is written into picture memory. A color and opacity is assigned to each voxel within the volume and stored as a red (R), green (G), blue (B), and opacity (A) component, three dimensional data volume. The RGBA assignment for each voxel is determined based on the percentage component composition of the materials represented in the volume, and thus, the percentage of color and transparency (based on 100% reference material color and transparency values) associated with those materials. Such voxels stored within the picture memory for the component channel volume are sometimes referred to herein as RGBA voxels.

Next, the voxels in the RGBA volume are used as mathematical "gels" or filters such that each successive voxel filter is overlayed over a prior background voxel filter. Through a linear interpolation, a new background filter is determined and generated. The interpolation is successively performed for all voxels (or groups of voxels, e.g., voxel scan lines) up to the front most voxel for the plane of view. The method is repeated until all display voxels are determined for the plane of view.

The present inventive method of reconstruction provides for the 2D projection of volumes where surfaces of objects within a discrete data volume can be detailed and displayed even where surfaces could not previously be found because image volume data is course or the sampling rate is low. The present invention provides for visualization of hidden surfaces within the volume such that all surface boundaries are clearly and accurately defined.

Thus, given a discrete sample data volume of a complex volume data set or object, and finding the percentage of composition of component materials for each voxel, it is possible to render complex images without initially being provided with specific information as to boundary location within each voxel. The present inventive method and apparatus thereby provides for viewing volumes without having a geometric model (but rather having only a 3D raster volume) such that it is possible to create a complex projection of the 3D volume, using partial transparency and color that can have the characteristics of a mathematical geometric solid model. Objects can be viewed which partially obscure other objects, and spatial relationships between objects can be accurately rendered.

The present inventive method of shading provides for the rendering of surfaces and boundaries to subvoxel accuracy. A three dimensional gradient at the center of each RGBA voxel is computed based on the opacity of neighboring voxels. The gradient volume consists of XYZL voxels, where the X,Y and Z components represent the change in opacity in each direction, and L is the length of the amplitude of the gradient. By multiplying each RGBA voxel by the length of its gradient, surfaces remain, but solid regions become more transparent.

In order to shade the surface of the displayed image volume, a source of light is chosen as being at a fixed point in space relative to the image volume. Light vectors L are defined extending from the light source to the center of each voxel in the image volume. A shading function is generated based on the angle of the gradient vector of each voxel and the light vector associated with each voxel. In the preferred embodiment of the present invention, the shading function includes user determined variables which allow for the addition of back lighting and side lighting to provide additional surface highlighting.

Other objects and advantages of the present invention will become more apparent upon a reading of the Detailed Description of the Invention in connection with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating the architecture of the present invention.

FIG. 2 is a block diagram illustrating the division of the picture memory of the present invention.

FIG. 3 is block diagram illustrating the architecture of the channel processor of the present invention.

FIG. 4a is a graph showing a typical input data histogram.

FIG. 4b is a graph illustrating classification of a CT input data histogram.

FIG. 5 illustrates a plurality of voxels used to generate a two dimensional display.

FIG. 6 illustrates image reconstruction for successive voxel scan lines and works.

FIG. 7 illustrates the calculation of projected pixel output based on the concatenated filtering of the present invention.

FIG. 8 illustrates the method of generating a gradient vector for a selective voxel in two-dimensional space.

FIG. 9 illustrates a shaded image section, shaded in accordance with the present invention.

FIG. 10 illustrates the generation of a gradient vector for a selected voxel in three dimensional space.

FIG. 11 illustrates the surface shading effect of the present invention.

FIG. 12 is a flow chart setting forth the steps in implementing the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Notation and Nomenclature

The detailed description which follows is presented largely in terms of algorithms and symbolic representations of operations on data bits within a computer memory. The algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art.

An algorithm is here, and generally, conceived to be a self-consistent sequence of steps leading to a desired result. These steps are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared and otherwise manipulated. It proves convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like. It should be kept in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities.

Further, the manipulations performed are often referred to in terms, (such as adding or comparing) which are commonly associated with mental operations performed by a human operator. No such capability of a human operator is necessary, or desirable in most cases, in any of the operations described herein which form art of the present invention; the operations are machine operations. Useful machines for performing the operations of the present invention include general purpose digital computers or other similar devices. In all cases the distinction between the method of operations and operating a computer, and the method of computation itself should be noted. The present invention relates to methods for operating a computer in processing electrical or other (e.g., mechanical, chemical) physical signals to generate other desired physical signals.

The present invention also relates to apparatus for performing these operations. This apparatus may be specially constructed for the required purposes or it may comprise a general purpose computer as selectively activated or reconfigured by a computer program stored in the computer. The algorithms presented herein are not inherently related to any particular computer or other apparatus. In particular, various general purpose machines may be used with the teachings herein, or it may prove more convenient to construct more specialized apparatus to perform the required method steps.

While voxels are for purposes of convenience sometimes referred to as three dimensional elements having a three dimensional volume, it should be appreciated that voxels define points in a three dimensional space.

In addition, the following description, numerous details are set forth such as algorithmic conventions, specific numbers of bits, etc., in order to provide a thorough understanding of the present invention. However it will be apparent to one skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known circuits and structures are not described in detail in order not to obscure the present invention unnecessarily.

System Architecture

The system architecture of the present invention is illustrated in FIG. 1. In the preferred embodiment, the image system comprises a host computer 10 coupled to a bit map terminal 12 and a system bus 14. The system bus 14 is coupled to a channel processor 16, a memory controller 18, and a video controller 20. The memory controller 18 is coupled to the channel processor 16 and video controller 20 and to a picture memory 26. The video controller 20 is coupled to the memory 26 via a video bus 28. Channel processor 16 is coupled to the picture memory 26 via a picture bus 24. A color monitor 22 or other ouput device is coupled to video controller 20.

The channel processor 16 comprises a parallel processing computer system. In the preferred embodiment of the present invention, four sixteen bit parallel processors are utilized in the channel processor. Although it has been found advantageous to utilize four parallel processors in the present invention, it will be obvious that any number of processors may be used without departing from the spirit and scope of the present invention.

The host computer 10 is coupled to a bit map terminal 12 which displays a plurality of menus available to the user of the present invention. In the preferred embodiment, the host computer 10 is utilized to manipulate the histogram generated from the image data stored in the picture memory 26. A look up table may be stored in the host computer 10 containing preassigned color and opacity values for each peak intensity location in a histogram. In addition, the host computer 10 may be utilized to display the histogram on bit map terminal 12 so that user defined color and opacity values may be determined.

In the present invention, the system bus 14 is a sixteen bit bus coupling the host computer 10 to the other components of the image processing system. Although sixteen bit parallel processors are utilized in the present invention, eight bit, thirty two bit, or any other size processor may be utilized as well. Thus, other bit size system buses may be utilized as required.

The video controller 20 provides vertical and horizontal synchronization to the color monitor 22 and also includes a buffer memory for displaying screen data. The video controller 20 accesses the video bus 28 during vertical blanking periods to provide updated information to the buffer memory. The video controller includes a digital to analog (d/a) converter to convert digital signals into a video signal suitable for display on the monitor 22. In addition, the controller 20 includes look up tables for each of the RGB channels. The twelve bit digital word of the red (R) channel, for example, is mapped out through a look up table and a unique red color value is outputted and displayed upon color monitor 22. As noted, there are look up tables for each of the channels R, G and B. The video controller 20 functions as a three value in--three value out converter, which allows gamma corrections. The video bus 28 in the preferred embodiment is a bus divisible in 12 bit units and operating at up to least 420 megabytes per second. The picture bus 24 in the preferred embodiment operates at 240 megabytes per second.

The color monitor 22 of the present invention may be any commercial color monitor having 480 to 1125 number of scan lines. The present invention has the speed and capability to service high resolution, high scan line monitors.

The picture memory 26 is a random access memory (RAM) utilized for storing the original image data as well as classification data, color and opacity data and the composite RGBA volume data. The picture memory 26 is shown in detail in FIG. 2. As previously noted, each voxel is represented by 4 twelve bit digital words. One portion of the memory, original volume memory 26a, stores the data in its original form. In the present preferred embodiment, approximately 40 million bits are dedicated for storage of original data. Original image volume data is stored as a series of twelve bit words where each word represents the intensity level of one voxel of the original voxel volume. After the RGBA volumes are generated, they are stored in RGBA volume memory 26b. Each color value is represented by a twelve bit word and the opacity value A is also represented by a twelve bit word. Thus, each voxel in the RGBA volume memory is represented by 4 twelve bit words. In the preferred embodiment of the the present invention, 150 millions bits of memory are dedicated to the RGBA volume memory 26b. The composite voxels are stored in the two-dimensional projection memory 26d. As noted previously, the combination voxels are generated from concatenation of various scan lines of the RGBA volume data. The picture memory 26 also includes temporary work space memory 26c, RGBA shaded volume memory 26e and XYZL gradient volume memory 26f. Although the number of bits in various portions of the picture memory 26 have been described, it will be obvious to one skilled in the art that any suitable number of bits of memory may be utilized in practicing the present invention.

Referring again to FIG. 1, memory controller 18 is utilized to arbitrate all accesses to the picture memory 26. Memory controller 18 enables data to be written into the picture memory 26 through means of picture bus 24.

Channel processor 16 is utilized to generate the histogram from which classifications of the voxels is made. Channel processor 16 is shown in detail in FIG. 3, and comprises scratch pad memory 17 and four multiplier/arithmetic logic units (ALU) 15a through 15d, respectively. In the preferred embodiment, scratch pad memory 17 includes 64K sixteen bit word memory locations. Channels processor 16 utilizes the scratch pad memory 17 for temporary storage of data for manipulation by the multiplier/ALU pairs. Each of the multiplier/ALU pairs 15a through 15d is dedicated to one of the four co-processors that comprise the channel processor. This parallel approach allows the same instruction to be executed on different data. Such a system is referred in the art as a single instruction, multiple data stream (SIMD) system. One such inventive system is more fully described in U.S. patent application Ser. No. 748,409, filed June 24, 1985, and owned by the assignee of the present invention. In the preferred embodiment, one channel is utilized for the R (Red) values of the voxels, one channel is utilized for the G (Green) values, one for B (Blue) and one for A (opacity) values.

The initialization of the processing system occurs when the original image volume data is read into the host computer 10. The original image data may be transmitted in a compressed form and require decoding and expansion. In addition, depending on the source of the data, the bits/pixel resolution of the raw data may not match the preferred twelve bit/pixel resolution of the present invention. Therefore, if required, the host computer or channel processor may artificially enhance the original image data to an acceptable resolution.

The host computer 10 then outputs a request signal on the system bus 14 requesting the channel processor 18 to allow the original image data to be written into the picture memory. The host computer 10 then outputs the original image data onto the system bus 14 into the channel processor 16. The channel processor 16 then outputs the original image data onto the picture bus 24 where it is written into the original image volume memory 26a of the picture memory 26.

If desired, or necessary the original image volume is then classified by the channel processor 16. A scan line from the original volume memory 26a is then placed on the picture bus 24 and inputted to the channel processor 16. The channel processor 16 counts the number of voxels at each intensity level of the 2,049 definable intensity levels to generate a histogram, the channel processor 16 may then classify the peaks pursuant to previously defined look up tables or a program hierarchy. Additionally, the histogram may be displayed by the host computer 10 on the bit map terminal 12 so that the classification may be manually performed by a user.

After classification, the classification data is sent through system bus 14, to the channel processor 16, and in particular to the scratch pad memory 17. The channel processor computes a plurality of lookup tables based on classification data from the host. The scratch pad memory 17 contains a plurality of look up tables so that RGB and opacity values may be assigned to the classified voxels. The channel processor 16 requests one scan line (a one dimensional array of voxels) at a time from the memory controller and processes the scan line through the stored look up tables. The input to the channel processor look up tables is a monochrome scan line and the output is an RGBA classified scan line. The scratch pad memory 17 of the channel processor 16 includes at least three buffers, namely a scan line buffer, a look up table, and an output buffer for output classification. Although described in conjunction with look up tables, any suitable method of assigning or generating color values may be utilized in the present invention.

The output from the output buffer is coupled through a picture bus 24 to the RGBA volume memory 26b of picture memory 26. Each slice of the original image volume is stored as an array in the RGBA volume memory 26b. The "concatenated filtering" of the present invention is performed by the channel processor 16. A background image volume is initially read into the scratch pad memory of the channel processor one scan line at a time. As each succeeding image volume is read into the channel processor 16, the RGBA values are concatenated by the multiplier/ALU Units 15a-d respectively. This composite image volume is stored in the projection memory 26d of picture memory 26.

To display the composite image, the contents of memory 26d is outputted onto the video bus 28 into the video controller 20. The digital data stored in the projection memory 26d is converted to video analog data by the controller 20 and outputted to color monitor 22 where it is displayed in a raster scanning fashion.

Partial Volumes

Classification

An image volume, that is, a volume of picture elements (voxels) representing a three dimensional image, is read into host computer 10 and, after any required decoding or decompression, into original image memory 26b of picture memory 26. Thus, the object(s) under consideration, an array of two dimensional arrays, can be thought of and mathematically represented as as a three dimensional ordered array in picture memory. The image volume may be an unprocessed image array that may be obtained by various methods known in the art.

While the present invention has wide application, it is described for purposes of example in connection with the three dimensional display of computed tomographic ("CT") images. Input image volume data, represented as an ordered CT image volume array in this context, may be obtained from a variety of tomographic imaging methods, e.g., x-ray computed tomography, ultrasound sector scanning, nuclear magnetic resonance, etc. In other contexts, the image volume input data may be obtained using other imaging methods, such as seismic imaging, or the result of computer model or simulation, (such as a fluid flow simulator), for example. The image volume in the present example is stored as an ordered array of 2D images where each image is a 2D ordered array of 12 bit numbers.

The CT scan or other image volume provides monochromatic grey scale input data to the image processing system of the present invention. This CT input image volume data is stored in the initial volume memory 26(a) of picture memory 26 as an ordered array of 12 bit binary numbers, each representing a given CT scan or other image volume data (voxel). In the present example, the image volume information which is provided by the CT scan input represents information about four discrete materials in the anatomy under consideration; namely, air, fat, soft tissue, and bone.

In the present example, the intensity of the grey scale data depends on the x-ray density of the material represented from the original imaging method. Referring to FIG. 4(a), grey scale intensity data for each voxel is plotted in a histogram 30 that provides a distribution of the number of voxels in the image volume versus intensity. The histogram 30 is generated by the channel processor 16 from image volume input data and is transmitted to the host computer 10 via system bus 14.

The grey scale level histogram 30 is a function which shows, for each grey level, the number of voxels in the image volume that have a particular grey level. The abscissa 32 is grey level intensity, shown in this case to be from 0 to 2048. The ordinate 34 is frequency of occurrences (e.g., number of voxels of the image volume at each intensity). Thus, the area under the histogram curve 36 represents the total number of voxels in the image volume.

The histogram 30 is derived from a particular image which, in the present example, corresponds to the twelve-bit pixel information of the CT scan. The resulting histogram is then filtered by the host computer 10 to remove noise, (i.e., to supress high-frequency variations while preserving the shape of the input function), using low pass filtering techniques widely known in the art and results in a smoothed histogram curve as shown in FIG. 4a.

In the prior art imaging systems, all voxels are classified as representing 100% of single material. Thus, using prior art techniques, a binary classification is made for each voxel, in which all voxels within a particular intensity region are classified as one or the other of those materials represented by the input volume image data. Using prior art techniques for the present CT scan example, voxels would be classified as either bone, fat, tissue or air. In the prior art, a threshold grey level is chosen as a cutoff at some point between histogram peaks. All voxels within a given intensity range are thereby categorized either 100% air, soft tissue, bone or fat. This information is then stored in memory as a two bit or one bit value.

For CT renderings, this binary classification is suitable for peak regions of the histogram where a voxel content clearly falls within one discrete material classification or another. However, such "hard" classification requires that a discrete binary determination be made concerning the material classification of all voxels within the image volume.

For example, in the prior art, at surface boundaries (such as at an interface where tissue is attached to bone), a voxel crossing an edge location is classified as either bone or tissue. Thus, where voxels cross between surfaces or along edges, or where the local material is less than 1 voxel wide, the edge or surface boundary will be lost, and an imprecise rendering will result. Such binary value assignment, therefore, introduces significant aliasing error, resulting in a surface rendering that is inaccurate.

In the present invention, voxels are classified, in accordance with their associated intensities, as being composed of between 0% and 100% of each material type represented in the image volume.

Referring to FIG. 4b, significant peaks P and troughs T are identified in the histogram 30. An intensity range 40 characterized as 100% of a material is determined for all voxels falling within a selected intensity range to the left and right of significant histogram peaks P. In the present embodiment for a CT scan, this range is determined by a linear approximation. The 100% material range is approximated as all voxels P falling within the intensity band defined as .vertline.p-t.vertline./4 where p is the intensity associated with a significant peak P and t is an intensity associated with an adjacent significant trough T. All voxels within this range on either side of a given significant peak P are classified as being a 100% "pure" material value. Thus, in the present example, all voxels within these ranges are categorized as 100% fat, air, soft tissue, and bone, respectively. Voxels outside these intensity range values are categorized as containing image information representing some percentage of the "pure" materials to the left and to the right of the voxels. Therefore, they are combination material voxels each having a partial percentage composition of these materials.

For convention, voxels classified as being composed of less than 100% of any single material are referred to as partial volumes in connection with the present invention. Further, voxels classified as 100% of one material are sometimes referred to as such. However, in more general terms, utilizing the present invention, classification is based on partial volume determination. Thus, as a more general convention, all classified voxels are "partial volumes", where voxels classified as a single homogenous material are "partial volumes" having 100% of one material, and 0% composition of other component materials.

In determining partial volume classification, the percentage of each material contained in voxels falling outside the 100% material intensity ranges is found. In the present preferred embodiment, the percentage voxel material content is determined by performing a linear interpolation for voxels having intensities falling between the limits of adjacent 100% material intensity values. A ratio is then found of: (1) the intensity differences from a 100% value boundary to the voxel intensity under consideration (numerator) versus (2) the intensity range value between surrounding 100% material value limits (denominator). This ratio provides the percentage voxel composition for each adjacent material at a particular intensity value between 100% value ranges.

Referring now to FIG. 4b, partial volume percentages are determined by the channel processor calculation performed for all partial volume voxels by a linear interpolation for each intensity between adjacent 100% material intensity value limits. Alternatively, classification may be made pursuant to values located in a look-up table or made pursuant to inspection and user determined values. In the present preferred embodiment, for a given voxel associated with partial volume intensity I(pv), its percentage composition is: ##EQU1## where PVab is the partial volume voxel(s) under consideration; I(PV)ab=intensity of partial volume voxels under consideration between 100% material intensity regions a and b; I(Pb)a=intensity at peak boundary value a for 100% material type; and I(Pb)b=intensity at boundary value b for 100% material type;

When calculating the fraction of material associated with boundary a, I(Pb)a is used; with boundary b, I(Pb)b is used. Thus, each voxel is classified as either; (1) a partial volume made up of a percentage of more than one image volume material type, or (2) a "pure" volume made up of 100% of one image volume material type.

In the present preferred embodiment, the classification of grey scale data is performed for a musculo-skeletal CT scan model. In this model, as shown in FIG. 4b, the relationship of material grey scale intensities is such that fat 40 is always adjacent to soft tissue 44; soft tissue 44 always surrounds fat 40 and also always surrounds bone 46. It should be appreciated, however, that other classification schemes can be used to determine percentage voxel material composition for other input data, such as for example, seismic image volume input data. Also, such classification may be used for applications where input data includes material adjacencies in which multiple materials (and/or their associated histogram intensities) may be concurrently adjacent to each other and are so represented by image volume input data. Further, although straight line interpolat