WikiPatents - Community Patent Review
Create Free Account  |  License or Sell Your Patent  |  WikiPatents Marketplace  |  WikiPatents Blog
Username:  Password:  
    
Advanced Search
Apparatus and method for photogrammetric surgical localization    

Get related patents on CD
United States Patent5389101   
Link to this pagehttp://www.wikipatents.com/5389101.html
Inventor(s)Heilbrun; Mark P. (Salt Lake City, UT); McDonald; Paul (Salt Lake City, UT); Wiker; J. Clayton (Salt Lake City, UT); Koehler; Spencer (Salt Lake City, UT); Peters; William (Salt Lake City, UT)
AbstractA method and apparatus for defining the location of a medical instrument relative to features of a medical workspace including a patient's body region are described. Pairs of two-dimensional images are obtained, preferably by means of two video cameras making images of the workspace along different sightlines which intersect. A fiducial structure is positioned in the workspace for defining a three dimensional coordinate framework, and a calibration image pair is made. The calibration image pair comprises two 2D projections from different locations of the fiducial structure. After the calibration image pair is made, the fiducial structure is removed. A standard projection algorithm is used to reconstruct the 3D framework of the fiducial structure from the calibration image pair. Appropriate image pairs can then be used to locate and track any other feature such as a medical instrument, in the workspace, so long as the cameras remain fixed in their positions relative to the workspace. The computations are desirably performed with a computer workstation including computer graphics capability, image processing capability, and providing a real-time display of the workspace as imaged by the video cameras. Also, the 3D framework of the workspace can be aligned with the 3D framework of any selected volume scan, such as MRI, CT, or PET, so that the instrument can be localized and guided to a chosen feature. No guidance arc or other apparatus need be affixed to the patient to accomplish the tracking and guiding operations.
   














 Title Information Submit all comments and votes
 
Patent Text Patent PDF Print Page Summary File History
Plain text PDF images Print Summary File History Custom Search
Drawing from US Patent 5389101
Apparatus and method for photogrammetric surgical localization - US Patent 5389101 Drawing
Apparatus and method for photogrammetric surgical localization
Inventor     Heilbrun; Mark P. (Salt Lake City, UT); McDonald; Paul (Salt Lake City, UT); Wiker; J. Clayton (Salt Lake City, UT); Koehler; Spencer (Salt Lake City, UT); Peters; William (Salt Lake City, UT)
Owner/Assignee     University of Utah (Salt Lake City, UT)
Patent assignment
All assignments
Company News
Publication Date     February 14, 1995
Application Number     07/871,382
PAIR File History     Application Data   Transaction History
Image File Wrapper   Patent Term   Fees
Litigation
Filing Date     April 21, 1992
US Classification    
Int'l Classification    
Examiner     Cohen; Lee S.
Assistant Examiner     Gilbert; Samuel
Attorney/Law Firm     Trask, Britt & Rossa
Address
Parent Case    
Priority Data    
USPTO Field of Search    
Patent Tags     photogrammetric surgical localization
   
Enter a comma (,) or semicolon (;) between multiple tag words/phrases.
Describe this patent:
 Amusing   
 Clever   
 Complex   
 Efficient   
 Historic   
 Important   
 Innovative   
 Interesting   
 Practical   
 Simple   
[no votes]
Patent WIKI

Share information and news about this patent, including information and news about the technology, inventors, company, ligation and licensing.

 References Submit all comments and votes
 
*references marked with an asterisk below are user-added references
 U.S. References
 
Add a new US reference:  
ReferenceRelevancyCommentsReferenceRelevancyComments
5178146
Giese
600/411
Jan,1993

[0 after 0 votes]
5107839
Houdek

Apr,1992

[0 after 0 votes]
5005578
Greer
600/414
Apr,1991

[0 after 0 votes]
4945914
Allen
600/426
Aug,1990

[0 after 0 votes]
4884566
Mountz
606/130
Dec,1989

[0 after 0 votes]
4638798
Shelden
606/130
Jan,1987

[0 after 0 votes]
4618978
Cosman
378/164
Oct,1986

[0 after 0 votes]
4583538
Onik
606/130
Apr,1986

[0 after 0 votes]
4551678
Morgan
324/300
Nov,1985

[0 after 0 votes]
 Foreign References
 Other References
 Market Review Submit all comments and votes
   
Market Size
Estimate the gross annual revenues of the relevant market sector:
> $10B
$5B - $10B
$2B - $5B
$500M - $2B
$100M - $500M
$10M - $100M
$1M - $10M
$500K - $1M
$100K - $500K
< $100K
[No votes]
$0
 
$0   $2.5B   $5B   $7.5B   $10B

[0 market size comments]
Market Share
Estimate the percentage of the relevant market sector this invention will capture:
75% - 100%
50% - 74.99%
25% - 49.99%
10 - 24.99%
5 - 9.99%
2 - 4.99%
1 - 1.99%
< 1%
[No votes]
0.0%
 
0%   25%   50%   75%   100%

[0 market share comments]
Reasonable Royalty
What percentage of gross sales should the inventor or assignee be paid?
75% - 100%
50% - 74.99%
25% - 49.99%
10 - 24.99%
5 - 9.99%
2 - 4.99%
1 - 1.99%
< 1%
[No votes]
0.0%
 
0%   25%   50%   75%   100%

[0 reasonable royalty comments]
Public's "Guesstimation" of Royalty Value
Market SizeN/A[No votes]
xMarket ShareN/A[No votes]
xReasonable RoyaltyN/A[No votes]

N/A

[0 Guesstimation of Royalty Value Comments]
License Availablity
If you are NOT the owner or assignee, answer here:
Yes, license is available for purchase

No, license is not currently available



[No votes]
[0 license availability comments]
License Availablity
If you ARE the owner or assignee, answer here:
Yes, license is available for purchase

No, license is not currently available



[No votes]
[0 owner/assignee comments]
Competitive Advantage
Does this invention have a significant competitive advantage over similar technologies?
Yes

No



[No votes]
Most helpful competitive advantage comment
[No comments]

[0 competitive advantage comments]
Commercial Alternatives
Are there viable commercial alternatives for this invention?
Yes

No



[No votes]
Most helpful commercial alternative comment
[No comments]

[0 commercial alternatives comments]
 Technical Review Submit all comments and votes
 Claims Submit all comments and votes
 


What is claimed is: PG,27

1. Apparatus for defining a location of a medical instrument relative to features of a medical workspace including a patient's body region, comprising:

workspace imaging means positionable for producing a plurality of pairs of images of the medical workspace, each of said image pairs comprising two images made along one of each of a different one of two sightlines, said sightlines intersecting at an angle;

digitizing means operably disposed for digitizing each of said images of said image pairs to produce sets of digital image signals, one said set of digital image signals corresponding to each of said images;

fiducial means removably positionable in said workspace for providing a series of fiducial points held in fixed spatial relation to one another; and

computing means connected to said digitizing means to receive said digital image signals therefrom, said computing means being operable to:

establish a workspace coordinate framework in three dimensions from a first selected one of said pairs of images made when said fiducial means is positioned within said workspace,

determine workspace coordinates in said workspace coordinate framework of any selected point which can be identified from both images of said first selected image pair,

receive a set of scan coordinates for each of three or more selected landmarks present in a scan made in a scan coordinate framework, and correlate said scan coordinates with the workspace coordinates of said landmarks as derived from one of said image pairs, said landmarks being selected from the group of anatomic features, surgical implants, and radiologic implants,

compute conversion functions for converting the scan coordinates of a selected feature in said scan to corresponding workspace coordinates in said workspace framework, and for converting observed workspace coordinates of a selected feature observable in both of said images of a selected one of said image pairs to corresponding scan coordinates in said scan coordinate framework, and

perform said conversion functions for user-selected features found in said scan or in a subsequent selected one of said image pairs.

2. The apparatus of claim 1, wherein said computing means includes memory means for storing data including a plurality of instrument dimensions, each of said instrument dimensions comprising a known distance between a fiducial mark on an instrument and a preselected operative portion of said instrument, and said plurality including instrument dimensions for a plurality of instruments, said plurality including surgical instruments, pointers, and electrodes, and wherein said computing means is further operable to compute either extrapolated workspace coordinates or extrapolated scan coordinates of said operative portion of said instrument from said instrument dimensions and the workspace coordinates of said instrument in said image pair.

3. The apparatus of claim 2, wherein said computing means further includes computer graphics means operable to compute a plurality of sets of display signals, each said set representative of at least one of said images of said workspace or of a volume scan, and said apparatus further includes display means communicatively connected to receive said display signals from said computing means, for displaying an image encoded by a selected one of said sets of display signals.

4. The apparatus of claim 3, further including at least one instrument having a shank and an operative portion, wherein two fiducial marks are provided on said shank, and wherein at least one distance which is the distance between one of said fiducial marks and said operative portion is stored in said memory means of said computing means.

5. The apparatus of claim 1, wherein said workspace imaging means includes one or more x-ray imaging devices, and said fiducial points of said fiducial means are formed of radio-opaque material.

6. The apparatus of claim 1, wherein said workspace imaging means comprises one or more video cameras.

7. The apparatus of claim 1, wherein said workspace imaging means comprises two video cameras.

8. The apparatus of claim 1, wherein said fiducial means includes at least six said fiducial points distributed in at least two distinct planes.

9. The apparatus of claim 8, wherein said fiducial means comprises at least four rods interconnected to form a cage-like array, each of said rods having first and second ends, and said rods disposed with said first ends in a first plane and said second ends in a second plane which is parallel said first plane, and wherein said fiducial points include at least some of said first and second ends.

10. The apparatus of claim 8, further including at least one additional fiducial point, and wherein said computing means is further operable to compute from said six fiducial points expected workspace coordinates of said at least one additional fiducial point, to determine actual workspace coordinates of said at least one additional fiducial point from said digitized signals, and to compare said expected workspace coordinates to said actual workspace coordinates.

11. The apparatus of claim 1, wherein said angle is between about 5 degrees and 175 degrees.

12. The apparatus of claim 1, wherein said computing means includes a memory component, and said memory component is constructed to contain fiducial reference data for a preset minimum number of said fiducial points of said fiducial means, said fiducial reference data comprising the physical coordinates of each individual said fiducial point.

13. A fiducial structure for establishing a three-dimensional coordinate framework from a photographic image pair, comprising a support frame and a set of fiducial indicators connected and supported by said support frame, said support frame comprising a plurality of rod-like elements interconnected to form an open cage, wherein said fiducial indicators are formed as a series of eight objects interconnected by four of said rod-like elements, with individual said objects disposed respectively near a first end of each of said four rod-like elements and near a second end of each of said four rod-like elements.

14. A method of localizing a feature of interest in a workspace, comprising the steps of:

providing a fiducial structure comprising

a sufficient number of fiducial indicators arranged to define a three-dimensional coordinate system and having known 3D coordinates, and

support frame means for supporting and connecting said fiducial indicators, said support frame means constructed to hold said fiducial indicators in fixed relation to each other;

making a calibration image pair of the workspace comprising two 2D images of the workspace made along different sightlines, and with the fiducial structure positioned in the workspace such that a preset minimum number of fiducial points are visible in both images of the calibration image pair;

digitizing respective 2D images of the calibration image pair;

determining a first plurality of 2D coordinates from one member of the calibration image pair and a second plurality of 2D coordinates from the other member of the calibration image pair;

computing a 3D workspace coordinate framework from said first and second pluralities of 2D coordinates and the known 3D coordinates of the fiducial structure;

removing the fiducial structure from the workspace;

making a subsequent image pair comprising two 2D images of the workspace having a patient's body region therein, the two 2D images being made along the same respective sightlines as the calibration image pair;

digitizing the respective 2D images of the subsequent image pair; and

determining workspace 3D coordinates of a feature of interest in the workspace from the respective 2D images of the subsequent image pair.

15. The method of claim 14, further including the step of repeating said steps of making a subsequent image pair and determining the workspace 3D coordinates of the feature of interest, until a medical procedure is complete.

16. The method of claim 14, wherein the preset minimum number of fiducial points is six.

17. The method of claim 14, further including the steps of:

providing volume scan data reflective of a volume scan having a defined volume scan 3D framework including scan 3D coordinates for selected features;

aligning the volume scan 3D framework with the workspace 3D framework to produce a mapping function for mapping the scan 3D coordinates of a selected point to corresponding workspace 3D coordinates;

selecting a target in the volume scan and determining target workspace 3D coordinates for the target using the mapping function; and

comparing the target workspace 3D coordinates with the workspace 3D coordinates of the feature of interest.

18. The method of claim 15, wherein said step of repeating is performed for one or more different selected features of interest and one or more different targets selected from the volume scan.

19. The method of claim 17, wherein the feature of interest is a medical instrument having an operative portion, and further including the steps of computing an instrument vector representing a linear direction of the instrument, computing coordinates representative of a position of the operative portion of the instrument, and extrapolating the instrument vector from the coordinates of the operative portion for a distance sufficient to determine whether the instrument vector will intersect the target.

20. The method of claim 19, wherein the calibration and subsequent image pairs are made by video imaging means.

21. The method of claim 20, further including a step of providing a visual display of the workspace comprising one image of the subsequent image pair, or a stereo image derived from both images of the subsequent image pair.

22. The method of claim 21, further including a step of overlaying an icon representing a target feature on the display of the workspace.

23. The method of claim 17, wherein the calibration and subsequent image pairs are made by video imaging means.

24. The method of claim 23, further including a step of overlaying an icon representing the target feature on the display of the workspace.

25. The method of claim 23, further including a step of providing a visual display of the workspace comprising one image of the subsequent image pair, or a stereo image derived from both images of the subsequent image pair.

26. The method of claim 17, further including a step of displaying the target and the feature of interest according to their respective workspace coordinates and in a visual context comprising an image of the workspace presented in terms of said 3D workspace coordinate framework.

27. Apparatus for localizing a medical instrument relative to features of a patient's body region, comprising:

a fiducial structure removably positionable in a medical workspace, and comprising

a sufficient number of fiducial indicators arranged to define a three-dimensional coordinate system and having known 3D coordinates, and

support frame means for supporting and connecting said fiducial indicators, said support frame means constructed to hold said fiducial indicators in fixed relation to each other;

workspace imaging means positionable for producing a plurality of pairs of images of said medical workspace, each of said image pairs comprising two images made along one of each of a different one of two sightlines, said sightlines intersecting at an angle;

digitizing means operably disposed for digitizing each of said images of said image pairs to produce sets of digital image signals, one said set of digital image signals corresponding to each of said images; and

computing means connected to said digitizing means to receive said digital image signals therefrom, said computing means being operable to:

determine a first plurality of 2D coordinates from one member of a calibration image pair and a second plurality of 2D coordinates from an other member of said calibration image pair, said calibration image pair being an image pair made with said fiducial structure positioned in said medical workspace,

compute a 3D workspace coordinate framework from measured physical 3D coordinates of said fiducial structure in combination with said first and second pluralities of 2D coordinates,

compute an algorithm for converting any 2D coordinates derived from a subsequent image pair made after said fiducial structure is removed from said medical workspace, to corresponding 3D workspace coordinates,

determine a set of 2D image coordinates reflective of a current position of a feature of interest present in both images of said subsequent image pair, and

convert said set of 2D image coordinates to a set of corresponding workspace 3D coordinates.

28. The apparatus of claim 29, wherein said feature of interest is a medical instrument having an operative portion, and said computing means is further operable to compute an instrument vector representing a linear direction of said instrument, to compute coordinates representative of a present position of said operative portion of said instrument, and to compute extrapolated vector coordinates from said coordinates of said operative portion and said instrument vector at a distance from said operative portion sufficient to determine whether said instrument vector intersects said target.

29. The apparatus of claim 28, wherein said computing means is further operable to:

receive volume scan data reflective of a volume scan having a defined volume scan 3D framework;

correlate said volume scan 3D framework with said workspace 3D framework to produce a function for mapping scan 3D coordinates of a selected point to selected point workspace 3D coordinates;

determine target workspace 3D coordinates for a user-selected target by applying said mapping function to said scan 3D coordinates of said selected point; and

compare said target feature workspace coordinates and said selected point workspace coordinates.

30. The apparatus of claim 28, further including display means operably interconnected with said computing means and said workspace imaging means for providing a visual display of said medical workspace, said visual display comprising one of said images of said subsequent image pair, a stereo image derived from both images of said subsequent image pair, or an image derived from said volume scan data.
 Description Submit all comments and votes
 


BACKGROUND OF THE INVENTION

1. Field

The application is related to techniques for mapping internal structures in the body of an animal or human, and more particularly to such technique for localizing a medical instrument with respect to anatomical features or the like during surgical or other medical procedures.

2. State of the Art

Various scanning apparatus and methods are known for imaging and mapping body structures, which provide target location data for surgical and other medical procedures. One group of methods, including still photography, videography, radiological x-rays, and angiography, typically produces only a two-dimensional projection of a three-dimensional object. For purposes of this application, this first group will be termed "two-dimensional" or "2D" imaging.

A second group of methods, of which computerized tomographic (CT) scanning, positron emission tomography (PET) scans, and magnetic resonance (MRI) imaging are exemplary, provide three-dimensional (abbrev. "3D" herein) information about internal structures (i.e., structures not visible from the exterior of the patient). The three-dimensional information about the internal volume is reconstructed from multiple scans of a known thickness (generally about a millimeter) made along parallel planes displaced from each other by a known distance, usually of the order of millimeters. An example of such a reconstructed volume image is depicted in FIG. 1A, including the contours of a selected anatomical feature within the brain. In this application, methods in this second group will be referred to as "volume" scanning or imaging.

In performing resection or other surgical manipulations, it is highly desirable to correlate the location of instruments, patient anatomical features, or other elements or structures placed in the surgical field, and generally as seen by the surgeon, with the location of internal targets or features as visualized by one of the volume scanning techniques. Such a correlation process is often termed "localization".

A commercially available device for localization in neurosurgery is the Brown-Roberts-Wells (abbrev. BRW) localizer (U.S. Pat. Nos. 4,341,220, and 4,608,977). The BRW system includes a large ring-like structure which surrounds the patient's head and is fixed in place. The ring establishes a 3D coordinate system with respect to the patient's head. A separate calibration unit having an array of rod elements is fixed to the ring to surround the head during the production of volume scan and/or 2D images. The rod elements have known coordinates in the 3D coordinate system established by the ring, and produce spots in the volume scans. Other features in the volume scans can then be assigned coordinates in the 3D coordinate system established by the ring, by correlation with the known coordinates of the rod elements producing the spots.

After the images are made, the calibration unit is detached from the ring, and a guidance arc calibrated to the 3D coordinate system of the ring is attached in its place. The guidance arc provides coordinate reference information which may be used to guide a medical instrument. The medical instrument is usually attached to the guidance arc.

The BRW system has several disadvantages. The ring is cumbersome and uncomfortable for the patient, but it must be affixed in place when the volume and/or 2D scans are made, and kept there until the medical procedure is complete. It is possible to remove the ring after the scans are made, but precise repositioning is critical to avoid error in localization. Accurate repositioning is difficult, so present practice generally is to keep the ring in place until after the surgery. When not attached to the guidance arc, the position of a medical instrument in terms of the 3D coordinate system of the ring, and therefore in respect to the features identifiable in the volume or 2D scan, is not accurately known.

U.S. Pat. No. 4,618,978 to Cosman discloses a localizer device for use with a BRW-type system, including an open box composed of connected rods, which surrounds the patient's head and constitutes a calibration unit.

Alternatively, cranial implants of radio-opaque or MRI-opaque materials can be made. Generally, a minimum of three implants are required for establishing a three-dimensional space in volume scans. At present this method is considered very undesirable, in part because of the risk of infection or other complications of the implants.

Accordingly, a need remains for rapid, reliable, and inexpensive means for localizing a medical instrument relative to points of interest including both visible anatomical features and internal features imaged by volume and/or 2D methods. A need further remains for such means which does not require the physical attachment of a reference unit such as the BRW ring to the patient. Highly desirably, such means would be useful to track the position of a medical instrument in real time, and without requiring that the instrument be physically attached to a reference guide.

OTHER TERMS AND DEFINITIONS

A coordinate system may be thought of as a way to assign a unique set of numerical identifiers to each point or object in a selected space. The Cartesian coordinate system is one of the best known and will be used in this paragraph by way of example. In the Cartesian coordinate system, three directions x, y, z are specified, each corresponding to one of the three dimensions of what is commonly termed 3D (three-dimensional) space (FIG. 1B). In the Cartesian system, any point can be identified by a set of three values x, y, z. The x, y and z directions can be said to establish a "three-dimensional framework" or "coordinate framework" in space. A selected point "A" can be described in terms of its values x.sub.a, y.sub.a, z.sub.a ; these values specify only the location of point A. A different point B will have a different set of values x.sub.b, y.sub.b, z.sub.b. Such a set of values x,y,z for any selected point is referred to herein as the "coordinates" or "locational coordinates" of that point. When the position of a feature larger than a single point is being described, these terms are also understood to refer to a plurality of sets of x,y,z values. Other types of coordinate systems are known, for example spherical coordinate systems, and the terms "coordinates" and "locational coordinates" should further be understood to apply to any set of values required to uniquely specify a point in space in a given coordinate system.

The term "fiducial" is used herein as generally understood in engineering or surveying, to describe a point or marking, or a line, which is sufficiently precisely defined to serve as a standard or basis reference for other measurements.

SUMMARY OF THE INVENTION

The invention comprises apparatus and a method for defining the location of a medical instrument relative to elements in a medical workspace including a patient's body region, especially (but not limited to) elements seen by the surgeon. The apparatus develops a calibrated 3 dimensional framework of the workspace from a pair of 2D images made from different fixed locations, and aligns the workspace framework with a 3D scan framework defined by a volume scan. A pair of video cameras is the present preferred imaging means for obtaining the 2D image pairs. The apparatus is then operable to locate and track the position of a medical instrument during a medical procedure, with respect to features observable in either the workspace images or in the volume scan. A pictural display of such location and tracking information is provided to aid a medical practitioner performing the procedure.

The apparatus may be described as follows. Workspace imaging means are provided and positioned for producing a plurality of pairs of 2-dimensional images of a medical workspace. Each image pair comprises two such images made in effect simultaneously along respective different sightlines which intersect at an angle. Digitizing means are operably disposed for digitizing each image to produce corresponding sets of digital output signals, one set for each image.

Calibration means are removably positionable in the workspace for calibrating the workspace in terms of a three-dimensional coordinate framework. The 3D workspace framework is derived by computation from the two 2D projections of an image pair made with the calibration means positioned in the workspace. The calibration means comprises at least six fiducial points held in fixed spatial relation to each other, and having known spatial parameters which define an arbitrary Cartesian 3-dimensional coordinate system. These spatial parameters include 3D location coordinates of each of the fiducial points. Optionally but desirably, at least some of the actual distances between fiducial points should be known, to calibrate the workspace in terms of a suitable distance unit such as millimeters.

A computing means is connected to receive the digital output signals reflective of the images. The computing means also has data input means for receiving scan data from a volume scan of the patient's body region. The scan data define a scan 3D coordinate framework and internal anatomical structures therein. The computing means is further constructed or programmed to perform the following steps: 1) establish a workspace coordinate framework in three dimensions from an image pair made with said fiducial structure positioned within the workspace; 2) determine the locational coordinates in the workspace framework of any selected point which can be identified from both images of said pair; 3) correlate the scan locational coordinates for each of three or more selected landmarks observable in the scan with the workspace locational coordinates of the same landmarks as derived from a video image pairs; 4) use the correlation of the workspace coordinates and the scan coordinates of the landmarks, to derive a transformation algorithm for mapping selected other features from either the scan framework to the workspace framework, or the converse; and 5) provide display signals encoding a display reflective of one or both of the workspace images and/or a volume scan, as selected by a user. Display means are provided for displaying the images encoded by the display signals.

Optionally but highly desirably, the computing means has computer graphics capability for producing graphic icons overlaid upon the displayed images. Such icons include a cursor which the user employs to select features in the displayed images for computation of their coordinates or other operations.

A method of surgical guidance may be described as follows. First, a fiducial structure having six or more fiducial points defining two distinct, non-orthogonal planes is positioned in a medical workspace. Workspace imaging means are disposed for making pairs of two-dimensional images of the workspace in which the two member images are made along different but intersecting sightlines. A calibration image pair comprising images of the workspace with the fiducial structure is made. The fiducial structure is removed from the workspace.

A projection algorithm is applied to reconstruct a workspace 3D coordinate framework from the calibration image pair. At least one additional 3D scan framework is obtained from a corresponding volume scan of the patient's body region. At least three landmarks identifiable in both the volume scan and the workspace image pair are selected, and the coordinates for the three landmarks are determined in both the workspace framework and the scan framework. From these determined coordinates, a process is developed for aligning the scan framework with the workspace framework, and transformation algorithms for converting coordinates from one of the frameworks to the other are computed.

A target of interest in the volume scan is identified, and its scan coordinates are determined and converted to workspace coordinates. A feature of interest in the workspace, such as a fiducial mark on a scalpel, is identified. The workspace coordinates of the fiducial mark and of the scalpel tip (whose distance from the fiducial mark is known), plus a vector describing the direction of the scalpel, are determined. Optionally but highly desirably, both the target and the scalpel including the scalpel tip position are displayed in an image of the workspace. The path of the scalpel tip is extrapolated along the vector for a distance sufficient to determine whether the tip will reach the target on this path. If not, the direction of the scalpel is adjusted and the process of localizing the tip and extrapolating its path is repeated until the extrapolated path is deemed adequate by a user, and/or until the medical procedure is complete.

BRIEF DESCRIPTION OF THE DRAWINGS

In the figures, which illustrate what is presently regarded as the best mode for carrying out the invention, like reference numbers indicate like elements of the apparatus:

FIG. 1 depicts a volume scan and a coordinate system;

FIG. 2 is a block diagram depicting the basic elements of a video localization system of the invention;

FIG. 3 depicts an embodiment of the fiducial structure in greater detail;

FIG. 4 depicts a pair of images made from different positions of a surgical workspace including a patient's head, with the fiducial structure of the invention positioned for calibrating the workspace.

DETAILED DESCRIPTION OF THE ILLUSTRATED EMBODIMENT

FIG. 2 is a block diagram depicting the basic elements of a working embodiment of a video localization system of the invention. A pair of video cameras 200, 202 are positioned for making a pair of images along respective sightlines 204, 206, of a medical workspace 208 which includes a patient's body region here shown to be the patient's head 210. Cameras 200, 202 are arranged to have an angle 212 between sightlines 204, 206, such that both cameras image the workspace 208. Workspace 208 is effectively defined by the overlapping fields of view of the respective images made by cameras 200, 202. Angle 212 is preferably between about 30.degree. and 150.degree. . However, any angle greater than zero degrees and not equal to 180.degree. can be used.

Alternatively, cameras 200, 202 may be replaced by a single camera which is moved back and forth between first and second positions to take images along respective sightlines 204, 206. In the latter case, it is important that the camera be precisely positioned in the first and second positions when making the respective images of an image pair. Positioning means may be provided for establishing fixed attachment points for attachment of a camera, to facilitate such repositioning. Whether one camera or two cameras are used, what is significant is that the system takes pairs of images of workspace 208, each member of an image pair made along different sightlines intersecting in workspace 208.

A fiducial structure 220 (described in greater detail with reference to FIG. 3) is shown positioned in the workspace 208 proximal to the head 210. During use, fiducial structure 220 can be held in position by any suitable support means (not shown). One suitable support means would be a bar with a clamp arm attached to a ring stand or the like. Notably, fiducial structure 220 is neither affixed to, nor in contact with, the patient's head 210. Fiducial structure 220 may be removed from workspace 208 when it is not required.

Cameras 200, 202 are communicatively connected to image digitizing means 230, which produces two sets of digitized image signals, each representative of a respective image detected by one of the two cameras. Digitizing means 230 is in turn connected to send the digitized image signals to computing means 232.

Computing means 232 receives the digitized image signals from digitizing means 230 and is operable in response to compute display signals representative of the digitized video image(s) of workspace 208 as seen by one or both of cameras 200, 202. Computing means 232 comprises at least a central processing unit, memory means which includes both volatile and nonvolatile memory components, data input means, an image processing/computer graphics subunit, and output means for outputting display signals. The foregoing components of computing means 232 are functionally interconnected generally as known in the art of computing. In a further embodiment, computing means 232 is operable to combine images of the workspace made from each of the two different positions to produce a single stereo image.

Computing means 232 supplies the display signals to a display unit 240 which may be a video display, a CRT monitor, or the like. Display unit 240 converts the display signals to a video image of the workspace 208 as seen by either or both of cameras 200, 202. Display unit 240 is positioned for ready viewing by medical personnel performing procedures in the workspace. Preferably, display unit 240 is constructed to provide sufficient resolution to adequately distinguish significant components in images of the workspace 208. In FIG. 2, display unit 240 is depicted as having a single viewing screen showing the image as seen by camera 200. This embodiment is provided with a single screen for displaying visual depictions of the available scans and images. These may include the image made by camera 200, the image made by camera 202, scan images derived from volume scanning methods, X-ray images made by X-rays and including angiograms, etc., as selected by a user operating computing means 232. The user may switch the display from one to another of the various visual depictions, as desired. Also, one or more features of a first selected depiction, or the entire first selected depiction, can be overlaid on a second selected view.

Alternatively, display unit 240 may contain a plurality of viewing screens arranged for simultaneously displaying in separate screens, the selected depictions.

Computing means 232 also provides graphic display signals to the display unit 240, to produce graphic icons overlaid upon the selected displayed image. The graphic icons should include a cursor which can be positioned by the user at a feature of interest in the displayed image.

Computing means 232 is further constructed, or alternatively programmed, to compute a workspace coordinate framework which defines workspace 208 in terms of three-dimensional Cartesian coordinates in useful distance units, for example millimeters. The workspace coordinate framework is computed from the two digitized 2-dimensional images of the fiducial structure 220 provided respectively by cameras 200, 202, plus the known location parameters of fiducial points on fiducial structure 220 (described in more detail in reference to FIG. 3). In the working embodiment, computing means 232 performs these computations according to a well-known projection algorithm, originally developed by Bopp and Krauss (Bopp, H., Krauss, H., An orientation and calibration method for non-topographic applications, Photogrammetric Engineering and Remote Sensing, Vol. 44, Nr. 9, Sept. 1978, pp. 1191-1196).

The memory means of computing means 232 is constructed or programmed to contain the known location parameters of fiducial structure 220, which are required for performance of the computations producing the workspace coordinate framework from the two 2D images. In the working embodiment, these known location parameters include three-dimensional Cartesian coordinates for each of the fiducial points and the actual distances between some of the fiducial points as measured from fiducial structure 220. The latter distances are not required for establishing the workspace framework, but are used to calibrate the framework in terms of useful real distance units.

Once the workspace coordinate framework has been computed, computing means 232 is further operable to compute the 3D location coordinates within the workspace framework of any feature of interest whose position may be observed by means of the images made with both of cameras 200, 202. Such workspace location coordinates will be accurate provided the two images are made from substantially the same positions relative to workspace 208 as during the establishment of the three-dimensional framework with the fiducial structure.

Features which can be observed by means of the images made by cameras 200, 202 include both features actually seen in both images, and features which are not within the field of view of one or both images but whose position can be indicated by use of a pointer with at least two fiducial marks, where the distance between at least one of the fiducial marks and the tip of the pointer is known. Two fiducial marks are needed to establish the direction with respect to the workspace coordinate framework, of a vector representing the linear direction of the pointer. Alternatively, any other marker(s) useful to compute the vector direction may be employed.

Examples of features of interest include externally-placed portions of scan markers used for volume and/or 2D scans, anatomical features on or within the patient including skull surface contours, marks on the patient's skin, medical instruments and devices, etc.

Computing means 232 further has data input means 238 for receiving data representing one or more scans produced by volume imaging methods (PET, MRI, CT) and/or by 2D imaging methods (X-rays, angiograms) etc. In an alternate embodiment, computing means digitizes the CT and/or MRI volume scans and integrates the digitized volume data to establish the volume scan 3D coordinate system.

Once the workspace coordinate framework and any volume scan coordinate framework(s) have been established, computing means 232 is further operable to apply standard mathematical methods to align the scan coordinate framework(s) with the workspace framework. Knowledge of the coordinates in both the scan framework and the workspace framework of each of three selected landmarks is required and is sufficient for the alignment. Such landmarks may be anatomical features, scan markers which produce distinctive spots in the scan, or any other feature which can be unequivocally identified in both the images made by the imaging means and in the scan.

Using information derived from the mathematical operations used to align the volume scan framework with the workspace framework, computing means 232 is further operable to derive-transformation functions for converting scan location coordinates describing the position of a selected point in terms of the scan framework, to workspace location coordinates which describe the position of the same selected point in terms of the workspace framework. A term used in the art for this conversion process, which will also be used for purposes of this application, is "mapping" of coordinates from one framework to another.

Computing means 232 may also perform the converse operation, e.g. to map coordinates of a selected point from the workspace framework to the volume scan framework.

In a further embodiment, the system includes means for attaching at least two fiducial marks to instrument(s) to be used in the workspace. Alternatively, a set of instruments having at least two fiducial marks may be provided as part of the system. These fiducial marks permit tracking of the position of an operative portion of the instrument and extrapolation of its path. These operations will be described in greater detail hereinafter. In still another embodiment, features normally present on a medical instrument may be used as the fiducial marks, provided the distance between at least one of such marks and the operative portion is measured and provided to computing means 232.

In the working embodiment depicted in FIG. 2, which is a currently preferred embodiment, computing means 232, digitizing means 230, and display unit 240 take the form of a computer workstation of the type commercially available, having standard image processing capability and a high-resolution monitor. In this embodiment, the digitizing of all of the images made by the workspace imaging means, digitizin