Apparatus and method for photogrammetric surgical localization

What is claimed is:

1. Apparatus for defining a location of a medical instrument relative to features of a medical workspace including a patient's body region, comprising:

workspace imaging means positionable for producing a plurality of pairs of images of a medical workspace, each of said image pairs comprising two images made along one of each of a different one of two sightlines, said sightlines intersecting at an angle;

digitizing means operably disposed for digitizing each of said images of said image pairs to produce sets of image signals, one said set of image signals corresponding to each of said images;

fiducial means removably positionable in said workspace for providing a series of fiducial points held in fixed spatial relation to one another; and

computing means connected to said digitizing means to receive said image signals therefrom, and including memory structure having pattern recognition data and instrument structure data Stored therein, said pattern recognition data and said instrument structure data both being correlated to individual ones of a plurality of different medical instruments; said computing means being operable to:

establish a workspace coordinate framework in three dimensions from one of said pairs of images made when said fiducial means is positioned within said workspace,

determine workspace coordinates in said workspace framework of any selected point which can be identified from both images of said pair,

use said pattern recognition data to recognize a selected one of said medical instruments when it is visible in both images of one of said pairs, and

compute workspace coordinates of an operative portion of said selected medical instrument.

2. The apparatus of claim 1, wherein said workspace imaging means comprises at least two video cameras.

3. The apparatus of claim 2, wherein said computing means is further constructed to provide a signal which identifies said selected medical instrument, and said apparatus further includes means operably associated with said computing means for communicating said signal to a user.

4. The apparatus of claim 2, wherein said computing means is further operable to receive and correlate scan coordinates derived from a scan coordinate framework for each of three or more selected scan markers with the workspace coordinates of said selected scan markers as derived from one of said image pairs, to compute conversion functions for converting scan coordinates of any selected feature in a scan made in said scan coordinate framework to workspace coordinates in said workspace framework and for converting the workspace coordinates of any selected feature observable in both images of said image pairs to corresponding scan coordinates.

5. The apparatus of claim 4, wherein said computing means is further operable to extrapolate either workspace coordinates or scan coordinates of said operative portion of said instrument from selected instrument structure data and a set of observed instrument coordinates of said selected instrument.

6. Apparatus for defining a location of a medical instrument relative to features of a medical workspace including a patient's body region, comprising:

workspace imaging means positionable for producing a plurality of pairs of images of a medical workspace, each of said image pairs comprising two images made along one of each of a different one of two sightlines, said sightlines intersecting at an angle;

digitizing means operably disposed for digitizing each of said images of said image pairs to produce sets of image signals, one said set of image signals corresponding to each of said images;

fiducial means removably positionable in said workspace for providing a series of fiducial points held in fixed spatial relation to one another; and

computing means connected to said digitizing means to receive said image signals therefrom, and including memory structure having pattern recognition data and instrument structure data stored therein said pattern recognition data and said instrument structure both being correlated to each of a plurality of different medical instruments; said computing means being operable to:

establish a workspace coordinate framework in three dimensions from one of said pairs of images made when said fiducial structure is positioned within said workspace,

determine workspace coordinates in said workspace framework of any selected point which can be identified from both images of said image pair,

correlate scan coordinates for each of three or more selected scan markers with the workspace coordinates of the same said scan markers as derived from one of said image pairs,

compute conversion functions for converting the scan coordinates of any selected feature in a scan made in a scan coordinate framework to workspace coordinates in said workspace framework, and for converting the workspace coordinates of any selected feature observable in both images of said image pairs to scan coordinates in said scan coordinate framework,

use said pattern recognition data to recognize a selected one of said medical instruments which is visible in both images of one of said pairs, and compute workspace coordinates of a visible portion of said selected medical instrument and an operative portion of said selected medical instrument, said visible portion being visible in both images of said one image pair, and said operative portion not being visible in said one image pair.

7. The apparatus of claim 6, wherein said computing means is further operable to compute an instrument vector for said selected instrument, and to extrapolate a path along said instrument vector in said workspace framework or in said scan framework to determine whether said operative portion of said selected medical instrument will intersect a selected feature in said scan.

8. The apparatus of claim 7, wherein said workspace imaging means comprises at least two video cameras.

9. The apparatus of claim 8, wherein said computing means is further constructed to provide a signal which identifies said selected medical instrument, and said apparatus further includes means operably associated with said computing means for communicating said signal to a user.

10. A method of localizing a feature of interest in a workspace, comprising the steps of:

providing a fiducial structure as described in claim 1, having known 3D coordinates;

making a calibration image pair of a workspace comprising first and second 2D images of the workspace made along different sightlines, and with the fiducial structure positioned in the workspace such that a minimum number of fiducial points are visible in both images of the calibration image pair;

digitizing the respective 2D images of the calibration image pair;

determining a first plurality of 2D coordinates from the first 2D image of the calibration image pair and a second plurality of 2D coordinates from the second 2D image of the calibration image pair;

computing a 3D workspace coordinate framework from said first and second pluralities of 2D coordinates and the known 3D coordinates of the fiducial points:

making a subsequent image pair comprising two 2D images of the workspace having a medical instrument visible therein, the two 2D images being made along the same respective sightlines as the calibration image pair;

digitizing both the 2D images of the subsequent image pair;

identifying the medical instrument using a database of pattern recognition features corresponding to different instruments in a group of medical instruments; and

determining a set of workspace 3D coordinates of the identified instrument from the 2D images of the subsequent image pair.

11. The method of claim 10, further including the steps of: providing volume scan data reflective of a volume scan

having a defined volume scan 3D framework; aligning the volume scan 3D framework with the workspace 3D

framework to produce a function for mapping the scan 3D coordinates of a selected point to workspace 3D coordinates;

selecting a target in the volume scan and determining workspace 3D coordinates for the target using the mapping function; and

comparing the positions of the target feature and the feature of interest in terms of their respective workspace 3D coordinates.

12. The method of claim 11, wherein the feature of interest is a medical instrument having an operative portion, and further including the steps of computing an instrument vector representing a linear direction of the instrument, computing coordinates representative of a present position of the operative portion of the instrument, and extrapolating the instrument vector from the coordinates of the present position of the operative portion for a distance sufficient to determine whether the instrument vector will intersect the target.

13. The method of claim 12, further including a step of providing video imaging means for making the image pairs, said step of providing video imaging means being performed prior to said step of making a calibration image pair, and wherein the calibration and subsequent image pairs are obtained with the video imaging means.

14. An apparatus for stereotaxic localization of a medical instrument relative to a structure within a patient's body region, comprising:

workspace imaging means positionable for producing a plurality of pairs of images of a medical workspace, each of said image pairs comprising a first image made along a first sightline and a second image made along a second sightline,

a set of six fiducial markers associated with said workspace;

said first and second sightlines intersecting at an angle such that said set of six fiducial markers is visible in both said first image and said second image, said six fiducial markers being fixed with respect to one another and arranged to establish a 3 dimensional workspace coordinate framework;

digitizing means operably disposed for digitizing each of said images of said image pairs to produce sets of image signals, one said set of image signals corresponding to each of said images;

computing means connected to said digitizing means to receive said image signals therefrom, and including memory means for storing data including a plurality of instrument dimensions, each of said instrument dimensions comprising the distance between a fiducial mark on an instrument and a preselected operative portion of said instrument; said computing means being operable to:

establish said 3 dimensional workspace framework in three dimension using set of six fiducial markers,

determine workspace coordinates in said workspace framework of an instrument fiducial point visible in both said first and second images of an image pair,

determine the instrument in use from stored instrument structure date,

compute an instrument vector representing a linear direction of the instrument bearing the instrument fiducial point, and

compute coordinates representative of a position of an operative portion of the instrument from said instrument vector and the workspace coordinates determined for said visible instrument fiducial point.

15. The apparatus of claim 14, wherein said computing means is further constructed to extrapolate a path of the instrument vector for a desired distance from said coordinates of the operative instrument portion.

16. The apparatus of claim 15, wherein said computing means is further constructed to:

receive a set of scan coordinates for each of a plurality of landmarks present in a scan made in a scan coordinate framework, and to correlate said scan coordinates with the workspace coordinates of at least three of said landmarks as derived from one of said image pairs, said landmarks being selected from the group of anatomic features, surgical implants, radiologic implants, and fiducial structures adapted to be affixed to said patient;

compute conversion functions for convening scan coordinates of a selected feature in said scan to a set of corresponding workspace coordinates, and for convening workspace coordinates of a visible feature which is visible in both said first and second images of a selected one of said image pairs to a set of corresponding scan coordinates; and

use said conversion functions to compute one or more additional sets of corresponding workspace coordinates for user-selected features observable in said scan, and to compute one or more additional sets of corresponding scan coordinates for user-selected features visible in both said first and second images of pairs.

17. The apparatus of claim 14, wherein said memory means further stores pattern recognition data and said computing means is further constructed to use said pattern recognition data to recognize one of said medical instruments when it is visible in both images of one of said pairs, and to compute said workspace coordinates representative of an operative instrument portion for said recognized instrument.

18. The apparatus of claim 14, wherein said sightline intersection angle is between about 5 degrees and 175 degrees.

19. The apparatus of claim 14, wherein said six fiducial markers are selected from the group consisting of: fiducial markers on a fiducial structure adapted to be affixed to a patient; fiducial marks adapted to be inscribed on a patient's body; fiducial markers on a free-standing fiducial structure; or any combination thereof.

20. A method of stcreotaxic localization of a medical instrument with respect to a structure within a patient's body region, comprising the steps of:

providing imaging means positioned for making a plurality of image pairs of a medical workspace having a patient's body region disposed therein, wherein each of said image pairs comprises a first 2D image made along a first sightline and a second 2D image made along a second sightline which intersects said first sightline at an angle; providing fiducial means for establishing a workspace coordinate framework, and including six or more fiducial points which are fixed with respect to one another, spaced and arranged in 3 dimensions, and having known 3D spacings from one another;

making a calibration image pair comprising calibration 2D images, wherein at least six of the fiducial points are visible in first and second images of the calibration image pair;

digitizing the calibration 2D images to produced digitized calibration 2D images; computing a 3D workspace coordinate framework from the digitized calibration 2D images and the known 3D spacings of the fiducial points;

providing an internal scan of the patient's body region to identify one or more internal features;

providing a medical instrument having an operative portion and having known dimensions;

making and digitizing a subsequent image pair of tile workspace having a portion of the medical instrument visible in both images of the subsequent image pair to produce a digitized subsequent image pair;

computing workspace coordinates of the visible portion of tile medical instrument from the digitized subsequent image pair;

computing workspace coordinates of an operative portion of the medical instrument from the workspace coordinates of the visible portion; and

determining a locational relationship between the operative portion of the medical instrument and one or more internal features identified from the scan.

21. The method of claim 20, wherein fiducial markers are selected from the group consisting of: fiducial markers on a fiducial structure adapted to be affixed to a patient; fiducial marks adapted to be inscribed on a patient's body; fiducial markers on a free-standing fiducial structure; or any combination thereof.

22. The method of claim 20, further including a step of computing a conversion function for converting coordinates between the scan coordinate framework and the 3D workspace coordinate framework, said step of computing a conversion function being performed prior to said step of determining a locational relationship.

23. The method of claim 20, further including the steps of:

computing an instrument vector representing a present direction of the instrument; and

extrapolating the instrument vector into a space beyond a present location of the operative portion of the medical instrument.

24. The method of claim 20, wherein said step of making and digitizing a subsequent image pair, and all steps recited thereafter, are repeated in sequence to track the medical instrument for as long as desired by a user.

Description Submit all comments and votes

BACKGROUND OF THE INVENTION

1. Field

The application is related to techniques for mapping internal structures in the body of an animal or human, and more particularly to such technique for localizing a medical instrument with respect to anatomical features or the like during surgical or other medical procedures.

2. State of the Art

Various scanning apparatus and methods are known for imaging and mapping body structures, which provide target location data for surgical and other medical procedures. One group of methods, including still photography, videography, radiological x-rays, and angiography, typically produces only a two-dimensional projection of a three-dimensional object. For purposes of this application, this first group will be termed "two-dimensional" or "2D" imaging.

A second group of methods, of which computerized tomographic (CT) scanning, positron emission tomography (PET) scans, and magnetic resonance (MRI) imaging are exemplary, provides three-dimensional (abbrev. "3D" herein) information about internal structures (i.e., structures not visible from the exterior of the patient). The three-dimensional information about the internal volume is reconstructed from multiple scans of a known thickness (generally about a millimeter) made along parallel planes displaced from each other by a known distance, usually of the order of millimeters. An example of such a reconstructed volume image is depicted in FIG. 1A, including the contours of a selected anatomical feature within the brain. In this application, methods in this second group will be referred to as "volume" scanning or imaging.

In performing resection or other surgical manipulations, it is highly desirable to correlate the location of instruments, patient anatomical features, or other elements or structures placed in the surgical field, and generally as seen by the surgeon, with the location of internal targets or features as visualized by one of the volume scanning techniques. Such a correlation process is often termed "localization".

A commercially available device for localization in neurosurgery is the Brown-Roberts-Wells (abbrev. BRW) localizer (U.S. Pat. Nos. 4,341,220, and 4,608,977). The BRW system includes a large ring-like structure which surrounds the patient's head and is fixed in place. The ring establishes a 3D coordinate system with respect to the patient's head. A separate calibration unit having an array of rod elements is fixed to the ring to surround the head during the production of volume scan and/or 2D images. The rod elements have known coordinates in the 3D coordinate system established by the ring, and produce spots in the volume scans. Other features in the volume scans can then be assigned coordinates in the 3D coordinate system established by the ring, by correlation with the known coordinates of the rod elements producing the spots.

After the images are made, the calibration unit is detached from the ring, and a guidance arc calibrated to the 3D coordinate system of the ring is attached in its place. The guidance arc provides coordinate reference information which may be used to guide a medical instrument. The medical instrument is usually attached to the guidance arc.

The BRW system has several disadvantages. The ring is cumbersome and uncomfortable for the patient, but it must be affixed in place when the volume and/or 2D scans are made, and kept there until the medical procedure is complete. It is possible to remove the ring after the scans are made, but precise repositioning is critical to avoid error in localization. Accurate repositioning is difficult, so present practice generally is to keep the ring in place until after the surgery. When not attached to the guidance arc, the position of a medical instrument in terms of the 3D coordinate system of the ring, and therefore in respect to the features identifiable in the volume or 2D scan, is not accurately known.

U.S. Pat. No. 4,618,978 to Cosman discloses a localizer device for use with a BRW-type system, including an open box composed of connected rods, which surrounds the patient's head and constitutes a calibration unit.

Alternatively, cranial implants of radio-opaque or MRI-opaque materials can be made. Generally, a minimum of three implants are required for establishing a three-dimensional space in volume scans. At present this method is considered very undesirable, in part because of the risk of infection or other complications of the implants.

Accordingly, a need remains for rapid, reliable, and inexpensive means for localizing a medical instrument relative to points of interest including both visible anatomical features and internal features imaged by volume and/or 2D methods. A need further remains for such means which does not require the physical attachment of a reference unit such as the BRW ring to the patient. Highly desirably, such means would be useful to track the position of a medical instrument in real time, and without requiring that the instrument be physically attached to a reference guide.

OTHER TERMS AND DEFINITIONS

A coordinate system may be thought of as a way to assign a unique set of numerical identifiers to each point or object in a selected space. The Cartesian coordinate system is one of the best known and will be used in this paragraph by way of example. In the Cartesian coordinate system, three directions x, y, z are specified, each corresponding to one of the three dimensions of what is commonly termed 3D (three-dimensional) space (FIG. 1B). In the Cartesian system, any point can be identified by a set of three values x, y, z. The x, y and z directions can be said to establish a "three-dimensional framework" or "coordinate framework" in space. A selected point "A" can be described in terms of its values x.sub.a, y.sub.a, z.sub.a ; these values specify only the location of point A. A different point B will have a different set of values x.sub.b, y.sub.b, z.sub.b. Such a set of values x,y,z for any selected point is referred to herein as the "coordinates" or "locational coordinates" of that point. When the position of a feature larger than a single point is being described, these terms are also understood to refer to a plurality of sets of x,y,z values. Other types of coordinate systems are known, for example spherical coordinate systems, and the terms "coordinates" and "locational coordinates" should further be understood to apply to any set of values required to uniquely specify a point in space in a given coordinate system.

The term "fiducial" is used herein as generally understood in engineering or surveying, to describe a point or marking, or a line, which is sufficiently precisely defined to serve as a standard or basis reference for other measurements.

SUMMARY OF THE INVENTION

The invention comprises apparatus and a method for defining the location of a medical instrument relative to elements in a medical workspace including a patient's body region, especially (but not limited to) elements seen by the surgeon. The apparatus develops a calibrated 3 dimensional framework of the workspace from a pair of 2D images made from different fixed locations, and aligns the workspace framework with a 3D scan framework defined by a volume scan. A pair of video cameras is the present preferred imaging means for obtaining the 2D image pairs. The apparatus is then operable to locate and track the position of a medical instrument during a medical procedure, with respect to features observable in either the workspace images or in the volume scan. A pictural display of such location and tracking information is provided to aid a medical practitioner performing the procedure.

In a further embodiment, the computing means is operable to automatically recognize and track the position of selected medical or surgical instruments during a procedure, from the workspace images.

The apparatus may be described as follows. Workspace imaging means are provided and positioned for producing a plurality of pairs of 2-dimensional images of a medical workspace. Each image pair comprises two such images made in effect simultaneously along respective different sightlines which intersect at an angle. Digitizing means are operably disposed for digitizing each image to produce corresponding sets of digital output signals, one set for each image.

Calibration means are removably positionable in the workspace for calibrating the workspace in terms of a three-dimensional coordinate framework. The 3D workspace framework is derived by computation from the two 2D projections of an image pair made with the calibration means positioned in the workspace. The calibration means comprises a set of at least six fiducial points connected by a frame means consisting of a frame constructed to hold the fiducial points in fixed spatial relation to each other. The frame need not include any means for attaching the fiducial points to a patient. The set of fiducial points has known spatial parameters which define an arbitrary Cartesian 3-dimensional coordinate system. These spatial parameters include 3D location coordinates of each of the fiducial points. Optionally but desirably, at least some of the actual distances between fiducial points should be known, to calibrate the workspace in terms of a suitable distance unit such as millimeters.

A computing means is connected to receive the digital output signals reflective of the images. The computing means also has data input means for receiving scan data from a volume scan of the patient's body region. The scan data define a scan 3D coordinate framework and internal anatomical structures therein. The computing means is further constructed or programmed to perform the following steps: 1) establish a workspace coordinate framework in three dimensions from an image pair made with said fiducial structure positioned within the workspace; 2) determine the locational coordinates in the workspace framework of any selected point which can be identified from both images of said pair; 3) correlate the scan locational coordinates for each of three or more selected landmarks observable in the scan with the workspace locational coordinates of the same landmarks as derived from a video image pair; 4) use the correlation of the workspace coordinates and the scan coordinates of the landmarks, to derive a transformation algorithm for mapping selected other features from either the scan framework to the workspace framework, or the converse; and 5) provide display signals encoding a display reflective of one or both of the workspace images and/or a volume scan, as selected by a user. Display means are provided for displaying the images encoded by the display signals.

Optionally but highly desirably, the computing means has computer graphics capability for producing graphic icons overlaid upon the displayed images. Such icons include a cursor which the user employs to select features in the displayed images for computation of their coordinates or other operations.

A method of surgical guidance may be described as follows. First, a fiducial structure having six or more fiducial points defining two distinct, non-orthogonal planes is positioned in a medical workspace. Workspace imaging means are disposed for making pairs of two-dimensional images of the workspace in which the two member images are made along different but intersecting sightlines. A calibration image pair comprising images of the workspace with the fiducial structure is made. The fiducial structure is removed from the workspace.

A projection algorithm is applied to reconstruct a workspace 3D coordinate framework from the calibration image pair. At least one additional 3D scan framework is obtained from a corresponding volume scan of the patient's body region. At least three landmarks identifiable in both the volume scan and the workspace image pair are selected, and the coordinates for the three landmarks are determined in both the workspace framework and the scan framework. From these determined coordinates, a process is developed for aligning the scan framework with the workspace framework, and transformation algorithms for converting coordinates from one of the frameworks to the other are computed.

A target of interest in the volume scan is identified, and its scan coordinates are determined and converted to workspace coordinates. A feature of interest in the workspace, such as a fiducial mark on a scalpel, is identified. The workspace coordinates of the fiducial mark and of the scalpel tip (whose distance from the fiducial mark is known), plus a vector describing the direction of the scalpel, are determined. Optionally but highly desirably, both the target and the scalpel including the scalpel tip position are displayed in an image of the workspace. The path of the scalpel tip is extrapolated along the vector for a distance sufficient to determine whether the tip will reach the target on this path. If not, the direction of the scalpel is adjusted and the process of localizing the tip and extrapolating its path is repeated until the extrapolated path is deemed adequate by a user, and/or until the medical procedure is complete.

BRIEF DESCRIPTION OF THE DRAWINGS

In the figures, which illustrate what is presently regarded as the best mode for carrying out the invention, like reference numbers indicate like elements of the apparatus:

FIG. 1A is a cartoon of a volume scan of a patient's head;

FIG. 1B depicts a 3-dimensional coordinate system;

FIG. 2 is a block diagram depicting the basic elements of a video localization system of the invention;

FIG. 3 depicts an embodiment of the fiducial structure in greater detail;

FIG. 4 depicts a pair of images made from different positions of a surgical workspace including a patient's head, with the fiducial structure of the invention positioned for calibrating the workspace;

FIG. 5 is a flow chart of a portion of the operation of a further embodiment in which the control means is configured to provide object recognition and location of medical instruments and the like in the image field; and

FIG. 6 shows a top plan view of the probe.

DETAILED DESCRIPTION OF THE ILLUSTRATED EMBODIMENT

FIG. 2 is a block diagram depicting the basic elements of a working embodiment of a video localization system of the invention. A pair of video cameras 200, 202 are positioned for making a pair of images along respective sightlines 204, 206, of a medical workspace 208 which includes a patient's body region here shown to be the patient's head 210. Cameras 200, 202 are arranged to have an angle 212 between sightlines 204, 206, such that both cameras image the workspace 208. Workspace 208 is effectively defined by the overlapping fields of view of the respective images made by cameras 200, 202. Angle 212 is preferably between about 30.degree. and 150.degree.. However, any angle greater than zero degrees and not equal to 180.degree. can be used.

Alternatively, cameras 200, 202 may be replaced by a single camera which is moved back and forth between first and second positions to take images along respective sightlines 204, 206. In the latter case, it is important that the camera be precisely positioned in the first and second positions when making the respective images of an image pair. Positioning means may be provided for establishing fixed attachment points for attachment of a camera, to facilitate such repositioning. Whether one camera or two cameras are used, what is significant is that the system takes pairs of images of workspace 208, each member of an image pair made along different sightlines intersecting in workspace 208.

A fiducial structure 220 (described in greater detail with reference to FIG. 3) is shown positioned in the workspace 208 proximal to the head 210. During use, fiducial structure 220 can be held in position by any suitable support means (not shown). One suitable support means would be a bar with a clamp arm attached to a ring stand or the like. Notably, fiducial structure 220 is neither affixed to, nor in contact with, the patient's head 210. Fiducial structure 220 may be removed from workspace 208 when it is not required.

Cameras 200, 202 are communicatively connected to image digitizing means 230, which produces two sets of digitized image signals, each representative of a respective image detected by one of the two cameras. Digitizing means 230 is in turn connected to send the digitized image signals to computing means 232.

Computing means 232 receives the digitized image signals from digitizing means 230 and is operable in response to compute display signals representative of the digitized video image(s) of workspace 208 as seen by one or both of cameras 200, 202. Computing means 232 comprises at least a central processing unit, memory means which includes both volatile and nonvolatile memory components, data input means, an image processing/computer graphics subunit, and output means for outputting display signals. The foregoing components of computing means 232 are functionally interconnected generally as known in the art of computing. In a further embodiment, computing means 232 is operable to combine images of the workspace made from each of the two different positions to produce a single stereo image.

Computing means 232 supplies the display signals to a display unit 240 which may be a video display, a CRT monitor, or the like. Display unit 240 converts the display signals to a video image of the workspace 208 as seen by either or both of cameras 200, 202. Display unit 240 is positioned for ready viewing by medical personnel performing procedures in the workspace. Preferably, display unit 240 is constructed to provide sufficient resolution to adequately distinguish significant components in images of the workspace 208. In FIG. 2, display unit 240 is depicted as having a single viewing screen showing the image as seen by camera 200. This embodiment is provided with a single screen for dis