Integrating image fusion with nanoparticle contrast agents for diagnosis: a review

Abstract

Background

As newer technologies in the field of medical imaging continue to expand, development of unique techniques for optimizing image quality and minimizing radiation dose becomes very necessary for improve diagnosis of pathologies and patient safety. Different types of medical imaging devices have been developed for specific diagnostic purposes.

Main text

This article provides a brief overview on the need for co-registration of some medical images into a single image (image fusion), advantages of some nanoparticle contrast agents in medical imaging, and a discussion of present and future role of integrating image fusion with nanoparticle contrast agents in diagnosis.

Conclusion

The use of nanoparticle contrast agents together with image fusion is a promising technique in future medical imaging as is likely to reveal pathologies of ≤ 1 nm sizes.

Background

Today, medical imaging plays an important role in diagnosis and prognosis of pathologies. The first modern technology to have evolved from medical imaging was X-ray imaging. However, the rapid advancement of the technology led to the development of modern clinical technologies that are employed worldwide in hospitals/clinics for diagnosis and therapy. These include ultrasound (US), magnetic resonance imaging (MRI), computed tomography (CT), single-photon emission computed tomography (SPECT), and positron emission tomography (PET) [1]. These imaging modalities use different approaches for important analysis and provide different information on the human body for diagnosis. For example, CT scans provide information on the body structure, MRI scans provide detailed information on the tissue types and PET provides functional information on the anatomy being examined.

Also, each of these individual imaging modalities have unique strength and limitations in regards to spatial resolution, image contrast, signal-to-noise ratio, and sensitivity. The various imaging modalities do complement each other. For instance, in nuclear medicine imaging, PET often shows abnormalities with high contrast and insufficient anatomic detail which limits the identification of organ or tissue with the lesion. In addition to nuclear medicine imaging, the attenuation by patient of the emitted radiation in SPECT reduces the anatomic detail required in the image. These abnormalities could be corrected by combining the nuclear medicine imaging system (SPECT or PET) with other imaging systems such as CT, MRI, or ultrasound.

One major technique that has been employed to overcome these limitations and improve image quality is image fusion. Image fusion is to combine two or more images from different imaging modalities of the same scene into a single image without distracting (changing) the required and relevant features from each of the original images [2]. Recent advances in medical imaging from fusion of images from different imaging modalities have proven to significantly improve diagnostics and monitoring disease progression. Image fusion is very useful for evaluating patients receiving cancer care in the areas of diagnosis, staging, treatment planning, monitoring the response to therapy in addition to disease progression [3]. Image fusion from diagnostic imaging modalities such as CT or MRI provides a good definition of anatomy which removes the anatomic localization of the abnormalities and correction of the emission images for attenuation.

Another technique that has also been employed to improve image quality is through the use of nanoparticle contrast agents. Nanoparticle contrast agents are able to enhance images produced from these imaging modalities by highlighting specific diseases or features between two tissues within the anatomy under consideration. Current research in molecular imaging is working toward providing new opportunities for biomedical imaging with great promise for the development of unique imaging agents at the nano scale. These nano agents have unique optical, magnetic, and chemical properties which permit their use in the creation of imaging probes with better contrast enhancement, increased sensitivity, controlled biodistribution, better spatial resolution, and temporal information, multifunctional and multimodal imaging across imaging techniques such as MRI, SPECT, PET, and ultrasound. Recently, nanoparticle contrast agents based on polymers, proteins, lipids, metals and silica, carbon nanotubes, selenium-cadmium nanocrystals, and microbubbles are the most recommended nanoparticles for diagnostic and theranostic purpose [4].

Many different nanoparticles have been developed and evaluated over the years. Soon after the development of nanoparticles, systems such as polymers, liposomes, proteins, dendrimers, and micelles were basically used to deliver chemotherapy drugs to pathologic sites while reducing their accumulation in potential endangered healthy tissues. Today, the most widely and frequently used nanoparticle therapeutic agents are Doxil, Ambisome, and Abraxane. Efforts are being made toward the development of newer nanoparticle contrast agents for diagnosis purpose to be incorporated fully into the clinical environment [5]. Figure 1 illustrates specification of some new nanoparticles that are frequently used for diagnostic and therapeutic applications.

Fig. 1

Specification of some nanoparticles frequently used for diagnostic and therapeutic purposes [5]

The aim of this review is to provide a brief overview on the need for co-registration of some medical images, followed by the use of nanoparticle contrast agents and image fusion for diagnosis. Finally, the review provides a discussion on possible future benefits in medical imaging world through the integration of image fusion and nanoparticle contrast agents as a unique imaging technique for diagnosis of diseases.

Medical imaging modalities

Basically, there are four common groups of medical imaging technologies. These are magnetic resonance (for MRI systems), X-ray transmissions (for CT and planar X-ray systems), radiation emissions (for SPECT and PET systems), and acoustic or light reflections (for US systems) [6]. Figure 2a-g below shows image acquisition processes of these imaging equipments. The MRI uses a strong magnet and radiofrequency signals which polarizes and excite hydrogen nuclei within the abundant water molecules and fat in human tissue to produce a detectable signal which are reconstructed with the aid of a computer program to produce an image of the soft tissues. Magnetic resonance technology is considered safe and non-invasive because it uses non-ionizing radiation [12]. On the other hand, CT is mostly used in bone imaging without contrast agents. For lung, vascular and digestive imaging, inclusion of contrast agents such as iodine may be applied. When an X-ray beam is passed through the body by a rotating X-ray tube, the transmitted projection data are collected by an array of detector elements opposite to the X-ray tube. The projection data points are then analyzed mathematically to obtain a CT image [13]. Molecular imaging (SPECT and PET) techniques are very effective in providing information about the functions of cells, tissues, and organs. They are also non-invasive, similar to MRI. To produce images using SPECT or PET, radionuclides are injected into the body which are absorbed by the organ of interest. By gamma (γ) decay, energy emitted is detected by gamma camera which is reconstructed into an image with information on the spatial distribution of radionuclide. The SPECT uses radionuclides that emit γ rays directly but with lower energy as compared to the radionuclides used in PET [9]. Finally, medical US equipment uses ultrasound waves which typically operate within a frequency range of 1 to 15 MHz to produce images. When ultrasound is propagated through the body of a patient, there is attenuation within the tissues of the body. Reflected sound waves are sent back to interact with the piezoelectric element and the resulting voltages are then amplified and relayed onto the image processing circuit to produce a two or three (2D or 3D) dimensional image. Ultrasound imaging has been used extensively for detecting cancerous and benign tumors, assess the development of fetus, and visualization of blood flow in different organs [11, 14, 15].

Fig. 2

a MRI imaging system. b Alignment of protons without magnetic field. c Alignment of protons with magnetic field [7, 8]. d Shows the image acquisition of the SPECT system, where the gamma camera rotates about the patient to acquire a projection image at each angle. e With the PET imaging system, the patient is located within a ring of detectors and the positron annihilation emits two gamma photons in opposite direction which are picked up by two detectors [9]. f CT imaging system [10]. g Ultrasound imaging system [11]

Image fusion

Image fusion involves a series of systematic approach that aims at creating a more detail and an ideal output image. The general steps that are followed in image fusion are image registration, feature extraction, decision labeling, semantic equivalence, radiometric calibration, and image fusion itself (as illustrated in Fig. 3).

Fig. 3

Steps in image fusion procedure

Lately, image registration and fusion procedures have been made easier through the use of software tool kits. Examples of these tool kits include Insight segmentation and registration toolkit (ITK), Elastix, Advanced Normalization Tools (ANTs), NiftyReg, Medical Image Processing, Analysis, and Visualization (MIPAV), Medical Image Processing, Analysis, and Visualization (MIPAV), and OsiriX [16].

Image registration

The purpose of image registration is to compare or integrate data sets between different measurements. In image registration, the input images (images of the same scene from different modalities) are compared and aligned to each in order to measure the similarities (e.g., cross correlation and mutual information) between them. The resultant measurement must not be affected by noise and background changes [16].

Registration is done between a source image and a target image. Usually, important information are extracted from the source image and are transformed by a transformation model such that it matches perfectly with information on the target image. The choice of transformation model depends on the type of variations present between the images. Variations are classified into three groups, namely, spatial variations, volumetric distortion variations, and variations of interest. So, in transforming an image, a transformation function is applied either to the entire image (called global) or subsections of the image (called local). With global transformation, a single transformation is computed on some volume of interest while that of local transformation, at least two composite transformations are computed on sub-images that cannot generally be described a global transformation [17].

Types of image registration

Some types of registration methods that can be applied are correlation and sequential based, Fourier based, point mapping with feedback, point mapping without feedback, basis functions method (interpolation and approximation), and elastic based model [17, 18].

Correlation and sequential based methods

The correlation and sequential based methods are early approaches used in image registration. It is a systematic statistical primary tool mostly used for template matching or pattern recognition to determine the degree of similarity between images by applying a cross-correlation function. This method is relevant to images that are misaligned by a small rigid or affine transformation where only translation and rotations are allowed. The cross-correlation function directly computes sum of difference squares between the source and target image at each location of the source image. The correlation will reach its peak only if the source image perfectly matches the target image. The parameter used to determine the degree of matching is called the correlation coefficient (C_c) as shown in equation 1.

$$ {C}_c=\frac{covariance\left(T,S\right)}{\sigma_T{\sigma}_s}=\frac{\sum_x{\sum}_y\left(S\left(x,y\right)-{\mu}_s\right)\left(T\left(x-u,y-v\right)-{\mu}_T\right)}{\sqrt{\sum_x{\sum}_y{\left(T\left(x-u,y-v\right)-{\mu}_T\right)}^2{\sum}_x{\sum}_yS\left(x,y\right)-{\left.{\mu}_s\right)}^2}} $$

(1)

Where μ_s and σ_s are the mean and standard deviation of the source and μ_T and σ_T are the mean and standard deviation of the target, x and y are the positions of the source, u and v are locations for which the source image is placed over the target image. The resultant value of C_c shows a linear similarity between the source and target image [17, 19]. The computed C_c values are usually between −1 and +1. When the C_c value is determined to be ≅ +1, it means the source and target images are highly similar and are accurately registered. However, when the source and target images are dissimilar and nor properly registered, the computed C_c value would be ≅ −1 [20].

Fourier method

The Fourier method registers the source and target image by matching them using information in the frequency domain. This method of registration is more effective when applied to images obtained under different conditions of illumination and sensors. The source and target image are matched by the translation property of the Fourier transform using the phase angle.

The Fourier transform of an image f (x, y) is given by equation 2:

$$ F\left({w}_x,{w}_y\right)=\left|F\left({w}_x,{w}_y\right)\right|{e}^{i\varphi \left({w}_x,{w}_y\right)} $$

(2)

Where |F(w_x, w_y)| is the magnitude of the Fourier transform and φ(w_x, w_y) is the phase angle which determines the phase shift of the images at each frequency. The translation of the images is done using the phase angle. So, for the two images (i.e., source and target image) f₁ and f₂ which varies by displacement (Δ_x, Δ_y) only, to obtain equation 3 yields equation 4:

$$ {f}_2\left(x,y\right)={f}_1\left(x-{\Delta}_x,y-{\Delta}_y\right) $$

(3)

The Fourier transform of the two images in equation 3 is given by

$$ {F}_2\left({w}_x,{w}_y\right)={F}_1\left({w}_x,{w}_y\right){e}^{-j\left({w}_x{\Delta}_x+{w}_{y\Delta y}\right)} $$

(4)

Equation 4 shows that the two images have the same magnitude of Fourier transform but different phases ${e}^{j\left({\varphi}_1-\kern0.5em {\varphi}_2\right)}$ that are directly related to their displacements.

The two images are then registered by taking the inverse Fourier transform which yields an impulse function as shown in equation 5 below.

$$ \frac{F_1\left({w}_x,{w}_y\right){F}_{2\ast}\left({w}_x,{w}_y\right)}{\left\lceil {F}_1\left({w}_x,{w}_y\right){F}_{2\ast}\left({w}_x,{w}_y\right)\right\rceil }={e}^{-j\left({w}_x{\Delta}_x+{w}_{y\Delta y}\right)} $$

(5)

Where F* is a complex conjugate function of F [17].

Point mapping

Point mapping is one of the recent approaches used in image registration. It is mostly applied to images whose misalignment cannot be determined due to uncertainty of the actual depth in the scene. As a result of this uncertainty, a general transformation (global method) is used to match the source and target image with the aid of landmarks such as smooth surfaces or distortions present in both images. However, if the landmarks become more local, point mapping methods (i.e., point mapping with feedback and point mapping without feedback) are used to determine the misalignment. Image registration with point mapping using the global method involves three steps. In the first step, the features present in the images are computed carefully. The second step is to assign control points to the computed feature points in the target image. Finally, a two-dimensional (2D) polynomial function is applied to the control points to map the source and target image.

For images whose misalignment are known (e.g., small rigid or affine transformation), point mapping feedback methods are used. Point mapping with feedback is used when feature detection and matching of the images are difficult to be performed. The feedback is used between the stages of control points in order to determine the correct transformation to register the images. Point mapping with feedback becomes necessary when the features detected in the target image is ambiguous or the presence of uncorrected variations between the source and target image. As a result of this, point mapping feedback methods such as relaxation, hierarchical, cooperation, and clustering are used for the registration of the source and target image. The relaxation method for example uses the translation properties of transformation to register images, while the clustering method with feedback evaluates all the pairs of feature matches for registration. The transformation parameters for all the pairs of feature matches are determined from which the transformation that best matches the largest number of points is found and used for the registration.

The point mapping without feedback is most appropriate for registering images whose transformation required to align the source and target image is unknown. To generate a single transformation for point mapping without feedback, global methods based on point matching are employed. However, this is only possible when there are enough control points available to obtain any transformation parameters through approximation or interpolation [17].

Basis functions registration method

This type of registration is a mathematical process that registers images by building a model through interpolation or approximation without causing any physical or biological changes to the images.

Approximation

The least-squares regression analysis is one of the basic functions used to obtain transformation parameters through the approximation method by matching points that satisfy it as nearly as possible. The method assumes local noise distortions for the matches that cannot be removed by transformation as a result of differences of interest between the source and target image. This makes it difficult to find a transformation to accurately match the control points. To overcome this difficulty, there must be enough statistical information available for approximation. This is achieved by ensuring that the number of match points is greater than the transformation parameters. This would help to generate a reliable transformation that would map the source image onto the target image because a single transformation is usually required to map one image onto the other [17]. Another basic function that is employed in the approximation method is the radius basis function (RBF). This type of RBF is performed between the interpolation point and identification point. The RBF simply registers images based on the identification point in the medical images by setting the radius basic approximate function to zero (0) [18].

Interpolation

Interpolation method is more applicable to manually controlled points where there are fewer but accurate matches that ensures the matches and their corresponding control points are exact. Here, interpolation uses polynomials to generate the transformation that best matches the source and target image. The coefficients of the polynomials have to be determined by a system of N equations through the mapping of each of the N control points. To register the images, first, anatomic landmarks are manually found in the source image and cross correlated with pixels near the corresponding landmarks in the target image. This process generates a set of matched control points. By using linear regression to fit a low order polynomial, a transformation is generated which is applied to map the source image onto the target image. Registration with interpolation is able to correct affine distortions (e.g., translation, rotation, scale, and shear) [17]. Alternatively, images can also be registered with interpolation using many iterative algorithms by maximizing measure calculated from the voxel values. However, this approach is very expensive due to the fact that interpolation is done for each iteration. As a result, low-cost interpolations such as trilinear or close neighbor iteration is usually recommended until the desired transformation is achieved. Trilinear interpolation iterative algorithms use low pass filters that removes high spatial frequency components of the processed images which minimizes registration errors [19].

Elastic model

Elastic model of registration is a local method that develops unique deformation transformation to map control points of the source image onto the target image by modeling distortions (e.g., 2D projections and 3D objects) present in the image as the deformation of elastic material. Transformation functions such as weighted mean, piecewise linear, thin plate splines, and multiquadric are usually compared and applied during the elastic model process. The process of developing the registration transformation results in some small amount of local stretching of images which is used to correct local non-linear deformations in the processed images. The degree of stretching is used to determine the energy state of the elastic material. In simple terms, the image is modeled as an elastic body. The similarities of features or control points generated between the source and target image as a result of modeling, acts as an external force, which “stretch” the elastic body. The stretch that is produced is returned to its normal state by an opposite parameterized stiffness or smoothness constraints. The threshold energy state of the elastic body is then computed using iterative numerical methods following the deformation transformation and it is that which defines the registration features or control points of the source and target image. The images are then registered by matching important structures between the source and target image. Elastic registration is relevant for the correction of deformed images that results from the use of intravenous contrast agents in the process of accessing the anatomy, patient movement, or during breathing [17, 21].

Steps in image fusion

Image registration

The first step to image fusion is image registration (discussed above). This requires identification and extraction of important points (also called detector points) within the image that convey relevant information about the scene. The important points can be identified by using the intensity coordinates (measured in units of millimeter) at a given voxel point p within the 3D images, usually expressed as I(r,c,z) without altering image noise, image contrast, or blurring. After extraction of detector points, the next step is to estimate the similarities between them. Image descriptors (such as color, texture, shapes, color blobs, and corners) are used to represent detector points after which they are matched to obtain control points in the image. These control points represent the relevant features that are common between the processed images. The control points are extracted either statistically (e.g., mean and variance) or by texture (e.g., smoothness, coarseness, and regularity) such that they have distinct features.

With the registration process, one of the input images is marked as the source/moving image while the rest of the other images are called target/fixed images. The data sets representing the source image is spatially aligned with that of the target image. This is done by the application of a parametric transformation model to the control points to estimate the geometric relation between the input images. The parametric transformation model geometrically maps the coordinates of the source image to the target coordinates using different degrees of freedom (DoF). The DoF defines the number of ways in which the transformation can be changed. Preferably, bigger numbers of DoF are used because it permits greater transformation scope to make one image match the other. The image composition step is finally applied to the geometric relation to combine the registered images into a larger image [16, 22].

Feature extraction

In feature extraction, feature maps for each of the input images are produced as a result of the extraction of distinctive features of the registered images [16].

Decision operators

After feature extraction, decision operators are applied to label the feature maps to produce a set of decision maps. Where the decision maps do not have a link to the same object or phenomena, semantic equivalence is applied to connect these maps to a common object or phenomena [16].

Decision operators

Next to decision operators is radiometric calibration where the feature maps are aligned with the input images to obtain input images of a common scale for image fusion. Finally, the input images are combined into a single output image. The final output image contains detailed information of the scene with improved visibility when compared to any of the input images [16].

Nanoparticle contrast agents

Microbubbles

Microbubbles (MB) are contrast agents that are also commonly used in medical imaging and as careers for targeted drug delivery. They are gas-filled bubbles with a diameter range of 0.5 and 10 μm. Historically, they were developed to enhance ultrasound scan because of their ability to resonate in an ultrasound beam, as they contract and expand when pressure is applied to them in the ultrasound wave. Since 1990 after their development, microbubbles have been used as a blood pool ultrasound contrast agent and their applications are expanding worldwide. When microbubbles are injected intravenously, they flow in a circle, stick to the target receptors on vascular endothelium, and finally accumulate at the disease site. Echoes received from the targeted bubbles reveal the pattern of biomarkers within the tissues. The shell of microbubbles comes in the form of protein, lipids, or polymer. The purpose of the shell is to basically maintain the microbubbles integrity after preparation and in vivo administration. This is achieved by surface tension and Laplace pressure reduction that causes the gas to diffuse out very fast from the microbubbles [23].

Ion oxide nanoparticle

The use of contrast agents in MRI was developed based on the nuclear magnetic resonance (NMR) phenomena which operates on two principles. These principles are NMR processes of spin relaxation, T1 (longitudinal) and T2 (transverse). Different kinds of contrast agents have been developed and used in MRI based on these two principles. So far, the most widely used MRI contrast agents are paramagnetic and super paramagnetic contrast agents. These contrast agents work by shortening the T1 or T2 relaxation time of protons located nearby. When the rate of stimulated emission is increased from high energy state to low energy state, T1 relaxation time is shortened. This is achieved by the use of paramagnetic metal ions contrast agents which accelerates the T1 relaxation time and produce “bright” contrast in T1-weighted images. Also, the super paramagnetic ion oxides (SPIOs) are used to increase T2 relaxation time to produce “dark” contrast on T2-weighted images [24].

Gadolinium MRI contrast agent

Today, the most widely used contrast agent in MRI scan is gadolinium (Gd III) paramagnetic ion. Gadolinium is very effective in reducing protons T1 relaxation time, thereby increasing the signal intensity in MRI. Gd is a silvery-white malleable metal that belongs to a group of lanthanide series with seven unpaired electrons more than any other ion which makes it more effective in T1 relaxation time. It also has a symmetric S state that produces a low electronic relaxation rate. There are several Gd (III) base MRI contrast agents. Some of these includes Gd-DOTA, Gd-DO3A-butrol, Gd-DTPA, Gd-BOPTA, Gd-DTPA-BMA, and Gd-HP-DO3A. All these Gd (III) MRI-based contrast agents have similar properties. They are very efficient in the diagnosis of brain pathologies because they are extracellular agents excreted by glomerular filtration and therefore do not cross the intact blood-brain barrier. The development of Gd (III) contrast agents are geared toward either targeted compounds, responsive agents, or the combination of targeted compounds and responsive agents. The targeted compounds are contrast agents that are developed to accumulate in specific organ or area of the body. Also, responsive agents are those that are developed to alter the relaxivity of protons because of their microenvironment where they distribute [25]. Structurally, Gd contrast agents come into two forms based on the type of ligand used (as illustrated in Fig. 4). One is the linear agent which is an elongated organic molecular ligand that wraps around the Gd (III) ion. The other is the macrocyclic agent which is in a form of cage-like ligand structure with the ion trapped in the central cavity.

Fig. 4

Structure of Gd (III) MRI base contrast agent [26]

When Gd (III) contrast agent is injected into the body, the paramagnetic properties of the metal ion affect the local magnetic field and changes the relaxation of the local hydrogen protons in the vicinity of the gadolinium. This forces the region of protons to fall back to the baseline low energy state quicker, thereby reducing the T1 relaxation time which regrows the longitudinal magnetization. The faster the longitudinal magnetization recovers, the stronger the T1 signal and the brighter that area appears on the T1-weighted image.

Gold nanoparticles

Gold nanoparticles (AuNPs) have been identified as the contrast agent with unique material properties, suitable for several medical imaging modalities. Traditionally, gold contrast agents were produced as gold solutions used in a stained glass window and as a purple dye. Due to their biocompatibility and numerous surface properties, gold nanoparticles have been proven as the safe platform for many biomedical technological research and is now prioritized in the development of chemistry, biology, and medicine. Some areas of AuNP applications include optical biosensors, drug delivery vehicles, imaging contrast agents, and laser-based treatment. AuNP are very flexible, as a result, they have been successfully formulated into different morphologies such as spheres, hollow shells, star shapes, rods, clusters, and cubes for specific purposes. The first application of AuNP was investigated as an X-ray contrast agent where it was injected intravenously and the vascular of mice and organs imaged. The active targeting of diseases using AuNP as a result of its chemical moieties attachments allow the use of AuNPs as contrast agents in multimodal imaging. Table 1 below shows the various imaging modalities achieved using AuNP [1].

Table 1 Imaging modalities achieved using AuNP by the inclusion of formulation methods

The future of image fusion and nanoparticle contrast agents

Over the past decades, researchers have reported the promising results in biomedical imaging using nanoparticles as contrast agents [27]. The application of nanoparticles in medical imaging has proven to enhance image contrast and improve visibility of features that would have been tedious to detect. Also, the recent advances in image fusion have proven to significantly improve diagnostics and monitoring disease progression.

The dramatic growth of both nanoparticle contrast agents in imaging and the benefits that image fusion technique has to offer have a great potential in future medical imaging research. Developing a new imaging technique using the concept of image fusion with the application of nanoparticle contrast agent materials would have a great influence and help address some challenges in medical imaging, especially the problem of early prognosis of various pathologies at the molecular level.

Image fusion involving X-rays has a variety of applications and benefits such as brain diagnosis/treatments, surgical planning procedures, cancer/tumor diagnosis, telemedicine, and 3D tumor simulations. These applications are possible when X-ray images are fused with MRI images as a result of the unique diagnostic properties (e.g., blood vessels, abnormal tissue, breast, pelvis, kidney, heart, and spleen) that MRI provides [16]. An earlier study conducted by De Silva et al. to guide invasive procedures using fused images of X-ray and MRI showed that external fiducial markers are capable of accurately targeting endomyocardial injections of iron-labeled mesenchymal stroma cells in the heart [28]. In the study, live X-ray (fluoroscopy) and 3D MRI images were segmented to determine matching points of the injected fiducial marker locations in the heart. The matched points were then registered by the application of a registration transform [28]. In addition, Hopp et al. evaluated the fusion of X-ray mammography and MRI images to determine their correlation for breast cancer detection using 11 clinical data sets [29]. Information on patient-specific biomechanical model from the images of both modalities were simulated for image registration. Contrast enhancement images of dynamic contrast enhance magnetic resonance imaging (DCE-MRI) were created and then fused with X-ray mammograms. Results of the study indicated that 10 out of the 11 datasets were accurately registered, with 61% of marked lesions in the mammogram overlapped contrast enhance regions and a clearly differentiable lesions from surrounding tissues by DCE-MRI. The study concluded that image fusion between X-rays mammography and dynamic MRI alone can produce a quantitative information for detection of breast cancer [29]. It is anticipated that if nano particle contrast agents were used in both studies, it could have increased the targeting accuracy of endomyocardial injections in the heart and improve the efficacy of lesion detection in the breast.

Further development in medical imaging techniques by the integration of image fusion using four or more imaging modalities coupled with multifunctional nanoparticle contrast agents are likely to lead to improve image quality. In particular, this technique would improve image contrast and spatial resolution as is highly likely to resolve and reveal pathologic images of ≤ 1 nm. For instance, there is contrast enhancement (~ 2 folds) between hepatoma and normal liver tissue with gold nanoparticles using only X-ray CT. Also, nanoparticles that have been synthesized for MRI imaging has significantly resulted in improved image contrast such as detection of lymph node metastasis, enhancement of membrane translocating properties, improved visualization of brain tumors, detection of human colon carcinoma, cellular trafficking, and ultrasensitive molecular imaging [30].

Advances in CT hardware cone-beam scanners alone could produce a resolution of 75 μm if iterative reconstruction and motion correction techniques are applied. Laboratory and industrial CT scanners are able to resolve individual cells on pathology specimen as good as 50 nm. Also, radionuclide imaging with nanoparticle materials such as ⁶⁴Cu-PSTM and ¹⁸F-FDG (for PET imaging) and that of ¹¹¹In-oxide and ^99mTc-HMPAO (for SPECT imaging) are able to detect labels at nanomolar and picomolar sensitivity scale. The maximum detection of CT with contrast agents (10⁻³ M) is less sensitive when compared with MRI (~ 10⁻⁵ M) and radionuclide imaging (~ 10⁻¹⁰ M). This low sensitivity could be improved with dense nanoparticle contrast agents. Multimodal imaging using PET or SPECT together with other imaging modalities such as CT or MRI is very simple, common, and has been used widely [13, 31].

Some polymer molecules such as dendrimers have successfully been used to produce a dual nanoparticles contrast agent for imaging with CT and SPECT. This double labeling contrast agent which contains gold nanoparticles and diatrizoic acid has enhanced contrast in individual CT and SPECT images [32].

Also, microbubbles which was previously introduced as contrast agents to enhance ultrasound scans due to its strong ability to reflect waves effectively than body tissues, now have widespread applications in industry, medicine, and imaging. Super paramagnetic nanoparticle contrast agents have been incorporated into the polymer matrix of the microbubbles or between its shell layers to produce a new contrast agent for drug delivery. This means that by incorporating four or more different nanoparticle contrast agents into the polymer matrix of microbubbles, a multifunctional contrast agent could be produced for use in multimodal imaging. Multimodal imaging with the use of multifunctional contrast agent saves time and money as it reduces inter-variability effects on the same subject and hence the number of subjects required in a study. When multimodal images of the same scene are fused together, the information obtained from the fused image could be used for numerous medical applications. The application of mixed images of the human body provide both anatomical and functional imaging for more qualitative and informative data.

Advances in biomedical imaging has led to diverse management of symptomatic cancer patients from screening to treatment and monitoring, through diagnosis and staging. Imaging techniques such as mammography and colonography are used to screen symptomatic patients for non-specific markers. If there are any suspected cases after screening, diagnosis is performed for biopsies using other imaging techniques such as endoscopy or catheterization laboratory. This is then followed by treatment using appropriate therapeutic techniques. Ideally, the future role of imaging in cancer should be such that, the equipment and techniques used for screening should have the ability to identify specific markers at the molecular level. Also, diagnostic imaging equipment and techniques should be capable of providing quantitative information on biopsies as well as perform functional whole-body imaging [33].

The idea of integrating nanoparticle technology with image fusion will present several benefits, particularly when it comes to providing assistance in the diagnoses and case management of symptomatic cancer patients. For instance, when this technology is fully developed and integrated with other biomedical imaging techniques such as hybrid imaging, image-guided techniques, in vitro imaging, and endoscopy, there would be easy identification of specific markers for symptomatic patients, assessment of pre-tumor and post-tumor staging, as well as monitoring of response to therapy when targeted noninvasive nanoparticles that are capable of amplifying imaging signals are used with this technique. Also, it would provide enhanced fused imaging data during pre-surgical and post-surgical procedures, and reduce sentinel node mapping.

As the field of image fusion techniques and nanoparticles contrast agents continue to expand:

• Identification of newer and suitable nanoparticle contrast agents that could be used to image specific tumor biomarkers will continue.

• Design and construction of a single multimodal imaging technologies such as CT-MRI-SPECT-ultrasound device would emerge.

• Newer multifunctional nanoparticle contrast agents using different polymer molecules for multimodal imaging would be developed.

• Development of multi sequential algorithms for distinguished image enhancement purposes through image fusion would continue.

• Development of newer 3D-tumor model procedures with the application of nanoparticle contrast agents to different image fusion methods would also continue.

Conclusion

If image fusion using four or more imaging modalities (such as X-ray CT, MRI, ultrasound, SPECT, and PET) together with multifunctional nanoparticles contrast agent is fully utilized to develop a new imaging technique, its role may become increasingly larger in the history of medical imaging as it would pave the way for the discovery of newer technologies in medical imaging.

The application of nanoparticle contrast agents in image fusion would provide one of the unique, outstanding, and exceptional medical imaging techniques in the history of medical imaging. After nanoparticle contrast agents have been fully utilized clinically, it is anticipated that this imaging technique would provide the platform for a new generation of screening of pathologies especially for cancerous diseases.

Recommendation

In order to help health practitioners to effectively visualize abnormalities on the diagnostic images with confidence, the following are recommended: firstly, the sensitivity of individual medical imaging devices should be increased through the use of nanoparticle contrast agents’ materials that are capable of interacting with the incident radiation to produce visible changes in the resulting image. Secondly, the resulting individual images from these imaging devices should be fused together in order to enhance the final image by exploring multiple sequential algorithms.