Landmark and Intensity Registration

The five 2D transverse MRI data sets shown in Fig. 7 were used to compare the performance of the unidirectional landmark (UL-TPS); consistent landmark (CL-TPS); consistent intensity (CI-TPS); and consistent landmark and intensity (CLI-TPS) thin-plate spline algorithms. These $256 \times 320$ pixel images with 1 millimeter isotropic pixel dimension were extracted from 3D MRI data sets such that they roughly corresponded to one another. Each data set was registered with the other four data sets for each of the four algorithms producing ten forward and reverse transformations for each algorithm. For brevity of presentation, we only present some of the results of the experiments that are representative of all of the results. A set 39 of corresponding landmarks were manually defined in data sets $B2$ and $B4$ and a subset of the 39 landmarks were manually defined in the additional 3 data-sets (see Fig. 7). Only data sets B2 and B4 had all 39 landmarks identified on them since it was not possible to locate the corresponding locations for all the landmarks on the other data sets due to missing or different shaped sulci. Only corresponding landmarks between two images were used for registration and calculating the landmark error, i.e., if one image set was missing landmark 15, then landmark 15 was not used for registration or for calculating the landmark error.

Figure 7: Five corresponding image slices from MRI acquired brains with manually identified points of correspondence.

Table II lists the parameters used for each algorithm and the computation time that each algorithm required to run on a single 667MHz alpha processor. The algorithmic parameters were chosen to demonstrate the registration performance of the algorithms independent of optimizing the run times. These computation times can be decreased significantly by optimizing the computer code and reducing the number of iterations. The CLI-TPS algorithm was run for 5 iterations of the CL-TPS registration algorithm followed by 95 iterations of the CI-TPS registration algorithm.

Table II: Summary of algorithm parameters and computation times on a single 667MHz alpha processor.

Algorithm	Iterations	Computation Time	$\chi$	$\rho$	$\sigma$	$\alpha$	$\beta$
UL-TPS	1	5 seconds	NA	NA	NA	NA	NA
CL-TPS	20	3 minutes	NA	NA	NA	1.0	0.0061
CI-TPS	1000	1 hour	500	0.0000075	0.10	NA	NA
CLI-TPS	300	1 hour	500	0.0000075	0.50	1.0	0.0061

The result of transforming MRI data set $B5$ in to the shape of $B2$ using each of the four registration algorithms is shown in Fig. 8. These results are typical of the other pairwise registration combinations. The images are arranged left to right from the worst to the best similarity match as shown by the corresponding difference images shown below the transformed images. The UL-TPS and CL-TPS algorithms perform almost identically with respect to similarity matching. The CI-TPS and CLI-TPS intensity based registrations produce better similarity match than the two landmark only methods. In particular, the intensity based methods match the border locations and non-landmark locations better than the landmark thin-plate spline or CL-TPS algorithms. The difference between the CI-TPS and CLI-TPS methods is that the CLI-TPS method produces much smaller landmark errors than the CI-TPS method which cannot be seen in the intensity difference images.

Figure 8: Intensity matching results for registering dataset $B5$ to dataset $B2$ with the four registration algorithms. The top row shows the data set $B5$ transformed into the shape of $B2$ using each algorithm and the bottom row shows the absolute difference image between the transformed $B5$ image and the target $B2$ image. Note that the intensity difference images of the CI-TPS and CLI-TPS are very similar since both algorithms minimize the intensity differences between the deformed template and target images. However, the difference between these two results is that the CLI-TPS also produces much smaller landmark errors which cannot be seen in the intensity difference images.

The images in Fig. 9 show the Jacobian of the forward and reverse transformations between images $B2$ and $B1$ produced by the CL-TPS(left two panels) and CLI-TPS(right two panels) algorithms, respectively. The value of the Jacobian at a point is encoded such that bright pixels represent expansion, and dark pixels represent contractions. Notice that the intensity pattern of the forward and reverse Jacobian images appear nearly opposite of one another since expansion in one domain corresponds to contraction in the other domain. These images show the advantage of using both landmark and intensity information together as opposed to just using landmark information alone. Notice that the CL-TPS algorithm has very smooth Jacobian images compared to the CLI-TPS algorithm. This is because the CL-TPS algorithm matches the images at the corresponding landmarks and smoothly interpolates the transformation between the landmarks. Conversely, the patterning of the local distortions in the CLI-TPS registration resemble the underlying intensity patterning. This indicates that combining the intensity information with the landmark information provides additional local deformation as compared to using the landmark information alone. This improved registration between landmarks produces more distortion of the template image and therefore there is a larger range of Jacobian values for the CLI-TPS algorithm than the CL-TPS algorithm as shown by the color bar scales.

Figure 9: This figure shows the Jacobians of the forward and reverse transformations for the registration of data sets $B2$ and $B1$ for the CL-TPS(left two panels) and CLI-TPS(right two panels) algorithms. The bright pixels of the Jacobian images represent regions of expansion, and dark pixels represent regions of contraction.

Inverse consistency error images are computed by taking the Euclidean norm of the difference between the forward and the inverse of the reverse transformations at each voxel location in the image domain. Figure 10 shows the inverse consistency error images for the registration of data sets $B2$ and $B5$ using the UL-TPS, CL-TPS, CI-TPS, and and CLI-TPS algorithms. Note that each images is on its own color-scale and that the UL-TPS algorithm has 10 to 200 times more maximum inverse consistency error than the consistent registration algorithms. The UL-TPS algorithm had 50 to 500 times more average inverse consistency error than the consistent registrations algorithms. This can be seen by comparing large regions of bright pixels in the UL-TPS image to the small regions of bright pixels in the other images. This figure shows that consistent registration algorithms produced forward and reverse transformations that had sub-voxel inverse consistency errors at all voxel locations. The inverse consistent errors in the UL-TPS and CL-TPS algorithms are greatest away from the landmark driving forces because the landmark driving forces are implicitly inverse consistent. The largest inverse consistency errors in the CI-TPS and CLI-TPS algorithms occur near edges where there is a correspondence ambiguity associated with the intensity matching solution.

Figure 10: Images that display the magnitude and location of forward transformation inverse consistency errors for matching data sets $B2$ and $B5$ with UL-TPS, CL-TPS, CI-TPS, and CLI-TPS registration algorithms.

Fig. 11 shows plots of the intensity similarity cost, landmark error cost, and the maximum inverse consistency error costs as a function of iteration for CLI-TPS registration of data sets $B2$ and $B4$. The protocol used for this experiment was 5 iterations of the CL-TPS algorithm followed by 95 iterations of the CI-TPS algorithm. The intensity similarity cost decreases during the CI-TPS algorithm when the intensity is being matched and increases during the CL-TPS algorithm as the landmarks are matched. Conversely, the landmark error decreases during the CL-TPS algorithm and increases during CI-TPS algorithm as the intensity is matched. The plot of the maximum inverse consistency error shows that switching from the intensity (CI-TPS) to the landmark (CL-TPS) algorithm causes a jump in the inverse consistency error which is quickly minimized. We observed that smaller landmark and intensity error is achieved by the CLI-TPS in one-third the number of iterations than by either CI-TPS or CL-TPS alone.

Figure 11: Plots of the intensity and landmark costs as a function of iteration for the CLI-TPS registration of data-sets $B2$ and $B4$.

The lower-right panel of Fig. 11 shows the minimum and maximum Jacobian values of the forward and reverse transformations as a function of iteration. These plots show that the inverse consistency constraint (ICC) causes the minimum Jacobian value of the forward transformation to track with the inverse of the maximum Jacobian value of the reverse transformation and vice versa. Note that these plots give an upper bound on the inverse consistency error since the minimum and maximum Jacobian values of the forward and reverse transformations do not correspond to the same points.

Table III summarizes the representative statistics collected from the experiments. Comparing the results of the UL-TPS and CL-TPS algorithms shows that the addition of inverse consistency constraint (ICC) improved the inverse consistency of the transformations with no degradation of the landmark matching. Note that for the UL-TPS algorithm, the inverse consistency error tends to be be larger as one moves away from landmarks and that the inverse consistency error can be decreased by defining more corresponding landmarks.

0.75mm

Table III: Experimental results produced by mapping MRI brain image 2 into images 1, 3, 4, and 5 (see Fig. 7). The thin-plate spline algorithms compared in this table are the unidirectional landmark (UL-TPS), averaged unidirectional landmark (AUL-TPS), consistent landmark (CL-TPS), consistent intensity (CI-TPS), and consistent landmark and intensity (CLI-TPS) algorithms. The statistics computed for these experiments were the average landmark error (ALE) in pixels, maximum landmark error (MLE), maximum inverse error (MIE) in pixels, average inverse error (AIE) in pixels, masked average intensity difference (MAID), minimum Jacobian value (MJ), inverse of the maximum Jacobian value (IJ) and the Jacobian error (JE).

Algorithm	Exp.	ALE	MLE	AIE	MIE	MAID	MJ	IJ	JE
None	b2b1	6.9	12			0.23
	b2b3	4.9	13			0.19
	b2b4	8.8	21			0.22
	b2b5	8.7	19			0.26
UL-TPS	b2b1	0.066	0.087	0.90	2.7	0.16	0.56	0.75	0.053
	b2b3	0.073	0.098	0.78	3.1	0.18	0.50	0.57	0.092
	b2b4	0.062	0.088	0.94	3.4	0.13	0.51	0.66	0.090
	b2b5	0.030	0.061	1.2	3.8	0.16	0.56	0.67	0.050
AUL-TPS	b2b1	0.016	0.029	0.0057	0.13	0.16	0.59	0.73	0.00048
	b2b3	0.017	0.053	0.0066	0.10	0.18	0.55	0.53	0.0023
	b2b4	0.030	0.065	0.0096	0.22	0.13	0.54	0.62	0.0010
	b2b5	0.031	0.046	0.0096	0.12	0.16	0.56	0.62	0.0011
CL-TPS	b2b1	0.000030	0.00011	0.0012	0.028	0.16	0.59	0.73	0.0011
20 iter.	b2b3	0.000034	0.00014	0.0016	0.022	0.18	0.55	0.53	0.0014
	b2b4	0.0083	0.083	0.079	0.42	0.13	0.54	0.62	0.0011
	b2b5	0.000006	0.00037	0.0024	0.015	0.16	0.56	0.62	0.00021
CI-TPS	b2b1	1.5	3.1	0.0045	0.048	0.097	0.26	0.47	0.011
1000 iter.	b2b3	1.6	2.9	0.0043	0.052	0.11	0.25	0.29	0.017
	b2b4	1.0	2.2	0.0040	0.063	0.084	0.26	0.44	0.0075
	b2b5	1.4	3.4	0.0044	0.099	0.092	0.18	0.32	0.0091
CLI-TPS	b2b1	1.1	2.0	0.020	0.40	0.091	0.19	0.37	0.036
300 iter.	b2b3	1.1	2.0	0.021	0.62	0.10	0.13	0.23	0.030
	b2b4	0.75	1.6	0.017	0.61	0.080	0.12	0.39	0.025
	b2b5	1.1	2.8	0.021	0.96	0.088	0.10	0.17	0.034

Table III also demonstrates that the CI-TPS and CLI-TPS registrations have a smaller average intensity difference but larger landmark errors. The CLI-TPS has smaller average intensity difference and smaller landmark errors than the CI-TPS registration algorithm. The CLI-TPS algorithm produces a better similarity match because the landmark driving force pulls the intensity driving function out of local minima. It should be noted that the large number of landmarks used in the CLI-TPS registration limits the effect of the intensity driving force in neighborhoods of the landmarks. In practice, when the the landmark points are more sparse the intensity driving force plays a more important role.

Copyright © 2002 • The University of Iowa. All rights reserved. Iowa City, Iowa 52242
Questions or Comments: gary-christensen@uiowa.edu