Trifocal Tensor Based Feature Matching Algorithm

Mingwei Shao and Pan Wang

（School of Information and Control Engineering, Qingdao University of Technology, Qingdao 266000, Shandong, China）

Abstract: Feature matching is of significance in the field of computer vision. In this paper, a trifoc?al tensor based feature matching algorithm is proposed for three views, including a trinocular vis?ion system. Initial matching point?pairs can be determined according to generic matching al?gorithms, on which an initial trifocal tensor of three views can be confirmed. Then the initial matching point?pairs should be re?selected. Meanwhile, the trifocal tensor will be recomputed. Iter?atively, the optimized trifocal tensor can be obtained. Compatible fundamental matrix of every two views can be determined. Furthermore, in the trinocular vision sensor, the trifocal tensor can be cal?culated based on the intrinsic parameter matrix of each camera. With the strict constraint provided by the trifocal tensor, feature matching results will be optimized. Experiments show that our pro?posed algorithm has the characteristics of feasibility and precision.

Key words: optics；trifocal tensor；feature matching

Feature matching is important and complex in computer vision. It is utilized in many tasks such as three dimensional(3D) reconstruction,object recognition, image mosaicking, object tracking and so on[1?5]. Feature extraction is the basis of feature matching. For example, scale in?variant feature transform(SIFT) is very robust in many kinds of transformations such as rota?tion, shifting, scaling, affine, perspective and illu?mination changes[6]. In this case, the SIFT detect?or with other affine invariant feature detectors[7?8]has seen maturity in real applications. For fea?ture matching, algorithms such as RANSAC and alignment methods[9?10]normally have an iterat?ive form aimed to optimize a well?defined object?ive function. As the constraint is weak, the mis?match problem is inevitable. In 3D reconstruc?tion, such as stereo vision model, fundamental matrix is used to provide another constraint[11].In this case, feature matching is improved but still not with a strict enough constraint, particu?lar for a series of images.

In this paper, a feature matching algorithm is proposed for a series of images, which gives a strict constraint. We use a trifocal tensor to provide a strict constraint like the one provided by a fundamental matrix in a stereo vision mod?el.

1 Trifocal Tensor

For three image pairs, the relationship of these corresponding image points according to the trifocal tensor is[12?13]

Indices in Eq.(2) repeated in the contravari?ant and covariant positions imply summation over the range (1, 2, 3) of the index. When im?age projections of feature points are obtained, the trifocal tensor of three views can be determined easily based on the algebraic minimization al?gorithm.

When the tensor is determined, the compat?ible fundamental matrix of every two views,which satisfies the constraint of tensor, can be deduced as

Heretofore, there are many approaches to calculate the trifocal tensor, a typical one is the algebraic minimization algorithm. In practice,sphere target or planar target can be utilized to determine the trifocal tensor. Relationship between spherical features on the sphere target and the trifocal tensor can be obtained from Ref. [14], while feature points on the planar tar?get can be expressed as Eq. (1). Then the trifoc?al tensor of three views can be determined easily.

2 Feature Matching Algorithm

2.1 Feature matching algorithm for images(more than three) only

For any pair of matching points (x and x′)in two images, the relationship can be expressed by the fundamental matrix F

When we get enough candidate matching points by a related feature extraction algorithm,matching point?pair can be confirmed by the con?straint expressed in Eq. (6).

For a series of images (more than two im?ages), every two images are treated as an image pair traditionally. Fundamental matrices of every two views can be confirmed separately but in?compatibly. Mismatching points are inevitable.In this case, the trifocal tensor can give a stricter constraint than the fundamental matrix. Take a series of images including three images for ex?ample. When we get plenty of feature points in three images, we can get the initial matching point?pair based on generic algorithms such as multi?scale edge matching algorithm, shape based matching algorithm and so on[15]. Then we can deduce the initial trifocal tensor from Eq. (1)when we get enough matching point?pairs.

Based on the initial trifocal tensor, spurious matching point?pairs will be removed. Then we can get a new trifocal tensor from the reselected matching point?pairs. Finally, we can obtain an optimized trifocal tensor with loop iterations. In this step, a minimized cost function is given as according to Eq. (1)

2.2 Feature matching algorithm for images captured in trinocular vision sensor

3 Experiments and Discussions

Any three views (images) with an overlap?ping region can be selected to verify our feature matching algorithm. We built a trinocular vision sensor. Each camera is calibrated by Zhang’s cal?ibration method[16]. The intrinsic parameter matrices are

The obtained optimal tensor according to Algorithm 1 and the trifocal tensor deduced from Eq.(8) (Algorithm 2) are listed in Tab.1. Two solutions of the trifocal tensor are nearly the same.

As listed in Tab. 1, two solutions of the tri?focal tensor are nearly the same. Features in three images are light stripes which are projec?ted by laser projectors. Light stripes are extrac?ted based on Steger’s extraction algorithm[17].Feature matching results are illustrated in Fig. 1.The results illustrated in Fig. 1a are results us?ing the trifocal tensor obtained according to Al?gorithm 1, while results in Fig. 1b are using the trifocal tensor determined according to Algori?thm 2.

Tab. 1 Trifocal tensor deduced according to different algorithms

Fig. 1 Feature matching results based on trifocal tensor

Fig. 2 Matching results based on trifocal tensor

Tab. 2 Matching results according to fundamental matrix(pixels)

As listed in Tab. 2, feature matching results based on the traditional matching algorithm are not precise enough. The root meansquare (RMS)error of matching points is (0.07, 0.08) pixels. By contrast, feature matching results based on the trifocal tensor are precise.

4 Conclusion

In conclusion, a trifocal tensor based feature matching algorithm for three views is proposed.Compared with the generic matching algorithm,the trifocal tensor can provide a strict constraint.In this case, the matching precision is improved.Experiments are conducted to verify the feasibil?ity and precision of our proposed algorithm. The algorithm is significant for any three views with an overlapping region, especially for a trinocular vision sensor.