Sampling Moiré method for full-field deformation measurement: A brief review

Qinghua Wang , Shien Ri

Research Institute for Measurement and Analytical Instrumentation, National Metrology Institute of Japan (NMIJ), National Institute of Advanced Industrial Science and Technology (AIST), 1-1-1 Umezono, Tsukuba 305-8568, Japan

ABSTRACT The sampling Moiré (SM) method is one of the vision-based non-contact deformation measurement methods, which is a powerful tool for structural health monitoring and elucidation of damage mechanisms of materials. In this review, the basic principle of the SM method for measuring the twodimensional displacement and strain distributions is introduced. When the grid is not a standard orthogonal grating and cracks exist on the specimen surface, the measurement methods are also stated.Two of the most typical application examples are described in detail. One is the dynamic deflection measurement of a large-scale concrete bridge, and the other is the residual thermal strain measurement of small-scale flip chip packages. Several further development points of this method are pointed out. The SM method is expected to be used for deformation measurement of various structures and materials for residual stress evaluation, crack location prediction, and crack growth evaluation on broad scales.

1. Introduction

Full-field deformation measurement is indispensable for the performance evaluation of materials and structures. Among the non-contact optical measurement methods, in addition to the wellknown digital image correlation (DIC) method, the sampling Moiré(SM) method is another vision-based deformation measurement technique [1–3] . The SM method uses a periodic pattern such as a grating or a grid as the deformation carrier and obtains the deformation distributions by performing accurate phase analysis of a single image based on phase-shifting [ 4 , 5 ]. Like Fourier transform (FT) [6] , windowed Fourier transform (WFT) [7] , spatial carrier phase-shifting (SCPS) [8] , digital Moiré [9] and wavelet transform [10] , the SM method also belongs to fringe phase analysis techniques. This method can also be seen as a special WFT algorithm [ 11 , 12 ].

Since the SM method was developed, its parameter selection and measurement errors have been studied [13] . The accuracy of phase difference measurement is from 1/100 to 1/10 0 0 of the grating pitch in the SM method [ 13 , 14 ]. Simulation results show that the relative error of strain measurement is less than 1% when the theoretical strain ranges from 100 to 50 0 0 με[15] . This method has been extensively used for displacement [16–19] , strain [ 20 , 21 ],and shape [22–25] measurements of a wide range of structures and materials. This method also has applications in flow measurement [26] , transparent liquid surface topography [27] and digital holography [ 28 , 29 ]. In recent years, a number of derivative methods of the SM method have been gradually developed, expanding the application areas of this technique.

Two-dimensional (2D) Moiré phase analysis [15] is essentially a combination method of the SM method and geometric phase analysis (GPA), which works well in displacement and strain measurement when the grating’s principal direction is not parallel to the analysis direction. 2D Moiré phase analysis has been applied to strain measurements of Ti-6Al-4V alloys [ 30 , 31 ] and carbon fiber reinforced plastics (CFRPs) [32] . The strain concentration detected before crack occurrence has been successfully used to predict the crack emergence zone [15] .

The second-order Moiré method [33] applies the SM method at two stages, so that it can effectively reduce periodic errors of phase measurement and has very strong noise immunity, which has been used for strain mapping of CFRPs under three-point bending. The scanning-based second-order Moiré method [34] utilizes the SM method to analyze the phase of the scanning Moiré fringes, and thus has the advantages of both the large field of view of scanning Moiré and the high accuracy of phase analysis, which has been verified in strain measurement of aluminum.

Fig. 1. Phase measurement principle of the SM method for a single-shot grating pattern. Reproduced with permission from [15] .

The orthogonal SM method [35] performs phase shifts in both thexandydirections simultaneously to obtain the 2D phase of a grid, which is then separated into two phases in two directions for deformation measurement. This method has been used for measuring the nominal strains of a polymer film and the crack opening displacements. Moreover, the multiplication SM method [36] can efficaciously weaken the effect of complex noise by extracting the second harmonic frequency of a grating, which is deeply suitable for displacement and strain measurement of composite materials such as CFRPs [37] .

The subpixel SM method [ 27 , 38 ] uses sub-pixel phase shift steps for phase analysis, which can reduce the periodic errors of the deformation measurement. Besides, the spatiotemporal phaseshifting method (ST-PSM) [ 39 , 40 ] uses 2D intensities in both the spatial- and temporal-domains to extract the phase information of fringe patterns and can also effectively attenuate the periodic errors of the phase analysis.

In this review, we will introduce the deformation measurement principle and the latest applications of the SM method. In the section introducing the measurement principle, details that need to be paid attention to during deformation measurement are also presented, such as how to calculate deformation when the specimen grid is not a standard cross grid and when there are cracks or nonnegligible defects on the specimen surface. In the sections about the applications, two of the most representative applications are described in detail, including deflection measurement of a bridge and residual strain measurement inside the flip chip packages.

2. Deformation measurement principle of SM

A spatial phase-shifting technique can be used to digitally analyze the phase information of a grid or grating pattern for deformation measurement. For 2D deformation measurement, the adopted deformation carrier is usually a 2D cross grid. Using a low-pass filter such as a moving average filter, a kernel filter or a FT filter, two parallel gratings can be extracted from a 2D grid image.

Suppose the principal direction of a grating is close to theydirection (vertical), and the grating pitch ispy before deformation in theydirection ( Fig. 1 ). The intensity of this grating image shown in Fig. 1 b can be expressed as

Based on the linear relationships between the displacements and the grating phase differences, i.e.,Δ?x(x,y) = 2 πux/px,Δ?y(x,y) = 2 πuy/py, the 2D displacements in thexandydirections can be acquired using

Fig. 2. Two-dimensional strain measurement process. Reproduced with permission from Ref. [32] .

However, when there are nonnegligible cracks or holes or large flaws on the surface of the specimen, phase unwrapping is not easily executed correctly. To avoid global phase unwrapping, the strain distributions can be directly measured from the partial derivatives of the initially wrapped phase differences [31] ( Fig. 2 ), where the partial derivatives are corrected by a local phase unwrapping algorithm using Eqs. (12) and (13) in Ref. [31] . The partial derivative of the phase difference in either direction is corrected by subtracting π when it is greater thannπ, or adding π when it is less than -nπ,wherenis a fractional coefficient and usually chosen within (0.7,1). After the partial differentials of phase differences in both directions are corrected, the normal strains and the shear strain can be determined using Eq. (7) without phase unwrapping of the whole image.

Furthermore, in the general case where the principal directions of the specimen grid are not exactly parallel to the analysis directions, the 2D displacements and strains can be calculated using Eqs. (25) and (26) in Ref. [15] by considering the effect the rotation angle. As for a broader case, when the fabricated grid is not a standard cross grid, i.e., the two principal directions are not perpendicular, and/or the grating pitches in different principal directions are not equal, the 2D displacements and strains are measurable using Eqs. (21) and (22) in Ref. [15] suitable for deformation measurement of arbitrary 2D periodic structures.

3. Applications in displacement measurement

The SM method has been extensively used for displacement measurements of structures and materials, such as highway and railway bridges [ 41 , 42 ], pipes [43] , rockets [44] , etc. The displacement information can be further used to evaluate the vibration[45] , deflection [42] , rotation angles [46] , crack opening conditions [47] and other characteristics of structures. Using 2D grids,the displacement of the research object can be measured in two directions simultaneously. With the help of high-speed cameras,dynamic displacement curves over time can be obtained by this method. Besides, using the DIC-aided SM method [48] , displacements exceeding half of the grating pitch can also be measured correctly.

As a representative example, we are about to present the deflection measurement of a concrete bridge when a train passes at high speed [42] . The photos of a concrete bridge located in Japan and the used camera as well as grid markers for displacement measurement at daytime and nighttime are shown in Fig. 3 .The bridge width was 12 m and the span length was 30 m. Four grid markers (Moiré markers) with a pitch of 50 mm were pasted on one sidewall of the bridge using an adhesive tape. Four highpower light emitting diode (LED) lights were used to illuminate each retroreflective grid markers at night. The angle between the LED lights and the camera was about 15 °. The LED lights were located closer to the bridge than the camera.

When a Japanese high-speed train called ‘Shinkansen’ passed through the bridge at 320 km/h, the grid images were recorded by a complementary metal oxide semiconductor (CMOS) camera(Basler, acA4096-40 μm, 4096 × 2168 pixels) with a C-mount 25 mm fixed focal length lens (RICOH, FLBC2518-9M) at a frame rate of 20 fps. The perpendicular distance between the camera and the bridge was around 23.5 m.

As the grid pitch on the recorded image was 11 pixels at the 1/2 point and 13 pixels at the fixed point, the sampling pitch was set as 11 and 13 pixels at the 1/2 point and the fixed point, respectively. To weaken the influence of camera vibration, the relative displacement of the center position (1/2 point) and the fixed point (0 point) was measured ( Fig. 4 ). The maximum downward displacement of the bridge center was about 1mm and the displacement was closely related to each car of the train.

Fig. 3. Experimental setup for deflection measurement of a concrete bridge: ( a ) daytime and ( b ) nighttime site photos; marker images of ( c ) day and ( d ) night. Reproduced with permission from Ref. [42] .

Besides, the displacements at the 1/2 point measured by the SM method with those from a 632 nm He-Ne laser Doppler vibrometer (U Doppler II, Railway Technical Research Institute) were compared at daytime and nighttime ( Fig. 4 c,d). The displacement trends measured from the SM method agreed well with those from the traditional laser Doppler vibrometer [42] . The difference in displacement values is mainly due to the influences of the camera’s random noise and atmospheric disturbances.

4. Applications in strain measurement

In addition to displacement measurement, the SM method and its derivative methods have also been widely applied to strain measurements of various materials and structures, including metals [31] , alloys [49–51] , CFRPs [ 32 , 52 , 53 ], semiconductors [21] , and atomic arrays [ 54 , 55 ], etc. These strain results are further used to detect defects [ 54 , 56 ], predict cracks, evaluate fatigue properties,calibrate microscope distortions [ 57 , 58 ], determine Young’s moduli [59] and thermal expansion coefficients [60] , etc. If the background noise is particularly strong, the fast Fourier transform filtered SM method [ 55 , 61 ] or the multiplication SM method can be used.

Fig. 5. Experimental setup for residual thermal strain measurement: ( a ) flip chip package; ( b ) microscope image of the cross section; ( c ) enlarged grid image; ( d ) thermal chamber under a microscope; ( e ) diagram of specimen clamping. Reproduced with permission from Ref. [21] .

Besides strain measurement, the SM method has been combined with an inversed problem for residual strain measurement[21] . If the internal stress and strain of the sample are assumed to be zero at the specimen formation temperature, the internal strain at different tem peratures is called residual thermal strain.The residual thermal strains including the normal and shear strains at an arbitrary temperature relative to the specimen formation temperature can be quantitatively measured. If the coefficient of thermal expansion is known, the residual thermal strain distributions relative to the free-shrinking state at a given temperature can also be measured.

Here we present a typical example of residual strain measurement in the micro regions of flip chip packages (FCPKGs) [21] .Grids with 3 μm pitch were fabricated on the cross-section of two FCPKGs at room temperature using a clamping jig installed in an ultraviolet nanoimprinting system (EUV-4200). Both FCPKGs were heated to the specimen formation temperature 150 °C, by a specially designed thermal chamber ( Fig. 5 ) that allowed heating tests with a laser microscope (Lasertec, Optelics Hybrid). The grid images (1024 × 1024 pixels) during the heating test were recordedin situwith an objective lens of 20 ×, and the grid pitch is around 8 pixels on the image.

The residual strain distributions at 125, 75 and 25 °C of FCPKGs relative to 150 °C were measured using the SM method with a sampling pitch of 8 pixels and the inverse problem. The residual thermal strain distributions at 25 °C of FCPKG containing underfill (UF-A) with low glass transition temperature (Tg) are illustrated in Fig. 6 a. The absolute value of the residual strain of UF-A in thexdirection is maximum on the right side of the die, especially around the corner of the die. The absolute values of both the residual strain in theydirection and the residual shear strain are highest below the die and around the die corner.

From the residual normal and shear strain distributions, the maximum and minimum residual principal strain distributions of FCPKGs containing two different types of underfills with different Tg were also measured ( Fig. 6 b). The residual thermal principal strain of FCPKG containing UF-A withTg= 345 K had large absolute values around the die, especially at the die corners. On the other hand, the strain concentration of FCPKG containing UFB withTg= 380 K was mainly observed in the buffer layer. Near the die corner, the maximum residual principal strain of UF-A was greater than that of UF-B.

By considering the coefficient of thermal expansion of underfills, the residual principal strain distributions relative to the freeshrinking states of FCPKGs at room temperature were also measured [21] . It was confirmed that the trend of the residual thermal strain distribution obtained experimentally was consistent with that obtained by the finite element method.

5. Conclusion

The SM method is a full-field deformation measurement method developed in recent years by performing phase analysis on digital grating images. As long as the grating images can be recorded, this method is suitable for the deformation measurement of a wide range of structures and materials from the atomic scale to the kilometer scale. Two typical applications including the dynamic deflection measurement of a concrete bridge and the residual thermal strain measurement inside flip chip packages, show this method’s practicality and usefulness. In recent

Fig. 6. Residual thermal strain distributions of flip chip packages (FCPKGs): ( a ) normal and shear strain distributions of FCPKG with UF-A; ( b ) comparison of the maximum and minimum principal strain distributions of FCPKGs with UF-A and UF-B. Reproduced with permission from Ref. [21] .

years, several derivative methods of SM have been developed, such as 2D Moiré phase analysis, second-order Moiré, subpixel SM, orthogonal SM, multiplication SM, and spatiotemporal phase-shifting.Further development points of the SM method include reducing periodic errors, decreasing the effect of background noise, weakening the influence of environmental disturbances, extending to three-dimensional deformation measurement, improving grating production techniques, etc.

Declaration of Competing InterestThe authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgment

This work was supported by Japan Society for the Promotion of Science (JSPS) KAKENHI (Grant Nos. JP20K04171 and JP20H02038 ).