Elec4622 Laboratory Project 3, T2 2024

July 13, 2024

1 Introduction

This is the third project, to be demonstrated and assessed within the regular scheduled laboratory session in Week 10. The project is worth a nominal 10 marks, but optional bonus marks are available.

In this project, you explore various aspects of block-based motion estimation, including sub-pixel precision motion and telescopic motion searches. These topics were covered in Week 8. You should review the lecture notes on motion estimation and also download and study the materials entitled "Lab 5" on the class web-site, since these provide you with an introduction to block-based motion estimation.

2 Tasks

Task 1: (5 marks) In your own time, complete the exercises in "Lab 5" - see the class web-site. In particular, you must complete the modifications to the "motion example" workspace, which are required to produce a colour output image which simultaneously shows the target frame. and the motion vectors

    Note that your program for this task needs to accept a block size parameter B and a searchrange parameter S, such that motion search range is ±S pixels in each direction.

    You should also be prepared to comment on (and demonstrate) the impact of different blocksizes and search ranges, on motion compensated MSE and the motion vector field.

Task 2: (1 mark) Modify the search criterion used to find motion vectors, so that the best vector isconsidered to be that which minimizes MSE over the block, rather than SAD. What impact does this have upon motion compensated MSE and the motion vector field? Can you find a good explanation for your observations?

Task 3: (3 marks) Modify the code from Task 2 to work with half pixel motion precision, using bi-linear interpolation of the reference frame.

    What impact does this have upon motion compensated MSE?

Task 4: (1 mark) Extend Task 3 to allow quarter pixel motion precision and comment on the implications for motion compensated MSE.

Task 5 (optional): (up to 1 bonus mark) Extend Task 4 to use windowed sinc interpolation with atl least 7-tap interpolation kernels, in place of bilinear interpolation

    What impact does this have on the motion compensated MSE?

Task 6 (optional): (up to 3 bonus marks) Modify Task 3 to use a telescopic search. Specifically, your program should initially search over a coarse grid whose motion vectors v are multiples of 4, still constrained to the search range of ±S in each direction. After that, your program should perform. an incremental search at pixel precision, modifying the coarse motion vectors by at most ±3 pixels in the horizontal and vertical directions. Finally, a half pixel refinement stage should be employed, modifying the full-pel motion vectors by at most ± in each direction.

    Time this telescopic search method and compare it with the original method of Task 3; be sure to use release builds for all timing.

    Also investigate the impact of the telescopic search on motion compensated MSE-you should be prepared to try to explain your observations.

Task 7 (optional): (up to 3 bonus marks) Modify Task 3 to use a multi-scale search with 3 scales. Specifically, you should use the Gaussian pyramid from Project-2 (based on windowed-sinc downsampling) to create representations of the two frames at 4/1 resolution and 2/1 resolution, in addition to the original resolution. Starting with the 4/1 resolution versions of the two frames, perform. a full-pixel search, for which the search range is [S/4] at the reduced resolution. After that, your program should double all the block motion vectors found from the 4/1 resolution search and perform. an incremental search within the 2/1 resolution versions of the two frames, modifying these doubled block motion vectors by at most ±2 pixels in each direction. Finally, the 2/1, resolution motion vectors should be doubled and used as the starting point for an incremental 2/1 pixel precision search at full resolution, modifying the doubled block motion vectors by at most ±1 2/1, pixels in each direction.

    The block size B for this program should be constrained to be a multiple of 4

    For the resolution search, the separation between adjacent blocks is only B/4 pixels, but you are recommended to extend the nominal 4/B × 4/B blocks by 1 pixel on each side, so that they overlap; in this way the 4/1 resolution block motion search always uses at least 9 pixels for block matching, rather than just 1 pixel, in the extreme case where B = 4.

    Time the telescopic search method and compare it with the oriignal method of Task 3; be sure to use release builds for all timing.

    Also investigate the impact of the telescopic search on motion compensated MSE- you should be prepared to try to explain your observations.

    It is expected that most students will attemp only one of Task 6 or Task 7, but if you do attempt both it will be instructive to compare the results you obtain in each case.

3 Assessment

You should not rely upon implementing this project within the scheduled laboratory sessions. Instead, youmust be prepared to demonstrate and explain your work in the Week 10 laboratory. You should make sureeach task can be run simply from the command-line.

    Note carefully: Lab demonstrators will expect to see the hand-drawn sketches you haveproduced as a critical part of the design process.

3.1 Team work, plagiarism and copying

You may feel free to re-use code from the previous laboratory sessions, so long as you understand it. You may also discuss the project with other students in the class, but your programs should otherwise be your own original work - this is not a group project!

You are required to submit your code via an Assignment item on the course's Moodle page, by the end of the same day as your scheduled laboratory session, following the instructions provided there. Your code may be cross-checked for plagiarism, so make sure that you do not copy any other student's actual implementation, or base your solution on one that you obtain from another student.

3.2 Managing the limited resource of demonstrator time

During your labs, demonstrators will have a major responsibility of marking your project. This is time consuming, and so you cannot expect to be marked only in the last hour of a lab session. To maximize your opportunity to be marked you should come prepared to the lab session in Week 10 with many elements of your project completed or at least partially working, so that you can ask a demonstrator to mark or look over your solution as early as possible within the lab session.