We have seen interpolation in the news concerning a recent court case. Here we cover what interpolation does to an image, not only because of the recent news but also because face recognition uses interpolation to better recognize a face – something we have covered in the past.
Interpolation means to take pixels in an image and calculate what their color might be when the image is resized. Many image processing libraries use bicubic interpolation to increase details on the face, which is the same bicubic interpolation talked about in the case. In the case of the Rittenhouse trial, we don’t know for sure if evidence provided to the jury used bicubic interpolation though.
For you to understand bicubic interpolation we must step back to linear interpolation. We perform linear interpolation all the time on graphs by extrapolating data between two datapoints. We can run into problems though if the data points are too far apart as seen below.
We can compensate for this error in the graph using cubic interpolation by inspecting not just the nearest data points but the next nearest data points. We take four data points total to determine the number we wish to find.
Pictures aren’t one-dimensional but two-dimensional, so we must account for both directions. This leads us to bicubic interpolation. To increase the resolution of a picture, we take the 16 pixels (4 pixels x 4 pixels) closest to the area we wish to expand and determine what color to put in the middle.
Problems can arise if we try to expand the picture too much. Whenever performing interpolation, we could add artifacts to the picture, so we limit on how far we expand a picture. This makes using bicubic interpolation on a picture to determine what happened in that picture unreliable in a trial setting. It can also cause problems in face recognition, but we can get around this by inspecting multiple areas on a face. If one area of the face become corrupted because of interpolation we can still match other areas of the face to determine if a face will match. This works because you don’t have a very high chance of getting one right, but you do have a high chance of getting at least one of many right. Like flipping a coin, if on one flip I bet heads, I have a 50% chance of winning the bet. If I bet out of 10 flips, I get at least one heads I have a 1023/1024 chance of winning that bet.
We must understand the interpolation algorithms we use before we start using them in critical areas like court trials. One single mistake from interpolation of an image in a trial can cause significant harm. Also, those operating face recognition software must also understand interpolation even though it doesn’t have as much of an impact.