Lossless rotation of JPEG images: There's more than one way to rotate a cat

90 degree rotation of an image. What can be easier? This is one of the simplest transformations - pixels just have to be moved around in a very simple way without being changed. No complex math, no averaging, no interpolation.

However, when it comes down to rotating JPEG, things start look more complicated. Suppose we want to rotate the following JPEG image:

Way 1. Lossy rotation of JPEG image

The conventional approach of JPEG modification would require decompressing JPEG into a bitmap, rotating the resulting bitmap, and re-compressing it back to JPEG.

The biggest disadvantage of such approach is degradation in quality of the image. Even though we recompressed the JPEG image with exactly same quality settings as the original image, we can see a progressive loss in image quality. As JPEG artifacts accumulate with each re-compression, the rotated image looks more "noisy":

Lossless rotation of JPEG

JPEG images are stored as a number of relatively small independently encoded rectangular blocks called Minimum Coded Units (MCU). Typical dimensions of MCU blocks are 8x8, 8x16, 16x8 and 16x16 pixels. You can often see boundaries between MCUs in low-quality JPEG images.

Fortunately enough, MCUs can be rotated in 90 degree steps or flipped without having to be decompressed into bitmap and re-compressed back to JPEG. This means that JPEG images can be rotated in 90 degree steps and flipped without loss in quality.

However, there is a small catch to lossless rotation - possible change in image dimensions.

In lossless operations an image is treated as a collection of MCU blocks rather than individual pixels (in fact, the image is never decoded to pixels for lossless transformations).

The following pictures show how pixels are grouped into blocks in our image:

As we can see, the blocks in the right column and in the bottom row are not complete, since the image width (190px) and height (234px) are not multiples of the MCU width and height (16px). Further we will refer to such images as having "odd" dimensions.

However, the JPEG algorithm cannot manipulate incomplete MCUs. Prior to compressing a bitmap JPEG compression libraries pad incomplete blocks to full blocks with whatever pixels they see fit. The common practice is to fill these incomplete parts with pixels similar to pixels in the last row (column), or by repeating last pixels. This approach allows to minimize JPEG artifacts and to reduce compressed size. MCUs stored inside JPEG files are always complete, applications decompressing JPEG (viewers, editors, etc.) just do not show extra pixels used to pad MCUs. Here is the full image contained within our JPEG file:

As mentioned above, applications decompressing JPEG images ignore extra pixels in incomplete blocks on the right side and at the bottom of a JPEG image. However, MCU blocks on the left side and at the top can only be shown in full. There is no way to instruct decompressor to hide extra pixels at these sides.

This introduces a problem with lossless rotation of images having "odd" dimensions. If we rotate our image 90 degree clockwise, incomplete blocks from the bottom end up on the left side.

Here are several of possible approaches to lossless rotation and flip of JPEG images:

Way 2. Lossless rotation of JPEG images, discarding partial blocks

The most common approach to lossless rotation / flip of JPEG images is to discard partial MCU blocks that end up on the left and on the top side of an image. In this case, if the image has "odd" dimensions, several rows or columns of pixels get lost. As we can see in our example we lose a column of 10 pixels (1900 pixels in total) on the left during rotation. As a result our 190x234 image becomes 224x190. On the bright side we can see that no JPEG degradation happened during the rotation.

If we rotate the image another 90 degree clockwise, another dimension gets rounded down to a multiple of the MCU size and we end up with a 176x224 image losing 3136 pixels more:

Though we avoided JPEG degradation, the result looks unpleasant.

Way 3. Lossless rotation of JPEG images, preserving partial blocks

Is there any way to avoid both the JPEG degradation and the loss of partial blocks? The answer is "Yes".

If we do not want to discard partial blocks we can preserve them. The only catch is we can only show them in full. In this case "odd" dimensions will grow during rotation and we will see extra pixels that were hidden before rotation. For example, our 190x234 image becomes the following 240x190 image:

Way 4. Lossless rotation of JPEG images, padding with color

What if we want to preserve partial blocks but do not like the look of those extra pixels?

A possible solution is to replace the extra pixels with the pixels we like. In our example we replace extra pixels with the white color:

The image dimensions increase and partial blocks have to be recompressed, since some pixels in them were replaced. The rest of the image suffers no degradation:

Which option of JPEG rotation is the best?

Like with so many questions in this life there is no absolute answer that would perfectly fit all situations. Sometimes you don't care about several pixels being lost, sometimes you want to save as many of them as there are there. In BetterJPEG we just decided to offer you the whole flexibility of choice.

External links

JPEG Lossless Rotation or Flip with Partial MCU