Aspect Ratios

In this blog entry, I am venturing into deep water without a life jacket, because I'm not an encoder, and I don't play one on TV. To keep things simple, I'm only going to discuss US standards (NTSC). European standards (PAL and Secam) only make matters worse.

The aspect ratio is the ratio between the number of pixels horizontally and the number of pixels vertically. This can be expressed as the actual number of pixels (e.g., 1280 x 720), as a ratio of whole numbers (e.g, 16:9), or as a ratio against one (e.g., 1.77:1). The last one is only approximate, because the decimal digits repeat indefinitely.

Up until the 1950s, aspect ratios were fixed at 4:3 (1.33:1), the so-called Academy Ratio, because everything was done on 16mm or 35mm film stock, and the frame size was 1.33:1. In the early 1950s, the movie industry created a raft of wide-screen formats - Cinemascope, VistaVision, Panavision, etc - at a variety of aspect ratios, including 2.40:1, 1.85:1, and 1.66:1. Some of these were done by anamorphically stretching the 35mm film stock, using special lenses; others used wider film stock, such as 65mm or 70mm. TV production retained 1.33:1 until this century. Wide-screen content such as movies had to be presented by one of two techniques - letterboxing, which showed the wide-screen image on a subset of the TV screen, with black bars and top and bottom; or pan-and-scan, which moved the viewpoint around to show the "essential" part of wide-screen image on the full TV screen.

The Japanese analog home-media formats - VHS tape and laserdiscs - were invented when all TVs were 1.33:1, and they basically copied the NTSC standard. Thus, it was fairly common for VHS tapes of wide-screen movies to be offered in two versions, letterboxed and pan-and-scan. (Laserdiscs were more typically letterboxed, because TV screens were bigger by then.) For fansubbers, then, there is (or should be) no issue about the aspect ratio of VHS and laserdisc-sourced material - it's 4:3 (1.33:1) prior to any cropping or upscaling.

Starting in 1984, Dr. Kerns Powers of the SMPTE Working Group on High-Definition Electronic Production proposed 16:9 (1.77:1) as a suitable wide-screen compromise between the US de-facto standard (1.85:1), the European standard (1.66:1), and the older Academy ratio (1.33:1). Driven by the EU, the proposal gained traction and became the standard for HDTV broadcasts, at 1280 x 720 or 1920 x 1080. And in the mid-1990s, the proposal was sufficiently far advanced that it was supported by the spanking new DVD standard.

DVDs can be considered an intermediate format, looking backward to the still-predominant 4:3 standard and forward to the forthcoming 16:9 standard. The actual DVD resolution was a compromise: 720 x 480 (1.5:1). Metadata on the DVD would tell a player whether to "stretch" the resolution anamorphically to 16:9 (1.77:1) or "squeeze" it to 4:3 (1.33:1). When TVs evolved to a 16:9 aspect ratio, the players had to handle four cases:

1. 16:9 display on a 16:9 TV.
2. 16:9 display on a 4:3 TV.
3. 4:3 display on a 16:9 TV.
4. 4:3 display on a 4:3 TV.

Blu-ray did away with the anamorphic mess. Blu-rays are always 16:9, and the content is letterboxed, pillarboxed, or cropped to fit. Modern web streams are the same.

However, DVDs had more than enough rope for fansubbers to hang themselves and six of their best friends too. A common mistake was to take the 720 x 480 (3:2) ratio as correct. The caused wide-screen material to look squashed and Academy ratio material to look stretched. This can be fixed pretty easily in players or by creating a Matroshka mux with the proper aspect ratio set.

Another mistake was to assume that the wide-screen ratio was 1.85:1. This was based on an unconscious assumption about the universal acceptance of US standards, as well as a set of arguments about pixel shape, but in fact, it's not what the standard says. A different justification was that 1.85:1 movies on DVD played on a wide-screen TV without letterboxing. In fact, there is letterboxing, but it falls in the overscan area and can't be seen. If the TV can turn off overscan, the letterboxing becomes visible, and a bit more of the frame appears at the edges. Encoding at 1.85:1 made anime looked subtly stretched. It wasn't as bad as the previous case - it's only 5% - but it's annoying if you're familiar with a DTV or HDTV broadcast of the same material.

Fansub encoders found a clever way out of this mess: they encoded at 720 x 480 and then set the display aspect ratio as part of Matroshka mux. That way, it was Not Their Problem. However, it created headaches for other members of the team, particularly the stylist and typesetter, who had to account for the stretch or squeeze in their work. Anamorphic encodes have been the bane of my existence as a typesetter.

At this point, you've probably gone "tl;dr" and are wondering what this has to do with Orphan, which works mostly from analog media anyway. Analog media are 4:3, so no headaches, right? Wrong. The DVD format has created problems in the handling of analog media, because analog captures were and are often distributed in DVD ISO format. That has led to mistakes about the origins of the material ("it's a DVD") and how to process it.

• aarinfantasy's groundbreaking release of Zetsuai 1989 used an ISO image of a laserdisc capture and assumed the dimensions were correct. After cropping the pillarboxing, the encode was 700 x 480 - too wide - which actually made the ultra-narrow character designs more reasonable.
• Toy-RC's groundbreaking release of Call Me Tonight also used an ISO image of a laserdisc capture. The encode used vertical expansion to correct the ratio (720 x 540) and is thus technically a 12.5% upscale.

More serious issues have arisen with letterboxed widescreen releases on laserdisc. The obvious way to deal with letterboxed laserdiscs is to encode them at 640 x 480, letterboxing and all. That's what Erik of Piyo Piyo Productions does routinely. I like the approach, because the letterboxing provides a nice place to put the subtitles without infringing on the image. Another simple approach is to crop the image exactly as shown, to 640 x whatever. That too is faithful to the original source.

However, sometimes encoders want to create a normal 480p (vertical) resolution encode, which involves upscaling. That poses a problem: what should the horizontal resolution be? Again, there's a simple answer: measure the original image to get the aspect ratio and upscale accordingly. However, sometimes the groups plump for the US widescreen standard (1.85:1), on the theory that the laserdisc must have been scaled wrong. However, that's incorrect. Japan did not use the US widescreen ratio of 1.85:1 but rather the "VistaVision" (aka European "flat") ratio of 5:3 (1.66:1).  It later showed up in Japan's first HDTV standard (Hi-Vision).

The result is that upscaled versions of widescreen Japanese movies sourced from laserdiscs can have incorrect aspect ratios. Orphan's first version of Hashire, Melos! was 1.85:1, because the encoder thought that's what widescreen movies had to be. As a result, it's too wide: more than 10% stretched horizontally. This can be seen by comparing the first version, done with "anamorphic stretch":

 

versus the second, which was done at standard laserdisc resolution with letterboxing intact:

It's subtle, but the faces in the first version are too wide.

A similar problem occurs in Toy-RC's B.B. Fish. It is encoded (non-anamorphically) at 1.77:1 (16:9), as though it were a DVD. Looking at an untouched frame of the laserdisc output, with letterboxing intact, shows that it's actually 1.66:1 (15:9). The laserdisc jacket even says "Vista version." Orphan's first attempt at encoding the widescreen release of Hashire! Ookami Shiroi was also too wide, as a comparison with the 1.33:1 (4:3) US dub clearly showed. The widescreen version is actually 1.66:1 (15:9).

By now, your eyes have probably glazed over (mine have). The bottom line is twofold:

• A DVD ISO does not mean a DVD source. The DVD ISO format is just a container. The encode has to be aware of the actual source media.
• A Japanese media property mastered in the 70s and 80s will conform to the Japanese standards of the period, not to modern standards.

Being cognizant of these two observations will save a world of problems in encoding analog media.