Showing posts with label typesetting. Show all posts
Showing posts with label typesetting. Show all posts

Friday, February 3, 2023

It's Broken, Broken, Broken

For some time, I've wondered why anime groups have taken to specifying exactly which video players they support for their releases. Some say "mpv only". Some specify others. I've finally figured out why. When it comes to compatible subtitle rendering, the fansub world is broken, broken, broken.

In the old days, most anime subbers and fans were on Windows, and most everyone used MPC-HC (media player classic home cinema) for playback and xyvsfilter for subtitle rendering. The Aegisub and CCCP projects helped enforce this by providing a uniform tool chain for the Windows world to use. Then, in short order, Aegisub stopped publishing stable releases, CCCP went out of business, and chaos ensued.

The root of the issue, as far as I can tell, is the substitution of  the portable subtitle rendering library libass for the Windows only library xyvsfilter. Using a portable library simplifies the process of producing portable players, like mpv and VLC. libass is pickier about syntax, and mistakes that xyvsfilter will accept (for example, \pos(x,y\fs40) cause libass to produce gibberish. There's no syntax checker for ASS that I know of, so it's all too easy for errors to slip by. But more importantly, libass is simply wrong in places, mostly notably in its handling of line wrapping.

ASS allows for four wrapping styles for multi-line subs, of which the default is 0: smart wrapping, top line longer. All scripts I edit are built that way. So I was baffled why the QCs for Yawara! kept asking for line breaks to be added or moved, until I found a smoking gun. Here is Aegisub's rendering of a scene from episode 72, with xyvsfilter:


And here is the same scene, with libass:

Now, I'm willing to allow some leeway about "smart wrapping", but mode 0 says top line longer. It doesn't say "top line longer unless the difference exceeds 'n' characters." In fact, libass is doing some very stupid wrapping in this case, breaking off the last word of a complete sentence. This isn't implementation leeway on an ambiguous spec. It's a bug, and it should be fixed. If I, as the editor and styling, specify top line longer, then that's what the renderer should so, unless it simply can't work due to hitting the margins.

I'm sure the younger set would tell me to give up on my ancient tool set and move to the "modern" world. However, I'm very set in my ways. I have a set of subtitling tools that work the way I like. If I were using libass, I'd be doing a lot more work to defeat this bug. In fact, I'd have to put explicit line breaks in every damned multi-line sub. Life's too short.

So here's Orphan's policy declaration. Our releases are supported on MPC-HC with xyvsfilter. If you want to use some other combination of playback and rendering, caveat emptor.


Monday, July 20, 2015

Typesetting Shirokuma Cafe

That which doesn't kill us makes us stronger.
                                Friedrich Nietzsche

If Nietzsche is to be believed, I'm a stronger typesetter than I used to be, but it was a near-run thing. It's taken about three months to typeset the whole show, and there were many times I was ready to throw in the towel and go back to Yawara!-style {\an8} signage.

Shirokuma Café is a very sign-heavy show. I think this is a direct consequence of its low budget. Stationary characters talking against static backgrounds don't give the eye much to do. (That's why the show works so well as a radio drama, once you’ve internalized the characters' images.) Adding signs is an inexpensive way to make the backgrounds more interesting. In addition, signs provide a way to add some humor for the adults watching the show, as I'll describe later. Accordingly, the signs are a vital part of the show's visual style; at least, some of them are.

Which Signs Matter?

In a show that's using signs as enrichment for backgrounds, not all signs are going to be important. In fact, most of them won't be.  The problem, of course, is how to know which ones do matter. The only real option is to translate them all and then decide.

The original translators provided the first clues. They translated about half the signs, and the ones that they translated are generally significant, usually for underlining a joke. (The endless puns are often reinforced with signs.) Unfortunately, the untranslated half can also matter.

When I started, I had to take a screenshot of every untranslated sign and then pester a translator to tell me what it meant. (Many thanks to convexity, deltakei, and Moho for putting up with this.) However, as I went along, I realized that the vast majority of the signs were in hiragana or katakana rather than kanji, so that the target audience of children could read them. Decoding the secondary alphabets is not a slam dunk, but it's much easier to deal with 40-odd characters than 4000. With the help of online sites like Nihongodict, I became more proficient at decoding hiragana and katakana and translated some signs myself.

The vast majority of the untranslated signs don't matter, and after a while, I started to omit signs with no relevance. For example, in episode 13, I typeset every menu item in the yakitori bar, complete with movement. In later episodes, I ignored those sorts of restaurant placards. In episode 20, I did as many of the festival booth signs as I could. Later, I didn't bother with most street signs.

But despite all the irrelevancies, every now and then an untranslated sign turned out to be significant. In episode 15, the "Dodo Bird" store sign is the punch line for the second half skit, and the joke simply isn't as good without it.

Insert or Overwrite?

When a sign is typeset, there's a fundamental choice to be made: insert the English into the scene, leaving the Japanese intact, or mask out the Japanese and overwrite it with the English. I ended up doing both, with very little consistency. Usually, though, I prefer to insert the English. My main reason is that the backgrounds for signs are rarely flat. They're often textured or shaded. Overwriting with a fixed color mask can produce a fake-looking result. However, if the background is uniform, the sign doesn't fade in or out, and there's no space for the English otherwise, I will overwrite the Japanese. The yakitori bar menu signs in episode 13 are all masked and overwritten.

Font Matching for Fun and Profit

If English in inserted into a sign, it really helps if the English font resembles the Japanese lettering. Thus, font matching is one of basic first steps in setting a sign. Polar Bear Café's signs are mostly done with a single font family. When I discovered this and realized that I had all the fonts, I sort of went overboard. Every sign was scrupulously matched for character shape, character weight, and so on. As a result, I ended up using more than 130 fonts.

Experienced typesetters don't do that. They realize that the subtle differences among Japanese fonts are rarely carried over into distinguishable differences in the English letters. Accordingly, they tend to typeset with a small repertory of fonts able to represent entire font families: gothics, minchos, etc. By the end of the series, I was able to see font families, but I still fell into the trap of trying to match fonts exactly.

One side effect of this is that each episode may contain eight or ten massive CJK (Chinese-Japanese-Korean) fonts, bloating the episode’s footprint and even breaking certain players.  Accordingly, I started compressing the CJK fonts, either by stripping OTF subfonts to create a "small" OTF, or by converting just the ISO-Latin characters into a really small TTF. Converting to TTF is more reliable, but it can have the undesirable side effect of changing character sizes. Accordingly, remember to compress fonts before use, not afterwards.

Compression had another undesirable side effect: elimination of special characters used in signs. As a result, many of the key fonts had to be compressed a second time, retaining specific special characters, and the episodes that used them redone. This is why episodes 1-13 and NCED01 will get v2s.

Color Matching

In addition to font matching, inserted English needs to match the color of the Japanese sign. Aegisub makes this easy with its color picker, but there are subtleties. As my wife the quilter points out, colors are not absolute; their appearance is changed by what surrounds them. I was constantly frustrated about this. An exact color match would appear faded if I added blur (and \blur1 is almost mandatory), or it would appear brighter if I added a dark border. I found I was often overriding the “exact” match for something that pleased my eye better. And that's a slippery slope, because my color sense is poor, at best. The yellow in OP1 doesn't really match well, for example. I made it too pale, and I still can't find a value of yellow that I like.

Movement

While some signs are static, many appear to move. Computer-based animation tools make it trivial to pan, zoom, or rotate a scene or sign to add some dynamism to a static background. If a sign moves, the English must move too (and if the sign is overwritten, the mask as well).

In the Dark Ages, the typesetter had to approximate movement with subtitle tags like \move, which assumed that motion was linear. This rarely looked good. Nowadays, motion capture software allows object movement to be tracked precisely, and Aegisub macros can translate the capture data into frame-by-frame typesetting. This bloats scripts enormously but provides very satisfying pans and scaling. (Motion capture is described in great detail in unanimated’s tutorial on typesetting.)

The motion capture software does have its limits, however. It's not terribly good at capturing rotation, particularly if that's combined with other forms of movement. In the first opening to Polar Bear Café, the circular show logo rotates and then begins to tilt down, eventually becoming a roadway. I had to fade the show title out a few frames into this sequence because tracking was lost as the circle began to deform. In episode 36, Grizzly’s door sign rotates on a pivot. The tracking software gradually lost the angle of the sign, and the sequence was ultimately set by hand.

Another limitation is that the motion-tracking software can't deal with true hand-drawn animation or deformations. Irregular changes between frames, particularly in object shapes, cause the software to goes off the rails. This was most evident in the Tanabata wish sequence in episode 13, where the paper tags containing the Tanabata wishes blow, twist, and curl in the wind. Every position in that sequence had to be set by hand. Fortunately, the animators were pressed for budget, and there are only 16 distinct configurations of the most critical sign, Panda's wish. However, the sequence also zooms in, so when a configuration repeated, the English had to be scaled and repositioned. That sequence took a week to do, mostly because I'd start to tear my hair out every few frames.

I’m fairly dissatisfied with the Tanabata signs. The English doesn’t appear to move correctly. This is because hand-drawn animation is not accurate. Angles and character sizes change from frame to frame, often incorrectly, but the eye forgives because it has no other reference point. However, when English is introduced, the discordance is obvious. The Japanese is moving "consistently," and so is the English, but they are not moving consistently with respect to each other.

Stock Signs

One interesting aspect of typesetting a long series with a low budget is getting to see how much animation (and signage) is reused.  Polar Bear Café uses stock shots, like the front of the zoo, the front of the café, and the front of Panda’s house, as a cheap way of indicating scene changes. If these stock shots contain signs, then the typesetting can be reused, with minor changes for coloration or zoom. I ended up compiling a "stock shot" index for the series, so that I could simply cut and paste the base typesetting for any stock sign.

The show also has standardized sets. Two of the most frequently used are the interior of the zoo office and the street in front of Rin Rin's florist shop. The zoo office has a whiteboard with the monthly schedule, always labeled "Schedule for the Month." Rin Rin’s shop has standard signs and is adjacent to a bookstore (just Books) and a gallery (Gallery Morita). Across the street is an antiques shop (just Antiques Shop). All of those signs were reused frequently.

Animation Errors

Another aspect of typesetting a long series in getting to see how often there are animation errors. Scenes are farmed out to different teams of animators, and sometimes details are inconsistent. For example, in the drive-thru restaurant sequence in episode 4, the microphone for ordering disappears and is replaced by a piece of horizontal trim as Polar Bear's car passes through. In various hand-animated sequences of Grizzly’s door sign in episode 36, the margins around the Japanese word vary wildly from frame to frame. BluRay editions often correct animation errors found in TV releases, but not in Polar Bear Café.

Summary

I'm still not a proficient typesetter. I stand in awe of how easy some of my colleagues make it look. However, I think Shirokuma Café looks better with typesetting, even if it's far from perfect. I hope you’ll agree. And if there are any highly experienced typesetters out there who are interested in the show… there are still some signs I have no idea how to do.

Tuesday, October 23, 2012

Technology Marches On

When softsubbing first became possible and then fashionable, it involved a number of compromises. Fancy sign typesetting wasn't really doable, and elaborate karaokes seemed to go the way of the dinosaur. The technology for elaborate typesetting, and the performance of softsubbed playback, simply wasn't there.

Well, time marches on. PCs have become powerful (I write this on a quad-core i5 that spends 99.9% of its time doing nothing). More importantly, the software for typesetting and subtitle playback has improved considerably. As a result, sign typesetting is approaching the glory days of hardsubbed AFX signs, and complex karaokes are rising from their graves.

For typesetting, the key development has been the use of motion tracking software, combined with automated frame-by-frame transforms of a base sign based on the tracked motions. This allows frame-by-frame typesetting of signs that move in non-linear ways, with none of the hassle associated with the manual process. Back in 2009, I set a 50-frame moving sign in Orphan's Hand Maid May by hand, and it took me hours, with manual computation of the position deltas between frames. I did a similar effort with a non-linear sign in Orphan's Space Neko Theater; after that, I swore off the practice. But with new software technology, it's no longer necessary.

The motion tracking software is based on the sorts of techniques used in motion-capture special effects. After designating an initial set of points to be tracked, the software follows the image (in this case, a sign) through subsequent frames, generating coordinates. Those coordinates feed an Aegisub automation script that applies the coordinate changes (including changes in angles) to an initial typesetting specification. The result is a frame-by-frame sign that accurately tracks the motion on the screen.

Complex softsubbed karaokes are a more recent development. When Polished released a DVD version of Tokimeki Memorial a few years back, the initial version, which emodied C1's hardcoded karaoke in the script, simply wouldn't play. Polished had to redo the scripts with a simple, line-timed karaoke. The problem wasn't the speed of PCs; it was the subtitle rendering software, vsfilter, which suffered from a number of design bottlenecks. Recently, the community became sufficiently fed up to code up a replacement, called xy-filter, which is significantly more efficient.

I saw this in action with the recent DVD redo of Rescue Wings. When topf(h) added the typesetting, he simply incorporated the Ureshii karaokes verbatim — even though some of them to thousands of lines. With xy-filter, they play back as smooth as butter on almost any modern machine. Now fancy karaokes are making a comeback. GotWoot's opening for the season's hit show, Magi, runs to 6500 lines. I'm sure more will follow.

These developments will not be without their detractors. Viewers with old PCs will be in trouble. Non-techies will have difficulties in figuring out how and where to install xy-filter. As with the advent of the MKV container, replacing OGM; of h.264, replacing XviD; and 10-bit encoding, replacing 8-bit; the fansub community will by and large ignore them. One hopes that recoders (who typically change the original format to MP4 for playback on tablets and phones) can provide relief to technology laggards.

So with beautiful signs and complex karaokes again with reach, I think it's time for updated versions of some classics. I'd love to see Amatsuki, Yume Tsukai, Nodame Cantabile, and other classics from Ureshii or C1 redone, with their original karaokes. (Some are probably still beyond reach: the Skip Beat OP karaoke is more than 8MB long.) If you've got the interest, and the raws, I have the scripts.

Wednesday, May 16, 2012

Signs and Portents

Another piece of the fansubbing process where I've occasionally dabbled is typesetting. Typesetting consists of two rather separate activities:
  • Styling: selecting and apply fonts, colors, spacing, and effects to dialog.
  • Typesetting proper: selecting and applying, fonts, colors, and effects to signs.
Let's look at each of those in turn.

Styling

At its core, styling is about font selection, and then about font size, effects, and spacing. I rarely do this on my own, as I don't have much feeling for typefaces; if left to me, every drama would be set in Corbel, and every comedy in DomCasualD. Fortunately, other team members with good typographic sensibilities make these decisions.

Font coloring can vary, but most dialog is set in white, with a black outline. Sometimes other outline colors are used, particularly in comedies, but white with a black outline is the least distracting.

The basic font style may have variations to represent different dialog modalities. For example, if the dialog represents thought, the subtitle may be set in italics. If the dialog is a flashback or overlaps another speaker, the subtitle may have a different outline color. It's easy to take this too far and end up with absurdities like different outline colors for each character. (In EPIC's version of Harukanaru Toki 3: Endless Destiny, the outline matched the character's hair color.) Personally, I prefer to keep the number of styles to a minimum: main and overlap, with a third for thought if the translator really insists.

The transition from 4:3 aspect ratios to 16:9 aspect ratios has affected spacing, particularly padding. When the typical anime was only 640 pixels across, it didn't really matter if the subtitle came fairly close to the left and right edges; the eye could encompass the line without moving. Further, horizontal space was at a premium, so wide left and right margins tended to create 3-line subtitles, a definite no-no. With 853 pixels across on SD, and 1280 on HD, there's more room for the subtitles and less pressure on the margins. Lines that come close to the margins require eye movement, particularly in HD, so wider margins are preferred. A 640x480 anime typically had 10 pixel margins. Today's 16:9 episodes have 60 pixel margins, or even wider.  Likewise, subtitles have been moved away from the bottom of the screen, with the original 10 pixel padding mask extended to 30 or 35 pixels.

Once the stylist has selected the dialog styles, the application of the styles is fairly mechanical and easily done during the editing process. The editor has to look out for a few potential issues:
  • Special characters. The selected font may not support em-dash (long dash), accented characters, or foreign characters. Em-dash can be emulated by {\fscx200}-{\r}, but other unsupported special characters must be replaced by their standard English counterparts.
  • Italics. Some fonts have true italics and don't require special treatment. When fonts lack true italics, italics are emulated by sloping the font to the right. This creates compression of any space that follows an italicized word, so that the italicized word appears to run into the next word. The fix is to elongate the space following the end of italics, e.g., {\i1}italics{\i0\fscx130} {\r)normal-text. (The scale factor may need to be 140 or 150, depending on the font.)
  • Sign clash. Dialog may clash with signs or with those scrolling messages that seem so prevalent in TV episodes these days. The dialog may need to be moved up or down with {\pos(x,y)} or moved to the top of the screen with {\an8}.
Typesetting

The minimal goal of typesetting is to put the translation of important signs somewhere on the screen; the maximal goal is to make the translated sign appear to be part of the original drawing. Yawara is an example of the minimal approach. Signs translations are placed at the top of the screen. No effort is made to integrate the English text with the picture on the screen.

More elaborate typesetting effects can be achieved with the SSA/ASS typesetting markups. These range from simple font selection and scaling to elaborate animations that will move, transform, or clip subtitles through multiple frames. Because SSA markups are embedded in the script, this is the only form of typesetting compatible with softsubs, and experts can achieve quite remarkable effects, albeit with great effort. For example, the comic book dialog in at the end of C1's Nodame Cantabile episode 23 is done entirely with SSA markups; the signs script is three times longer than the dialog script. One sign in FFFpeep's Nekogami Yaoyozuru has more than 300 lines, each covering one frame.

The most elaborate typesetting is done with photographic manipulation programs, like Adobe Photoshop, or special effects programs, like Adobe AfterEffects (AFX). These programs can create typesetting that is truly indistinguishable from the original. However, they have to be encoded into the video, which complicates and lengthens the subtitling process. Because most groups set speed as an overriding priority, AFX typesetting is declining.

As with styling, I prefer to leave typesetting to team members with greater experience and proficiency, but I have done a few shows myself, including the Orphan Fansub version of Hand Maid May and the Frostii episodes of Gosenzosama Barbanzai. For me, font selection remains the greatest challenge; figuring out which typesetting tags to employ to get a specific effect is actually a lot of fun.

The best way to learn typesetting is to try it. Take a script with interesting softsubbed signs, load it into Aegisub, and see what the typesetting tags are doing. By varying tag parameters, or removing tags entirely, you can get a good sense of what's happening. Then take an episode and try doing the signs yourself. You'll fall off the bike a few times, but eventually, you'll get the hang of it. And if all else fails, there's always the Yawara model:

    {\an8}Sign: what the sign says

Have fun!

Sunday, January 22, 2012

Much Ado About Nothing: Softsubbed Signs and Karaokes

The comments and forums of BakaBT contain a lot of whining about shows with hardsubbed signs and karaokes. Some members routinely downrate offerings if the signs and karaokes are not softsubbed. That got me wondering: seriously, folks, why do you care?

I can understand why viewers prefer softsubbed dialog. As an editor, I've been offended by more than one group's editing. If I care enough, I'll demux the script and fix the problems. For example, I recently fixed up Ryokunohara Labyrinth at Al_Sleeper's request. (You can download the revised subs here.) I'd do even more of this, except that editing sins are usually accompanied by both translation and timing problems. The former are beyond my skills, and the latter are too hard on my ergonomically-challenged wrists. So Polished/Ayako's Nagasarete Airantou, ARR's Urusei Yatsura Special and Space Neko Theatre, and many others, languish on the "To Do" list.

But signs and karaokes? Are viewers seriously concerned about fixing them up? I see no evidence that this ever happens. At most, viewers want to turn off karaokes. Well, let me break it to you gently: you can use chapters and skip the songs entirely if you're bothered. In fact, using mkvmerge, you can use ordered chapters to play a raw instead of the subtitled encode for the songs. Or you can mux the softsubbed script onto your own raw and omit those offending karaokes and signs.

From the fansub team's point of view, there are strong arguments for softsubbing everything. The most compelling is efficiency. The encoder can make a final version once, and after that, every change is done by muxing new scripts. This saves a significant amount of time. Another good argument is change management. With softsubs, any subtitle-related error can be corrected easily, with simple patches. That makes it much easier to produce a consistent final series release, although it also contributes to the proliferation of v2's, v3's, and v4's.

Still, there are limits to softsubbing, particularly for signs and karaokes. Complex effects, such as non-linear motion tracking, require frame-by-frame typesetting. For example, a 2.2 second clip of a moving sign in GotWoot's Showa Monogatari required 52 lines of typesetting. The signs script for the final episode of C1's Nodame Cantabile was more than 100KB long: three times the size of the dialog script. Typesetters can rarely take that amount of trouble. Instead, they'll approximate non-linear motion with \move and \t commands, resulting in signs that diverge from the motion of the Japanese text.

Another issue is performance. Complex effects can bring even the brawniest system to its knees. For example, tlacat6's Aoi Exorcist specials have a rotating title sign. My brand-new, quad-processor i5 cannot render that in real time. For its recent release of Denpa Teki, WhyNot warned that the viewer would need to alter parameters in MediaPlayer Classic to avoid performance lags.

For karaokes, the issue is visual creativity. Most softsubbed karaokes use either line-timing or simple k-timing (coloring). This is great for efficiency but boring to watch after a while. Visually creative karaokes are created by scripts and are too complex to be played as softsubs. This can get out of hand, but I would not have enjoyed losing, for example, the brilliant simplicity and thematic appropriateness of koda's opening karaoke for C1's Nodame Cantabile in the name of softsubbed purity.

As a result, some fansub teams, notably Frostii and AnimeYoshi, prefer hardsubbed signs and karaokes, despite the greater work they entail. Their typesetters work to an exacting standard: typesetting is successful if the English looks like it was drawn by the original animators. Their karaokes are intended to stand the test of multiple viewings. Is it worth it? Look at AnimeYoshi's logo for Another and how seamlessly it blends into the show's Japanese title. Look at Frostii's typesetting on Yurumates and Yurumates wa. Look at the karaoke's on C1's Nodame Cantabile or Ureshii's Rescue Wings. I think the results justify the extra work.

Who is really disadvantaged by hardsubbed signs and scripts? The only constituency I can think of is script stealers - ah, excuse me, script borrowers. If you're translating a script to another language, or creating a DVD or BluRay version of a TV series, it's helpful if everything is softsubbed. Well, I have news for you again: fansub groups don't exist for the convenience of your projects. Besides, if you ask politely enough, some fansub groups will provide project materials, including signs and karaokes, particularly for older shows desperately in need of better video.

So please, let's stop the whining and be appreciative of the hard work that goes into these shows; and if softsubbed purity is a make-or-break issue for you, watch someone else's version.