Seas Beyond the Sunset

Belated update: I’ve taken a few swipes at adding an HTML parser to Gargoyle behind the scenes, and I’m not excited about continuing down that road. Partly because I’m still putting off that text buffer rewrite, and partly because I can’t muster much enthusiasm for retrofitting the code with a standards-based layout engine.

I took a closer look at the much-maligned HTML TADS code and found a lot to like. Its parser and layout engine are both implemented in a reasonably platform-agnostic way, and QTADS has demonstrated that a non-Windows port is perfectly viable. My main objection now is that it lives on the wrong side of the API: rather than using Glk calls to display the output, it would primarily depend on Gargoyle’s internal, platform-specific pseudo-API.

Meanwhile, the most recent draft for Glk CSS suggests that interpreters will need to host WebKit or something like it to provide authors with the most capable, feature-rich implementation. If WebKit inclusion is the end goal, I may as well go for the gold from the start.

One of the planned features in TADS 3.1 is the ability for a story file to target a web browser for output. If Gargoyle were to bundle WebKit, it could host a browser instance for TADS as well as Inform. That would put the library exactly where I want it to be: capable of offering the full range of output effects for both major IF languages on the three primary desktop platforms.

The most important omission under this plan would be the existing set of multimedia TADS 2 and TADS 3 games. However, I am somewhat optimistic that the same implementation bits needed to make WebKit work will also make possible a full HTML TADS port.

So the next step is to investigate WebKit and see what it will take to get it running atop Gargoyle’s OS layer. Ideally I’d augment the collection of OS functions in a way that also supports an HTML TADS implementation. I have to be careful to avoid bloat, though – the OS code serves as a porter’s TODO list, and complicating it unnecessarily is a good way to ensure that Gargoyle won’t be widely ported.

It’s all very well to state that the text buffer code needs to be aware of logical structure beyond the character, but that still leaves the task of deciding which structures to recognize and implement. Characters, words, and paragraphs are the three obvious candidate.

Characters are tricky. On the one hand, I need them around so I know what to draw. On the other hand, they are something of an obstacle to a more coherent structural model. Which of the possible styles, if any, can legitimately be applied to characters? Certainly not indentation or justification.

You could make a case for the type settings, such as size, font and color, but varying any of those mid-word is likely to result in an unreadable mess. I could disallow it on the basis of aesthetics, but fortunately the Glk API guarantees that within the context of any single output call, all text will share the same style.

This is manifestly not true when dealing with HTML TADS, which dumps a set of tags into the buffer every time. But there the tags rather than the function calls provide the style boundaries. As long as I never mix characters from different calls or different tags into the same stylistic unit, I can safely ignore attributes at the character level.

On to words. Although indentation and justification still do not apply, all of the type settings do. It’s also important for the line breaking code to take words into account, as well as non-breaking characters and hyphens. Those can be treated as parts of words and word boundaries, respectively.

Whitespace – the stuff between words – deserves some special attention. Gargoyle features whitespace trimming. Spaces after a full stop will be tweaked. Runs of consecutive whitespace will generally be collapsed, unless the attributes suggest some formatting significance beyond mere spacing. My plan thought is to treat formatted spaces as words, in whole or part, and otherwise to eliminate whitespace altogether, handling it when deciding how to lay out a given line.

Font size presents a special headache. For height calculations, a line needs to be at least as high as the font size of its largest word. Paragraph height becomes a function of the number of lines; lines are constrained by the horizontal space available, and I need enough of them to display all the words in the paragraph. If words within the paragraph can have different heights, I have to add each line height separately. I may also have to adjust the line metrics above and below to make room for ascenders and descenders: an ugly business.

It’s tempting to decree that all words within a paragraph shall have the same height, as determined by the font size in effect when the first word is added to the paragraph. So tempting that this is in fact what I will do, absent some compelling reasons to reconsider. The FONT tag may force my hand, but as long as the rest of the foundation is solid, it should be straightforward to revisit the decision.

Otherwise, words take on the style attributes in effect at the time I demarcate them, as part of the process of printing text into the buffer. Words will never be combined across function calls or between HTML tags; once I’ve parsed a given round of output, I’ve got a complete set of words in hand. The characters making up those words will not change but the attributes might; both the Z-Machine and HTML TADS can retroactively apply colors to existing text. Something to bear in mind; not something to explore in great detail at this juncture.

Last but not least: the paragraph. Whatever the formal definition, I use the term to mean “all the words between two newline characters.” Here the indentation and justification styles become relevant, and the orphaned font size finds a home at last.

Paragraph indentation specifies the whitespace before the first word on the first line, while indentation covers the whitespace before the first word on every line. Spacing between words depends on the justification style. For center, flush left, and flush right, whitespace is synthesized between words: zero after a hyphen; one after a word; a configured number after a full stop. For justified text, additional spaces are distributed throughout the line to align it with the left and right margins.

If Glk CSS adds DIV and SPAN to the mix, these will constitute paragraph style entities. HTML tables do as well, though they pose a unique layout challenge. Depending on how it breaks down, I may need a structural element above the paragraph level to keep the code from getting bogged down in special cases.

Tomorrow I will work on the layout algorithm, and see if it still makes sense as high level code.

The perennial thorn in my side when dealing with Gargoyle is the complexity of the text buffer code. The redraw function alone consists of more than 300 lines of fairly dense code, most of which was hacked in after the fact to fix bugs and support new features like hyperlinks and text selection.

At a more fundamental level, the issue is that the buffer stores output as a series of discrete characters, written one at a time. Contextual information, such as whether the character is part of a word, sentence, or paragraph, is essentially discarded. This simplifies reflow and redraw operations, as you can break the character stream into lines of convenient length and iterate through those lines to display as needed. But it makes it rather complicated to style any unit larger than a character.

Even now there are some Glk styles that Gargoyle doesn’t implement: indentation and justification. Adjustable font sizes would be prohibitively complex to add to the current system; everything is built on the assumption that the buffer has a set number of lines with a fixed height and variable width.

The situation is about to get a lot worse; Glk CSS will add more ways to style spans of text, more ways to alter font attributes. It also provides no equivalent to the measure and distinguish functions, so there’s no nice way for Gargoyle to tell games that some styles just won’t work. Realistically, this was never more than a salve to my conscience, as game authors were unlikely to work around Gargoyle’s limits, but it did let me feel better about those limits.

In a sense, the situation is already worse. I’ve committed to adding support for HTML TADS in Gargoyle, which is a significant undertaking in its own right. Trying to implement a markup language with no structural sense of what is being marked up would be messy. Trying to maintain it would be a nightmare.

I don’t like to rewrite code. It’s hard to distinguish the legitimate special cases from the ugly workarounds, and the inevitable result is hours of testing just to get back to square one. Despite my remarks above, the text buffer code works well for hundreds of games and thousands of players. The overwhelming majority of text displayed on the screen is written to a buffer. In a real sense it is the face of Gargoyle, and any significant change risks giving it a black eye.

But I’ve reached a point where either I do the rewrite or I walk away. For a while, I was leaning toward the latter. With the occasional cherished exception, work on Gargoyle is thankless community service. It has zero relevance to my career and garners only modest accolades in a scene dominated by artists and technical wizards. Novelty trumps refinement as a cultural value, and nothing I contribute will be essentially unique.

For all that, I find myself gearing up for one more push. Partly because the memory of the last release has faded; partly because of Chris Spiegel’s efforts to get the source code into better shape. Partly because the project offers a lot of intellectual challenges and commensurate personal rewards. Partly because it feels like the right thing to do.