> For the complete documentation index, see [llms.txt](https://overleaf-pro.ayaka.space/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://overleaf-pro.ayaka.space/latex/in-depth-articles/14-how-do-tex-engines-typeset-tables.md). # How do TeX engines typeset tables ## How do TeX engines typeset tables ## Introduction: What does this series cover? Producing aesthetically pleasing tables can be a time-consuming task—whether you are using a visual layout tool, LaTeX or a markup language such as HTML or markdown. For LaTeX users in particular, typesetting tables ranks high on many people’s list of “pain points”, perhaps evidenced by “tables” being one of the most [highly-tagged topics on tex.stackexchange](https://tex.stackexchange.com/tags). In addition to the answers and examples on tex.stackexchange, even a cursory review of the support available on typesetting tables with LaTeX reveals a number of table-related information resources: * Overleaf’s [help pages](https://www.overleaf.com/learn/latex/tables) and other sites such as [learnlatex.org](https://www.learnlatex.org/en/lesson-08) * CTAN, the Comprehensive TeX Network, lists over [70 TeX/LaTeX packages](https://ctan.org/topic/table) related to creating tables * an entire [book on typesetting tables using LaTeX](https://www.amazon.co.uk/Typesetting-Tables-LaTeX-Herbert-Voss/dp/1906860254) * an excellent [online LaTeX table generator](https://www.tablesgenerator.com/latex_tables) A simple Google search for [LaTeX tables](https://www.google.com/search?q=latex+tables) yields a huge number of hits, listing many sites offering help, advice, examples and explanations. ### TeX, not LaTeX Given the wealth of literature available on typesetting tables using LaTeX, is there anything more to write about them—yet more table examples and enumeration/demonstration of package commands? Is there a way to approach the topic of typesetting tables which draws out or focuses on the fundamental principles and concepts of table typesetting? There is, but it requires peeling the LaTeX onion… We decided to produce an article series that seeks to provide readers with background information and explanations of the underlying *mechanics* of TeX-based table typesetting. Instead of focusing on typesetting tables using specific LaTeX macros/packages, we’ll explore the *underlying behaviour* of TeX engines: exploring the low-level typesetting machinery which provides the foundations upon which LaTeX macro commands are constructed. The ultimate objective is to tease out, and explain, the core methods and algorithms of TeX-based table typesetting—hopefully helping readers/users to make better sense of why tables behave as they do. An unavoidable consequence of this approach, peeling away the protective layers of LaTeX macro insulation, is exposure to messy low-level details from which users are usually (and gleefully) shrouded via layers of LaTeX macro code. It has taken a lot of time to research, write and illustrate these articles so it is our hope that they are a worthwhile addition to the literature, providing material that will inform readers and help them better understand this complex area of TeX typesetting. We should stress that this article series will *not* stray into discussing *aesthetics* of table design—that’s a topic replete with subjective preferences and one whose arguments must be played out elsewhere… ### Exploring the mechanics of TeX tables: How can you do that? To explore, and then write about, the lower-level mechanisms and processes taking place inside TeX engines, such as typesetting tables, Overleaf built (compiled) “debug” versions of Knuth’s TeX engine using Martin Rucket’s [Web2W process](https://w3-o.cs.hm.edu/users/ruckert/public_html/web2w/index.html). Traditionally, building TeX involves a process called Web2C, used within TeX Live, which generates C code by converting TeX’s original Pascal source code into its C equivalent. That process generates C code which was never intended to be read by humans, only by C compilers. The mechanically-generated C code is *exceptionally* difficult to read or modify for experimentation purposes. In contrast, the Web2W produces C source code (available [here](https://w3-o.cs.hm.edu/users/ruckert/public_html/web2w/ctex.c)) that is *orders of magnitude* more readable than code produced by Web2C. Consequently, Web2W’s C source code is much more amenable to modification for learning/experimentation purposes. Web2W produces a version of TeX (“CTeX”) that is *extremely* close to Knuth’s original program: “CTeX” does not include the many changes and enhancements introduced by the Web2C process—such as SyncTeX, command-line processing and file searching via Kpathsea. Although you sacrifice those valuable enhancements, the resulting (Web2W) C code is relatively easy to navigate using Knuth’s published TeX source code, even though TeX was written in Pascal. * **A note on names:** Strictly speaking, the name “TeX” must only refer to the original software written and published by Donald Knuth. Any modifications to his software must use a different name for the resultant TeX-based typesetting software. Here, we used the Web2W process to build an engine that still is, in effect, Knuth’s original software. However, for the avoidance of doubt we’ll use the term “CTeX” to mean the specific version built using Web2W but we’ll also use “TeX” to mean either Knuth’s original engine or a general term for the typesetting language based on the principles of Knuth’s TeX. We hope readers will forgive any occasional lapses in rigorous application/use of the strictly correct terminology: we hope meaning/intention is conveyed by context. The debug version of CTeX was run using the [Eclipse IDE](https://www.eclipse.org/downloads/packages/), making it possible to observe real-time processing of the C code which implements the low-level primitive (built-in) TeX commands, and algorithms, designed by Knuth to support table typesetting. The following short (approx. 90 seconds) video demonstrates the CTeX engine running inside the [Eclipse IDE](https://www.eclipse.org/downloads/packages/): {% embed url="" %} In addition to CTeX, we also compiled e-TeX to access a few additional primitives not present in Knuth’s original software. Although CTeX and e-TeX are, by now, old TeX-based engines, they are still suitable as a basis for exploring the mechanics of table typesetting because those underlying principles still apply to all TeX-based typesetting engines. ### Why use an older TeX engine to explore tables? Firstly, the printed book, [TeX:The Program](https://www.amazon.co.uk/Computers-Typesetting-TeX-Program-TEX/dp/0201134373), which lists and explains TeX’s source code, is still an extremely convenient guide to the murky waters of TeX’s inner workings, despite being published more than 30 years ago (1986). Of course, you can typeset TeX’s source code documentation yourself, for example see the Overleaf project [Typeset the source code documentation for TeX, e-TeX or pdfTeX](https://www.overleaf.com/latex/examples/typeset-the-source-code-documentation-for-tex-e-tex-or-pdftex/qkgfgyspnhcv). Since publication of TeX:The Program in 1986, new TeX engines have evolved, including pdfTeX, XeTeX and LuaTeX, all introducing features and commands that aren’t documented in TeX:The Program simply because they weren’t present in Knuth’s original software. For many core processes, such as TeX’s table typesetting, the code documented in TeX:The Program is still relevant as a basis for study—even though TeX’s source code is written in Pascal. In addition, Knuth’s TeX is comparatively easy, and fast, to compile—especially via the supremely useful [Web2W](https://w3-o.cs.hm.edu/users/ruckert/public_html/web2w/index.html) process developed by Martin Ruckert. Ease/speed of compilation make it much more convenient to modify TeX in simple ways—such as the creation of SVG graphics used later in this article series. ### Understanding the difference between TeX and LaTeX Many readers will already know that LaTeX is not actually an *executable* typesetting program but a large collection of commands (*macros*) that are, ultimately, written in a lower-level typesetting/programming language called TeX. Your LaTeX code produces typeset output only after it has been processed by an executable program called a *TeX engine*—the software which sits between your LaTeX code (document) and the typeset PDF. Today, users can choose between various TeX engines to typeset their LaTeX code, including pdfTeX, XeTeX and variants of LuaTeX. Those who are new to the TeX/LaTeX ecosystem are often, and understandably, bewildered by the plethora of cryptic-sounding names used for the tools they encounter: TeX, LaTeX, pdfTeX, pdfLaTeX, XeTeX, XeLaTeX, LuaTeX and LuaLaTeX. If you feel the same, help is at hand in the Overleaf article [What’s in a Name: A Guide to the Many Flavours of TeX](/latex/in-depth-articles/55-what-s-in-a-name-a-guide-to-the-many-flavours-of-tex.md) which explains the origin and meaning of all those terms. #### Dialects and TeX primitives Every TeX engine has a set of built-in commands called *primitives* which, collectively, comprise that engine’s “dialect” of the TeX typesetting language, reflecting the capabilities built into each engine. Here, the term “primitive” used to describe TeX engines’ built in commands does *not* mean those commands are basic or simplistic but that they are fundamental and indivisible: not constructed from other commands (unlike macros). All TeX engines share a large core set of primitives but some do contain primitives specific to that engine—giving rise to the notion of a “dialect” of TeX. Irrespective of which TeX engine is used to typeset your LaTeX document, its job is to process (“execute”) the collection of LaTeX commands (i.e., macros) used to write and construct your document. In effect, the TeX engine “converts” your LaTeX code (macros) *back into* their constituent TeX engine primitive commands which the engine can execute to perform the actual typesetting. You don’t *have* to use LaTeX commands to typeset documents with TeX engines—you *could* choose to construct your documents entirely with TeX primitives, i.e., directly in the low-level *TeX programming language* built into TeX engines. However, by today’s standards the TeX language is quite arcane and generally considered to be a difficult one in which to program—it can also require a large number of built-in primitives to accomplish your typesetting objective, making for an error-prone and potentially repetitive programming task. To avoid having to write directly in the TeX language, or constantly re-type the same series of commands, TeX engines let you create “shortcuts” called *macros*. By creating macros you define your own commands which package-up potentially long and complex sequences of TeX language primitives (or other macros) into a single “higher-level” command. TeX programmers can write very sophisticated macros which encapsulate a great deal of functionality within a single command—such as the commands provided by LaTeX. By using macro packages, such as LaTeX (or [ConTeXt](https://wiki.contextgarden.net/Main_Page)), document authors are (mostly) insulated from many tedious details, allowing them to focus on writing and typesetting rather than constantly wrangling with the complexities and nuances of the TeX language. ## In the beginning... Along with designing algorithms for typesetting mathematics and sophisticated line breaking, Knuth faced the challenge of programming his TeX software to typeset tables. Clearly, any table-construction algorithm(s) must not be unduly restrictive because that would frustrate users who need the freedom to create an almost infinite range of table layouts. Furthermore, table cells can contain a wide range of content including mathematics, graphics and pieces of text broken into nicely typeset lines—in fact, anything that TeX is capable of typesetting. Delivering this flexibility requires TeX’s table-building algorithms to work hand-in-hand with other parts of TeX’s typesetting machinery. However, TeX engines extract a price for the flexibility provided by their table-construction capabilities: numerous subtleties and nuances in the low-level behavior of built-in (primitive) table typesetting commands, of which there are 9: * **`\halign`**, **`\valign`**: core table construction commands * **`\tabskip`**: glue placed between \halign columns or \valign rows * **`\cr`**: “carriage return” mandatory terminator for all lines in a table * **`\noalign`**: inject material between \halign rows or \valign columns * **`\everycr`**: commands (token register) to read after detecting \cr * **`\span`**: a dual-function command: \span create cells spanning columns or rows, or expands commands in the table preamble (we will consider that in detail) * **`\omit`**: skip the templates for a particular cell * **`\crcr`**: used in macros to avoid errors if users forget a required \cr We’ll encounter these commands on our journey through table construction. ### Distant echoes of the challenges Buried in [the source code of TeX](https://www.overleaf.com/latex/examples/typeset-the-source-code-documentation-for-tex-e-tex-or-pdftex/qkgfgyspnhcv) is a somewhat daunting introduction to the topic of implementing the low-level commands \halign and \valign designed to typeset tables: > “It's sort of a miracle whenever \halign and \valign work, because they cut across so many of the control structures of TeX. Therefore the present page is probably not the best place for a beginner to start reading this program; it is better to master everything else first.” Knuth goes on to say > “Note that as \halign is being processed, we fearlessly give up control to the rest of TeX. At critical junctures, an alignment routine is called upon to step in and do some little action, but most of the time these routines just lurk in the background. It's something like post-hypnotic suggestion.” From these comments it seems reasonable to conclude that, even for Knuth, implementing TeX’s table typesetting was “something of a challenge”—to provide control and flexibility for the user but simultaneously ensuring TeX’s automated table construction algorithms were well orchestrated with TeX’s core typesetting processes. The author readily attests to the complexity of the code and algorithms behind TeX’s table typesetting capabilities but also to admiration for the sheer amount of capability contained within a relatively small amount of, albeit dense, Pascal (or C) code. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://overleaf-pro.ayaka.space/latex/in-depth-articles/14-how-do-tex-engines-typeset-tables.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.