> For the complete documentation index, see [llms.txt](https://overleaf-pro.ayaka.space/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://overleaf-pro.ayaka.space/latex/in-depth-articles/17-how-does-expandafter-work-a-detailed-macro-case-study.md).
# How does \expandafter work: A detailed macro case study
[Part 1](/latex/in-depth-articles/19-how-does-expandafter-work-an-introduction-to-tex-tokens.md) [Part 2](/latex/in-depth-articles/22-how-does-expandafter-work-the-meaning-of-expansion.md) [Part 3](/latex/in-depth-articles/21-how-does-expandafter-work-tex-uses-temporary-token-lists.md) [Part 4](/latex/in-depth-articles/20-how-does-expandafter-work-from-basic-principles-to-exploring-tex-s-source-code.md) [Part 5](/latex/in-depth-articles/17-how-does-expandafter-work-a-detailed-macro-case-study.md) [Part 6](/latex/in-depth-articles/18-how-does-expandafter-work-a-detailed-study-of-consecutive-expandafter-commands.md)
## Case study: \expandafter example from The $$\varepsilon\mathrm{\text{-}{\TeX}}$$ Manual
The $$\varepsilon\mathrm{\text{-}{\TeX}}$$ typesetting engine was derived from Knuth’s TeX software and originally intended as an “interim” step toward development of the [New Typesetting System](https://en.wikipedia.org/wiki/New_Typesetting_System) (NTS), written in the Java programming language. $$\varepsilon\mathrm{\text{-}{\TeX}}$$ was first developed in the late 1990s to add a suite of new primitive commands which provide additional functionality not available in Knuth’s original program. Although $$\varepsilon\mathrm{\text{-}{\TeX}}$$ has received periodic updates since its initial release, today it is not widely used as a standalone typesetting engine although its innovations have been absorbed into later generations of TeX: pdfTeX, XeTeX and LuaTeX.
The [$$\varepsilon\mathrm{\text{-}{\TeX}}$$ manual](http://mirror.ox.ac.uk/sites/ctan.org/systems/doc/etex/etex_man.pdf) contains an enlightening example of a macro which makes clever use of `\expandafter`:
```
\def\foo#1#2{\number#1
\ifnum#1<#2,
\expandafter\foo
\expandafter{\number\numexpr#1+1\expandafter}%
\expandafter{\number#2\expandafter}%
\fi}
```
`\foo` implements a looping mechanism, such that `\foo{7}{13}` produces `7, 8, 9, 10, 11, 12, 13`; however, `\foo` does not use any *assignments to variables* in order to control the looping process—which makes it an interesting macro to explore in some detail.
### Some background: expressions and assignments
An important element of `\foo`’s code is its use of the command `\numexpr`, a command from a set of four related primitives first introduced by $$\varepsilon\mathrm{\text{-}{\TeX}}$$: `\numexpr`, `\dimexpr`, `\glueexpr` and `\muexpr`. Their purpose is to construct so-called *expressions* which allow calculation/manipulation of TeX values of type number, dimen, glue, or muglue (respectively). As discussed on pages 8–9 of The $$\varepsilon\mathrm{\text{-}{\TeX}}$$ Manual, an important characteristic of *expressions* is their evaluation (calculation) does not require TeX to perform any *assignments*.
In programming terms, assignment is the process of setting (assigning) a variable to a have a particular value; for example, assigning `\count` register `99` to contain the value `12345` via `\count99=12345`. Many other types of assignment take place during TeX processing—such as assigning token registers to contain a series of tokens, assigning box registers to contain box content, and so forth.
To perform an assignment, such as `\count99=12345`, TeX needs to action (execute) the internal code which implements the behaviour of `\count` or any other primitive that performs some sort of assignment. However, there are times when TeX is performing pure *expansion* and, at those times, such assignments are not actioned—*at that point in TeX’s processing*. Examples of this situation include the following commands:
* `\edef\command {*token list*}` the “expanded definition” macro-definition command which expands tokens in *token list* and stores the results as the definition of `\command`.
* `\write *number* {*token list*}` expands tokens in `*token list*` and writes them out a file represented by `*number*`.
* `\directlua {*token list*}` this LuaTeX primitive command is used to pass Lua code to the built-in Lua interpreter. All tokens in `*token list*` are fully expanded before being passed to the Lua interpreter for execution.
#### Quick example of \edef
If we write the following basic macros:
```
\def\mycount{\count99=12345}
\edef\mymacro{\mycount}
```
`\edef` will expand `\mycount` into is constituent tokens but it goes no further: none of the commands contained in the definion of `\mymacro` will be actioned: i.e., the assignment of `12345` to `\count99` *does not happen at this point*; only when we call `\mymacro` will that assignment take place as TeX executes the code to process the `\count` primitive. When TeX is performing *expansion-only activities* any assignments will be actioned later in TeX’s processing, not during the expansion process itself.
#### Why are assignments of interest here?
When writing code to perform a loop—in any programming language—it is common practice to have a variable designated to act as a “loop counter”: used control the number of times a loop is executed. Looping is typically controlled by testing whether that designated loop-counter variable has reached a particular value—that variable is incremented (or decremented) for each iteration of the loop. However, modifying a loop-counter variable means assigning it a new value which, for TeX, usually requires the primitive command `\advance` to increment (or decrement) a value stored in a `\count` register. As we’ve seen, during TeX’s pure expansion process such assignments (including incrementing variables) cannot take place: the macro `\foo` cleverly circumvents this restriction.
### Back to explaining \foo
The macro `\foo` is able to control the looping process *without* needing to assign values to any variables: it controls how often the loop takes place using data arising from expansion: data values stored in temporary token lists. Using our knowledge of TeX’s usage (creation) of temporary token lists we can take a closer look to see exactly how `\foo` achieves its results.
**Remember**: We are working through the execution of a macro after the original text of its definition—contained in a physical `.tex` file—has been scanned (read-in by TeX) and converted to a token list representing the macro definition. In essence, we are following TeX’s processing of those stored *tokens* whilst it is reading and processing tokens in the macro definition contained somewhere in TeX’s memory. Any space characters originally present in the TeX code of the macro’s definition (text within the `.tex` file) will have been absorbed whilst TeX was scanning that text for commands (spaces as terminators), or they will have been converted to tokens, such as the space character after the comma (`,`) in `\ifnum#1<#2,` which arose from conversion of the end-of-line character (`\r`) into a space.
Because the TeX code in `\foo` uses multiple `\expandafter` commands, we’ll assist our explanation by adding subscripts to each `\expandafter`, indicating which one we are rererring to. In addition, we’ll extend the notation for tokens processed by `\expandafter` to $$\mathrm{T^i\_1}$$ and $$\mathrm{T^i\_2}$$, representing tokens $$\mathrm{T\_1}$$ and $$\mathrm{T\_2}$$ for `\expandafteri`: `\expandafteri` $$\mathrm{T^i\_1T^i\_2}$$
Here is the annotated macro code:
```
\def\foo#1#2{\number#1
\ifnum#1<#2,
\expandafter1\foo
\expandafter2{\number\numexpr#1+1\expandafter3}%
\expandafter4{\number#2\expandafter5}%
\fi}
```
`\foo` starts with `\number#1` which uses the expandable command `\number` to convert the first argument value into its typeset representation. The `\number` command works by generating a temporary token list containing character tokens which represent the individual digits contained in the numeric value that `\number` is operating on. That token list becomes TeX’s next input source. Here, that token list is read and the tokens are output to typeset the value of `#1`.
Next, the macro performs the test `\ifnum#1<#2` to check if the argument for `#1` is less than the argument passed in for `#2`. If so, a comma (`,`) token is output (typeset) followed by some space arising from the token that was generated from the linebreak character after the comma (`,`). That space character was first generated when TeX read this line from the `.tex` file.
The macro continues by processing this next section of code, which is the core of its operation:
```
\expandafter1\foo
\expandafter2{\number\numexpr#1+1\expandafter3}%
\expandafter4{\number#2\expandafter5}%
\fi}
```
In essence, this code generates a series of temporary token lists which result in multiple calls to the `\foo` macro, terminating when the if-test `\ifnum#1<#2` is no longer true. But *how* is looping controlled because no assignments are taking place: where is the “loop counter”?
Let’s start by looking at the code `\expandafter1\foo\expandafter2`. Note that we will use the subscript notation `token` (or `(token)`) to remind ourselves that, here, TeX is reading/processing numeric (integer) token values.
Here, we have the following tokens as input for `\expandafter1`:
* $$\mathrm{T^1\_1} =$$`\footoken` which is read-in and stored for later *re-insertion* back into the input
* $$\mathrm{T^1\_2} =$$`\expandafter2 (token)` which is expanded
For `\expandafter2` we have:
* $$\mathrm{T^2\_1} =$$`{token` which is saved for later *re-insertion* back into the input
* $$\mathrm{T^2\_2} =$$ `\numbertoken` which is expanded
**Note:**`\number` is an expandable command whose purpose is to “convert to tokens”: i.e., convert a numeric quantity into a series of character tokens which represent that quantity. When `\number` is expanded, the first thing that TeX does is to scan the input looking for integers: a process which triggers further expansion.
**The key to the story:** Here, `\number` is acting on the *expression* `\numexpr#1+1` which calculates the value of `#1+1`. The result of that calculation is processed by `\number` to convert it into a temporary token list containing character tokens representing the value of `#1 + 1`. That temporary token list, generated by `\number`, will eventually be read-in as the first argument to another call of `\foo`. Rather than incrementing a loop counter (via `\advance` and assignment), the use of `\numexpr` creates a new value but without assignment being necessary. Through this mechanism, the variable controlling the loop (`\foo`’s parameter `#1`) is incremented and iteration through the loop is controlled and terminated: quite ingenious!
Next, `\expandafter3` is processed, yielding:
* $$\mathrm{T^3\_1} =$$`}token` which is saved for later *re-insertion* back into the input
* $$\mathrm{T^3\_2} =$$`\expandafter4 (token)`, which is expanded:
For `\expandafter4` we have:
* $$\mathrm{T^4\_1} =$$`{token` which is saved for later *re-insertion* back into the input
* $$\mathrm{T^4\_2} =$$`\numbertoken` which is expanded and converts `#2` into another temporary token list.
Finally,`\expandafter5` is expanded:
* $$\mathrm{T^5\_1} =$$`}token` which is saved for later *re-insertion* back into the input
* $$\mathrm{T^5\_2} =$$`\fitoken`, which is an expandable command.
The expansion of `\fi` effectively terminates the `\ifnum` and, in effect, closes this iteration of the macro. TeX now completes re-insertion of all the tokens temporarily saved by the multiple `\expandafter` commands: this generates a series of single-token token lists arising from the tokens saved by each `\expandafter`. In addition TeX has also created token lists from through the action of `\number`.
### Assembling the token lists
In essence, the `\foo` macro generates a sequence of token lists: you can think of `\foo` as a token-list “manufacturing facility”. Those token lists are read by TeX to become the next sources of input. The clever part is contained in one of the earlier actions of `\foo`:
```
\expandafter1\foo\expandafter2
```
through which `\foo` arranges to call itself again but with different arguments that are stored in token lists constructed by `\number`. To make these token lists collectively behave as a macro call, the braces `{` and `}` have all been saved and re-inserted into the input (as single-token lists) by the actions of `\expandafter` commands.
[Part 1](/latex/in-depth-articles/19-how-does-expandafter-work-an-introduction-to-tex-tokens.md) [Part 2](/latex/in-depth-articles/22-how-does-expandafter-work-the-meaning-of-expansion.md) [Part 3](/latex/in-depth-articles/21-how-does-expandafter-work-tex-uses-temporary-token-lists.md) [Part 4](/latex/in-depth-articles/20-how-does-expandafter-work-from-basic-principles-to-exploring-tex-s-source-code.md) [Part 5](/latex/in-depth-articles/17-how-does-expandafter-work-a-detailed-macro-case-study.md) [Part 6](/latex/in-depth-articles/18-how-does-expandafter-work-a-detailed-study-of-consecutive-expandafter-commands.md)
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://overleaf-pro.ayaka.space/latex/in-depth-articles/17-how-does-expandafter-work-a-detailed-macro-case-study.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.