This article includes a list of general references, but it lacks sufficient corresponding inline citations. Please help to improve this article by introducing more precise citations. (March 2019) (Learn how and when to remove this message)

In computer science and mathematical logic, an infinite-tree automaton is a state machine that deals with infinite tree structures. It can be seen as an extension of top-down finite-tree automata to infinite trees or as an extension of infinite-word automata to infinite trees.

A finite automaton which runs on an infinite tree was first used by Michael Rabin^[1] for proving decidability of S2S, the monadic second-order theory with two successors. It has been further observed that tree automata and logical theories are closely connected and it allows decision problems in logic to be reduced into decision problems for automata.

Infinite-tree automata work on ${\displaystyle \Sigma }$-labeled trees. There are many slightly different definitions; here is one. A (nondeterministic) infinite-tree automaton is a tuple ${\displaystyle A=(\Sigma ,D,Q,q_{0},\delta ,F)}$ with the following components.

• ${\displaystyle \Sigma }$ is an alphabet. This alphabet is used to label nodes of an input tree.
• ${\displaystyle D\subset \mathbb {N} }$ is a finite set of allowed branching degrees in an input tree. For example, if ${\displaystyle D=\{2\}}$, an input tree has to be a binary tree, or if ${\displaystyle D=\{1,2,3\}}$, then each node has either 1, 2, or 3 children.
• ${\displaystyle Q}$ is a finite set of states; ${\displaystyle q_{0}}$ is initial.
• ${\displaystyle \delta :Q\times \Sigma \times D\rightarrow 2^{Q^{*}}}$ is a transition relation that maps an automaton state ${\displaystyle q\in Q}$, an input letter ${\displaystyle \sigma \in \Sigma }$, and a degree ${\displaystyle d\in D}$ to a set of ${\displaystyle d}$-tuples of states.
• ${\displaystyle F\subseteq Q^{\omega }}$ is an accepting condition.

An infinite-tree automaton is deterministic if for every ${\displaystyle q\in Q}$, ${\displaystyle \sigma \in \Sigma }$, and ${\displaystyle d\in D}$, the transition relation ${\displaystyle \delta (q,\sigma ,d)}$ has exactly one ${\displaystyle d}$-tuple.

Intuitively, a run of a tree automaton on an input tree assigns automaton states to the tree nodes in a way that satisfies the automaton transition relation. A bit more formally, a run of a tree automaton ${\displaystyle A}$ over a ${\displaystyle \Sigma }$-labeled tree ${\displaystyle (T,V)}$ is a ${\displaystyle Q}$-labeled tree ${\displaystyle (T_{r},r)}$ as follows. Suppose that the automaton reached a node ${\displaystyle t}$ of an input tree and is currently in state ${\displaystyle q}$. Let the node ${\displaystyle t}$ be labeled with ${\displaystyle \sigma \in \Sigma }$ and ${\displaystyle d(t)}$ be its branching degree. Then, the automaton proceeds by selecting a tuple ${\displaystyle (q_{1},...,q_{d(t)})}$ from the set ${\displaystyle \delta (q,\sigma ,d(t))}$ and cloning itself into ${\displaystyle d(t)}$ copies. For each ${\displaystyle 0<i\leq d(t)}$, one copy of the automaton proceeds into node ${\displaystyle t.i}$ and changes its state to ${\displaystyle q_{i}}$. This produces a run which is a ${\displaystyle Q}$-labeled tree. Formally, a run ${\displaystyle (T_{r},r)}$ on the input tree satisfies the following two conditions.

• ${\displaystyle r(\epsilon )=q_{0}}$.
• For every ${\displaystyle t\in T_{r}}$ with ${\displaystyle r(t)=q}$, there exists a ${\displaystyle (q_{1},...,q_{d(t)})\in \delta (q,V(t),d(t))}$ such that for every ${\displaystyle 0<i\leq d(t)}$, we have ${\displaystyle t.i\in T_{r}}$ and ${\displaystyle r(t.i)=q_{i}}$.

If the automaton is nondeterministic, there may be several different runs on the same input tree; for deterministic automata, the run is unique.

In a run ${\displaystyle (T_{r},r)}$, an infinite path is labeled by a sequence of states. This sequence of states form an infinite word over states. If all these infinite words belong to accepting condition ${\displaystyle F}$, then the run is accepting. Interesting accepting conditions are Büchi, Rabin, Streett, Muller, and parity. If for an input ${\displaystyle \Sigma }$-labeled tree ${\displaystyle (T,V)}$, there exists an accepting run, then the input tree is accepted by the automaton. The set of all accepted ${\displaystyle \Sigma }$-labeled trees is called tree language ${\displaystyle {\mathcal {L}}(A)}$ which is recognized by the tree automaton ${\displaystyle A}$.

Nondeterministic Muller, Rabin, Streett, and parity tree automata recognize the same set of tree languages, and thus have the same expressive power. But nondeterministic Büchi tree automata are strictly weaker, i.e., there exists a tree language that can be recognized by a Rabin tree automaton but cannot be recognized by any Büchi tree automaton.^[2] (For example, there is no Büchi tree automaton that recognizes the set of ${\displaystyle \{a,b\}}$-labeled trees whose every path has only finitely many ${\displaystyle a}$s, see e.g. ^[3]). Furthermore, deterministic tree automata (Muller, Rabin, Streett, parity, Büchi, looping) are strictly less expressive than their nondeterministic variants. For example, there is no deterministic tree automaton that recognizes the language of binary trees whose root has its left or right child marked with ${\displaystyle a}$. This is in sharp contrast to automata on infinite words, where nondeterministic Büchi ω-automata have the same expressive power as the others.

The languages of nondeterministic Muller/Rabin/Streett/parity tree automata are closed under union, intersection, projection, and complementation.

• Wolfgang Thomas (1990). "Automata on Infinite Objects". In Jan van Leeuwen (ed.). Formal Models and Semantics. Handbook of Theoretical Computer Science. Vol. B. Elsevier. pp. 133–191. In particular: Part II Automata on Infinite Trees, pp. 165-185.
• A. Saoudi and P. Bonizzoni (1992). "Automata on Infinite Trees and Rational Control". In Maurice Nivat and Andreas Podelski (ed.). Tree Automata and Languages. Studies in Computer Science and Artificial Intelligence. Vol. 10. Amsterdam: North-Holland. pp. 189–200.