In group theory, restriction forms a representation of a subgroup using a known representation of the whole group. Restriction is a fundamental construction in representation theory of groups. Often the restricted representation is simpler to understand. Rules for decomposing the restriction of an irreducible representation into irreducible representations of the subgroup are called branching rules, and have important applications in physics. For example, in case of explicit symmetry breaking, the symmetry group of the problem is reduced from the whole group to one of its subgroups. In quantum mechanics, this reduction in symmetry appears as a splitting of degenerate energy levels into multiplets, as in the Stark or Zeeman effect.

The induced representation is a related operation that forms a representation of the whole group from a representation of a subgroup. The relation between restriction and induction is described by Frobenius reciprocity and the Mackey theorem. Restriction to a normal subgroup behaves particularly well and is often called Clifford theory after the theorem of A. H. Clifford.^[1] Restriction can be generalized to other group homomorphisms and to other rings.

For any group G, its subgroup H, and a linear representation ρ of G, the restriction of ρ to H, denoted

${\displaystyle \rho \,{\Big |}_{H}}$

is a representation of H on the same vector space by the same operators:

${\displaystyle \rho \,{\Big |}_{H}(h)=\rho (h).}$

Classical branching rules describe the restriction of an irreducible complex representation (π, V) of a classical group G to a classical subgroup H, i.e. the multiplicity with which an irreducible representation (σ, W) of H occurs in π. By Frobenius reciprocity for compact groups, this is equivalent to finding the multiplicity of π in the unitary representation induced from σ. Branching rules for the classical groups were determined by

• Weyl (1946) between successive unitary groups;
• Murnaghan (1938) between successive special orthogonal groups and unitary symplectic groups;
• Littlewood (1950) from the unitary groups to the unitary symplectic groups and special orthogonal groups.

The results are usually expressed graphically using Young diagrams to encode the signatures used classically to label irreducible representations, familiar from classical invariant theory. Hermann Weyl and Richard Brauer discovered a systematic method for determining the branching rule when the groups G and H share a common maximal torus: in this case the Weyl group of H is a subgroup of that of G, so that the rule can be deduced from the Weyl character formula.^[2][3] A systematic modern interpretation has been given by Howe (1995) in the context of his theory of dual pairs. The special case where σ is the trivial representation of H was first used extensively by Hua in his work on the Szegő kernels of bounded symmetric domains in several complex variables, where the Shilov boundary has the form G/H.^[4][5] More generally the Cartan-Helgason theorem gives the decomposition when G/H is a compact symmetric space, in which case all multiplicities are one;^[6] a generalization to arbitrary σ has since been obtained by Kostant (2004). Similar geometric considerations have also been used by Knapp (2003) to rederive Littlewood's rules, which involve the celebrated Littlewood–Richardson rules for tensoring irreducible representations of the unitary groups. Littelmann (1995) has found generalizations of these rules to arbitrary compact semisimple Lie groups, using his path model, an approach to representation theory close in spirit to the theory of crystal bases of Lusztig and Kashiwara. His methods yield branching rules for restrictions to subgroups containing a maximal torus. The study of branching rules is important in classical invariant theory and its modern counterpart, algebraic combinatorics.^[7][8]

Example. The unitary group U(N) has irreducible representations labelled by signatures

${\displaystyle \mathbf {f} \,\colon \,f_{1}\geq f_{2}\geq \cdots \geq f_{N}}$

where the f_i are integers. In fact if a unitary matrix U has eigenvalues z_i, then the character of the corresponding irreducible representation π_f is given by

${\displaystyle \operatorname {Tr} \pi _{\mathbf {f} }(U)={\det z_{j}^{f_{i}+N-i} \over \prod _{i<j}(z_{i}-z_{j})}.}$

The branching rule from U(N) to U(N – 1) states that

${\displaystyle \pi _{\mathbf {f} }|_{U(N-1)}=\bigoplus _{f_{1}\geq g_{1}\geq f_{2}\geq g_{2}\geq \cdots \geq f_{N-1}\geq g_{N-1}\geq f_{N}}\pi _{\mathbf {g} }}$

Example. The unitary symplectic group or quaternionic unitary group, denoted Sp(N) or U(N, H), is the group of all transformations of H^N which commute with right multiplication by the quaternions H and preserve the H-valued hermitian inner product

${\displaystyle (q_{1},\ldots ,q_{N})\cdot (r_{1},\ldots ,r_{N})=\sum r_{i}^{*}q_{i}}$

on H^N, where q* denotes the quaternion conjugate to q. Realizing quaternions as 2 x 2 complex matrices, the group Sp(N) is just the group of block matrices (q_ij) in SU(2N) with

${\displaystyle q_{ij}={\begin{pmatrix}\alpha _{ij}&\beta _{ij}\\-{\overline {\beta }}_{ij}&{\overline {\alpha }}_{ij}\end{pmatrix}},}$

where α_ij and β_ij are complex numbers.

Each matrix U in Sp(N) is conjugate to a block diagonal matrix with entries

${\displaystyle q_{i}={\begin{pmatrix}z_{i}&0\\0&{\overline {z}}_{i}\end{pmatrix}},}$

where |z_i| = 1. Thus the eigenvalues of U are (z_i^±1). The irreducible representations of Sp(N) are labelled by signatures

${\displaystyle \mathbf {f} \,\colon \,f_{1}\geq f_{2}\geq \cdots \geq f_{N}\geq 0}$

where the f_i are integers. The character of the corresponding irreducible representation σ_f is given by^[9]

${\displaystyle \operatorname {Tr} \sigma _{\mathbf {f} }(U)={\det z_{j}^{f_{i}+N-i+1}-z_{j}^{-f_{i}-N+i-1} \over \prod (z_{i}-z_{i}^{-1})\cdot \prod _{i<j}(z_{i}+z_{i}^{-1}-z_{j}-z_{j}^{-1})}.}$

The branching rule from Sp(N) to Sp(N – 1) states that^[10]

${\displaystyle \sigma _{\mathbf {f} }|_{\mathrm {Sp} (N-1)}=\bigoplus _{f_{i}\geq g_{i}\geq f_{i+2}}m(\mathbf {f} ,\mathbf {g} )\sigma _{\mathbf {g} }}$

Here f_{N + 1} = 0 and the multiplicity m(f, g) is given by

${\displaystyle m(\mathbf {f} ,\mathbf {g} )=\prod _{i=1}^{N}(a_{i}-b_{i}+1)}$

where

${\displaystyle a_{1}\geq b_{1}\geq a_{2}\geq b_{2}\geq \cdots \geq a_{N}\geq b_{N}=0}$

is the non-increasing rearrangement of the 2N non-negative integers (f_i), (g_j) and 0.

Example. The branching from U(2N) to Sp(N) relies on two identities of Littlewood:^{[11][12][13][14]}

${\displaystyle {\begin{aligned}&\sum _{f_{1}\geq f_{2}\geq f_{N}\geq 0}\operatorname {Tr} \Pi _{\mathbf {f} ,0}(z_{1},z_{1}^{-1},\ldots ,z_{N},z_{N}^{-1})\cdot \operatorname {Tr} \pi _{\mathbf {f} }(t_{1},\ldots ,t_{N})\\[5pt]={}&\sum _{f_{1}\geq f_{2}\geq f_{N}\geq 0}\operatorname {Tr} \sigma _{\mathbf {f} }(z_{1},\ldots ,z_{N})\cdot \operatorname {Tr} \pi _{\mathbf {f} }(t_{1},\ldots ,t_{N})\cdot \prod _{i<j}(1-z_{i}z_{j})^{-1},\end{aligned}}}$

where Π_f,0 is the irreducible representation of U(2N) with signature f₁ ≥ ··· ≥ f_N ≥ 0 ≥ ··· ≥ 0.

${\displaystyle \prod _{i<j}(1-z_{i}z_{j})^{-1}=\sum _{f_{2i-1}=f_{2i}}\operatorname {Tr} \pi _{f}(z_{1},\ldots ,z_{N}),}$

where f_i ≥ 0.

The branching rule from U(2N) to Sp(N) is given by

${\displaystyle \Pi _{\mathbf {f} ,0}|_{\mathrm {Sp} (N)}=\bigoplus _{\mathbf {h} ,\,\,\mathbf {g} ,\,\,g_{2i-1}=g_{2i}}M(\mathbf {g} ,\mathbf {h} ;\mathbf {f} )\sigma _{\mathbf {h} }}$

where all the signature are non-negative and the coefficient M (g, h; k) is the multiplicity of the irreducible representation π_k of U(N) in the tensor product π_g ${\displaystyle \otimes }$ π_h. It is given combinatorially by the Littlewood–Richardson rule, the number of lattice permutations of the skew diagram k/h of weight g.^[8]

There is an extension of Littlewood's branching rule to arbitrary signatures due to Sundaram (1990, p. 203). The Littlewood–Richardson coefficients M (g, h; f) are extended to allow the signature f to have 2N parts but restricting g to have even column-lengths (g_{2i – 1} = g_2i). In this case the formula reads

${\displaystyle \Pi _{\mathbf {f} }|_{\operatorname {Sp} (N)}=\bigoplus _{\mathbf {h} ,\,\,\mathbf {g} ,\,\,g_{2i-1}=g_{2i}}M_{N}(\mathbf {g} ,\mathbf {h} ;\mathbf {f} )\sigma _{\mathbf {h} }}$

where M_N (g, h; f) counts the number of lattice permutations of f/h of weight g are counted for which 2j + 1 appears no lower than row N + j of f for 1 ≤ j ≤ |g|/2.

Example. The special orthogonal group SO(N) has irreducible ordinary and spin representations labelled by signatures^{[2][7][15][16]}

• ${\displaystyle f_{1}\geq f_{2}\geq \cdots \geq f_{n-1}\geq |f_{n}|}$ for N = 2n;
• ${\displaystyle f_{1}\geq f_{2}\geq \cdots \geq f_{n}\geq 0}$ for N = 2n+1.

The f_i are taken in Z for ordinary representations and in ½ + Z for spin representations. In fact if an orthogonal matrix U has eigenvalues z_i^±1 for 1 ≤ i ≤ n, then the character of the corresponding irreducible representation π_f is given by

${\displaystyle \operatorname {Tr} \,\pi _{\mathbf {f} }(U)={\det(z_{j}^{f_{i}+n-i}+z_{j}^{-f_{i}-n+i}) \over \prod _{i<j}(z_{i}+z_{i}^{-1}-z_{j}-z_{j}^{-1})}}$

for N = 2n and by

${\displaystyle \operatorname {Tr} \pi _{\mathbf {f} }(U)={\det(z_{j}^{f_{i}+1/2+n-i}-z_{j}^{-f_{i}-1/2-n+i}) \over \prod _{i<j}(z_{i}+z_{i}^{-1}-z_{j}-z_{j}^{-1})\cdot \prod _{k}(z_{k}^{1/2}-z_{k}^{-1/2})}}$

for N = 2n+1.

The branching rules from SO(N) to SO(N – 1) state that^[17]

${\displaystyle \pi _{\mathbf {f} }|_{SO(2n)}=\bigoplus _{f_{1}\geq g_{1}\geq f_{2}\geq g_{2}\geq \cdots \geq f_{n-1}\geq g_{n-1}\geq f_{n}\geq |g_{n}|}\pi _{\mathbf {g} }}$

for N = 2n + 1 and

${\displaystyle \pi _{\mathbf {f} }|_{SO(2n-1)}=\bigoplus _{f_{1}\geq g_{1}\geq f_{2}\geq g_{2}\geq \cdots \geq f_{n-1}\geq g_{n-1}\geq |f_{n}|}\pi _{\mathbf {g} }}$

for N = 2n, where the differences f_i − g_i must be integers.

Since the branching rules from ${\displaystyle U(N)}$ to ${\displaystyle U(N-1)}$ or ${\displaystyle SO(N)}$ to ${\displaystyle SO(N-1)}$ have multiplicity one, the irreducible summands corresponding to smaller and smaller N will eventually terminate in one-dimensional subspaces. In this way Gelfand and Tsetlin were able to obtain a basis of any irreducible representation of ${\displaystyle U(N)}$ or ${\displaystyle SO(N)}$ labelled by a chain of interleaved signatures, called a Gelfand–Tsetlin pattern. Explicit formulas for the action of the Lie algebra on the Gelfand–Tsetlin basis are given in Želobenko (1973). Specifically, for ${\displaystyle N=3}$, the Gelfand-Testlin basis of the irreducible representation of ${\displaystyle SO(3)}$ with dimension ${\displaystyle 2l+1}$ is given by the complex spherical harmonics ${\displaystyle \{Y_{m}^{l}|-l\leq m\leq l\}}$.

For the remaining classical group ${\displaystyle Sp(N)}$, the branching is no longer multiplicity free, so that if V and W are irreducible representation of ${\displaystyle Sp(N-1)}$ and ${\displaystyle Sp(N)}$ the space of intertwiners ${\displaystyle Hom_{Sp(N-1)}(V,W)}$ can have dimension greater than one. It turns out that the Yangian ${\displaystyle Y({\mathfrak {gl}}_{2})}$, a Hopf algebra introduced by Ludwig Faddeev and collaborators, acts irreducibly on this multiplicity space, a fact which enabled Molev (2006) to extend the construction of Gelfand–Tsetlin bases to ${\displaystyle Sp(N)}$.^[18]

In 1937 Alfred H. Clifford proved the following result on the restriction of finite-dimensional irreducible representations from a group G to a normal subgroup N of finite index:^[19]

Theorem. Let π: G ${\displaystyle \rightarrow }$ GL(n,K) be an irreducible representation with K a field. Then the restriction of π to N breaks up into a direct sum of irreducible representations of N of equal dimensions. These irreducible representations of N lie in one orbit for the action of G by conjugation on the equivalence classes of irreducible representations of N. In particular the number of distinct summands is no greater than the index of N in G.

Twenty years later George Mackey found a more precise version of this result for the restriction of irreducible unitary representations of locally compact groups to closed normal subgroups in what has become known as the "Mackey machine" or "Mackey normal subgroup analysis".^[20]

From the point of view of category theory, restriction is an instance of a forgetful functor. This functor is exact, and its left adjoint functor is called induction. The relation between restriction and induction in various contexts is called the Frobenius reciprocity. Taken together, the operations of induction and restriction form a powerful set of tools for analyzing representations. This is especially true whenever the representations have the property of complete reducibility, for example, in representation theory of finite groups over a field of characteristic zero.

This rather evident construction may be extended in numerous and significant ways. For instance we may take any group homomorphism φ from H to G, instead of the inclusion map, and define the restricted representation of H by the composition

${\displaystyle \rho \circ \varphi \,}$

We may also apply the idea to other categories in abstract algebra: associative algebras, rings, Lie algebras, Lie superalgebras, Hopf algebras to name some. Representations or modules restrict to subobjects, or via homomorphisms.

• Goodman, Roe; Wallach, Nolan (1998), Representations and Invariants of the Classical Groups, Encyclopedia Math. Appl., vol. 68, Cambridge University Press
• Helgason, Sigurdur (1978), Differential geometry, Lie groups and symmetric spaces, Academic Press
• Helgason, Sigurdur (1984), Groups and geometric analysis: Integral geometry, invariant differential operators, and spherical functions, Pure and Applied Mathematics, vol. 113, Academic Press, ISBN 978-0-12-338301-3
• Howe, Roger (1995), Perspectives on invariant theory, The Schur Lectures, 1992, Israel Math. Conf. Proc., vol. 8, American Mathematical Society, pp. 1–182
• Howe, Roger; Tan, Eng-Chye; Willenbring, Jeb F. (2005), "Stable branching rules for classical symmetric pairs", Trans. Amer. Math. Soc., 357 (4): 1601–1626, doi:10.1090/S0002-9947-04-03722-5
• Hua, L.K. (1963), Harmonic analysis of functions of several complex variables in the classical domains, American Mathematical Society
• Knapp, Anthony W. (2003), "Geometric interpretations of two branching theorems of D. E. Littlewood", Journal of Algebra, 270 (2): 728–754, doi:10.1016/j.jalgebra.2002.11.001
• Koike, Kazuhiko; Terada, Itaru (1987), "Young-diagrammatic methods for the representation theory of the classical groups of type B_n, C_n, D_n", Journal of Algebra, 107 (2): 466–511, doi:10.1016/0021-8693(87)90099-8
• Kostant, Betram (2004), A branching law for subgroups fixed by an involution and a noncompact analogue of the Borel-Weil theorem, Progr. Math., vol. 220, Birkhäuser, pp. 291–353, arXiv:math.RT/0205283, Bibcode:2002math......5283K
• Littelmann, Peter (1995), "Paths and Root Operators in Representation Theory", Annals of Mathematics, 142 (3): 499–525, doi:10.2307/2118553, JSTOR 2118553
• Littlewood, Dudley E. (1950), The Theory of Group Characters and Matrix Representations of Groups, Oxford University Press
• Macdonald, Ian G. (1979), Symmetric Functions and Hall Polynomials, Oxford University Press
• Molev, A. I. (1999), "A basis for representations of symplectic Lie algebras", Comm. Math. Phys., 201 (3): 591–618, arXiv:math/9804127, Bibcode:1999CMaPh.201..591M, doi:10.1007/s002200050570, S2CID 17990182
• Molev, A. I. (2006), "Gelfand-Tsetlin bases for classical Lie algebras", In "Handbook of Algebra", Vol. 4, (M. Hazewinkel, Ed.), Elsevier, Pp. 109-170, Handbook of algebra, 4, Elsevier: 109–170, arXiv:math/0211289, Bibcode:2002math.....11289M, ISBN 978-0-444-52213-9
• Murnaghan, Francis D. (1938), The Theory of Group Representations, Johns Hopkins Press
• Slansky, Richard (1981), "Group Theory for Unified Model Building", Physics Reports, 79 (1): 1–128, Bibcode:1981PhR....79....1S, CiteSeerX 10.1.1.126.1581, doi:10.1016/0370-1573(81)90092-2 available online
• Sundaram, Sheila (1990), "Tableaux in the representation theory of the classical Lie groups", Institute for Mathematics and Its Applications, IMA Vol. Math. Appl., 19: 191–225, Bibcode:1990IMA....19..191S
• Weyl, Hermann (1946), The classical groups, Princeton University Press
• Želobenko, D. P. (1973), Compact Lie groups and their representations, Translations of Mathematical Monographs, vol. 40, American Mathematical Society