In group theory, a field of mathematics, a double coset is a collection of group elements which are equivalent under the symmetries coming from two subgroups, generalizing the notion of a single coset.^[1][2]

Let G be a group, and let H and K be subgroups. Let H act on G by left multiplication and let K act on G by right multiplication. For each x in G, the (H, K)-double coset of x is the set

${\displaystyle HxK=\{hxk\colon h\in H,k\in K\}.}$

When H = K, this is called the H-double coset of x. Equivalently, HxK is the equivalence class of x under the equivalence relation

x ~ y if and only if there exist h in H and k in K such that hxk = y.

The set of all ${\displaystyle (H,K)}$-double cosets is denoted by ${\displaystyle H\,\backslash G/K.}$

Suppose that G is a group with subgroups H and K acting by left and right multiplication, respectively. The (H, K)-double cosets of G may be equivalently described as orbits for the product group H × K acting on G by (h, k) ⋅ x = hxk⁻¹. Many of the basic properties of double cosets follow immediately from the fact that they are orbits. However, because G is a group and H and K are subgroups acting by multiplication, double cosets are more structured than orbits of arbitrary group actions, and they have additional properties that are false for more general actions.

• Two double cosets HxK and HyK are either disjoint or identical.
• G is the disjoint union of its double cosets.
• There is a one-to-one correspondence between the two double coset spaces H \ G / K and K \ G / H given by identifying HxK with Kx⁻¹H.
• If H = {1}, then H \ G / K = G / K. If K = {1}, then H \ G / K = H \ G.
• A double coset HxK is a union of right cosets of H and left cosets of K; specifically,

  ${\displaystyle {\begin{aligned}HxK&=\bigcup _{k\in K}Hxk=\coprod _{Hxk\,\in \,H\backslash HxK}Hxk,\\HxK&=\bigcup _{h\in H}hxK=\coprod _{hxK\,\in \,HxK/K}hxK.\end{aligned}}}$

• The set of (H, K)-double cosets is in bijection with the orbits H \ (G / K), and also with the orbits (H \ G) / K under the mappings ${\displaystyle HgK\to H(gK)}$ and ${\displaystyle HgK\to (Hg)K}$ respectively.
• If H is normal, then H \ G is a group, and the right action of K on this group factors through the right action of H \ HK. It follows that H \ G / K = G / HK. Similarly, if K is normal, then H \ G / K = HK \ G.
• If H is a normal subgroup of G, then the H-double cosets are in one-to-one correspondence with the left (and right) H-cosets.
• Consider HxK as the union of a K-orbit of right H-cosets. The stabilizer of the right H-coset Hxk ∈ H \ HxK with respect to the right action of K is K ∩ (xk)⁻¹Hxk. Similarly, the stabilizer of the left K-coset hxK ∈ HxK / K with respect to the left action of H is H ∩ hxK(hx)⁻¹.
• It follows that the number of right cosets of H contained in HxK is the index [K : K ∩ x⁻¹Hx] and the number of left cosets of K contained in HxK is the index [H : H ∩ xKx⁻¹]. Therefore

  ${\displaystyle {\begin{aligned}|HxK|&=[H:H\cap xKx^{-1}]|K|=|H|[K:K\cap x^{-1}Hx],\\\left[G:H\right]&=\sum _{HxK\,\in \,H\backslash G/K}[K:K\cap x^{-1}Hx],\\\left[G:K\right]&=\sum _{HxK\,\in \,H\backslash G/K}[H:H\cap xKx^{-1}].\end{aligned}}}$

• If G, H, and K are finite, then it also follows that

  ${\displaystyle {\begin{aligned}|HxK|&={\frac {|H||K|}{|H\cap xKx^{-1}|}}={\frac {|H||K|}{|K\cap x^{-1}Hx|}},\\\left[G:H\right]&=\sum _{HxK\,\in \,H\backslash G/K}{\frac {|K|}{|K\cap x^{-1}Hx|}},\\\left[G:K\right]&=\sum _{HxK\,\in \,H\backslash G/K}{\frac {|H|}{|H\cap xKx^{-1}|}}.\end{aligned}}}$

• Fix x in G, and let (H × K)_x denote the double stabilizer {(h, k) : hxk = x}. Then the double stabilizer is a subgroup of H × K.
• Because G is a group, for each h in H there is precisely one g in G such that hxg = x, namely g = x⁻¹h⁻¹x; however, g may not be in K. Similarly, for each k in K there is precisely one g′ in G such that g′xk = x, but g′ may not be in H. The double stabilizer therefore has the descriptions

  ${\displaystyle (H\times K)_{x}=\{(h,x^{-1}h^{-1}x)\colon h\in H\}\cap H\times K=\{(xk^{-1}x^{-1},k)\colon k\in K\}\cap H\times K.}$

• (Orbit–stabilizer theorem) There is a bijection between HxK and (H × K) / (H × K)_x under which hxk corresponds to (h, k⁻¹)(H × K)_x. It follows that if G, H, and K are finite, then

  ${\displaystyle |HxK|=[H\times K:(H\times K)_{x}]=|H\times K|/|(H\times K)_{x}|.}$

• (Cauchy–Frobenius lemma) Let G^{(h, k)} denote the elements fixed by the action of (h, k). Then

  ${\displaystyle |H\,\backslash G/K|={\frac {1}{|H||K|}}\sum _{(h,k)\in H\times K}|G^{(h,k)}|.}$

• In particular, if G, H, and K are finite, then the number of double cosets equals the average number of points fixed per pair of group elements.

There is an equivalent description of double cosets in terms of single cosets. Let H and K both act by right multiplication on G. Then G acts by left multiplication on the product of coset spaces G / H × G / K. The orbits of this action are in one-to-one correspondence with H \ G / K. This correspondence identifies (xH, yK) with the double coset Hx⁻¹yK. Briefly, this is because every G-orbit admits representatives of the form (H, xK), and the representative x is determined only up to left multiplication by an element of H. Similarly, G acts by right multiplication on H \ G × K \ G, and the orbits of this action are in one-to-one correspondence with the double cosets H \ G / K. Conceptually, this identifies the double coset space H \ G / K with the space of relative configurations of an H-coset and a K-coset. Additionally, this construction generalizes to the case of any number of subgroups. Given subgroups H₁, ..., H_n, the space of (H₁, ..., H_n)-multicosets is the set of G-orbits of G / H₁ × ... × G / H_n.

The analog of Lagrange's theorem for double cosets is false. This means that the size of a double coset need not divide the order of G. For example, let G = S₃ be the symmetric group on three letters, and let H and K be the cyclic subgroups generated by the transpositions (1 2) and (1 3), respectively. If e denotes the identity permutation, then

${\displaystyle HeK=HK=\{e,(12),(13),(132)\}.}$

This has four elements, and four does not divide six, the order of S₃. It is also false that different double cosets have the same size. Continuing the same example,

${\displaystyle H(23)K=\{(23),(123)\},}$

which has two elements, not four.

However, suppose that H is normal. As noted earlier, in this case the double coset space equals the left coset space G / HK. Similarly, if K is normal, then H \ G / K is the right coset space HK \ G. Standard results about left and right coset spaces then imply the following facts.

• |HxK| = |HK| for all x in G. That is, all double cosets have the same cardinality.
• If G is finite, then |G| = |HK| ⋅ |H \ G / K|. In particular, |HK| and |H \ G / K| divide |G|.

• Let G = S_n be the symmetric group, considered as permutations of the set {1, ..., n}. Consider the subgroup H = S_n−1 which stabilizes n. Then S_n−1 \ S_n / S_n−1 consists of two double cosets. One of these is H = S_n−1, and the other is S_n−1 γ S_n−1 for any permutation γ which does not fix n. This is contrasted with S_n / S_n−1, which has ${\displaystyle n}$ elements ${\displaystyle \gamma _{1}S_{n-1},\gamma _{2}S_{n-1},...,\gamma _{n}S_{n-1}}$, where each ${\displaystyle \gamma _{i}(n)=i}$.
• Let G be the group GL_n(R), and let B be the subgroup of upper triangular matrices. The double coset space B \ G / B is the Bruhat decomposition of G. The double cosets are exactly BwB, where w ranges over all n-by-n permutation matrices. For instance, if n = 2, then

  ${\displaystyle B\,\backslash \!\operatorname {GL} _{2}(\mathbf {R} )/B=\left\{B{\begin{pmatrix}1&0\\0&1\end{pmatrix}}B,\ B{\begin{pmatrix}0&1\\1&0\end{pmatrix}}B\right\}.}$

Suppose that G is a group and that H, K, and L are subgroups. Under certain finiteness conditions, there is a product on the free abelian group generated by the (H, K)- and (K, L)-double cosets with values in the free abelian group generated by the (H, L)-double cosets. This means there is a bilinear function

${\displaystyle \mathbf {Z} [H\backslash G/K]\times \mathbf {Z} [K\backslash G/L]\to \mathbf {Z} [H\backslash G/L].}$

Assume for simplicity that G is finite. To define the product, reinterpret these free abelian groups in terms of the group algebra of G as follows. Every element of Z[H \ G / K] has the form

${\displaystyle \sum _{HxK\in H\backslash G/K}f_{HxK}\cdot [HxK],}$

where { f_HxK } is a set of integers indexed by the elements of H \ G / K. This element may be interpreted as a Z-valued function on H \ G / K, specifically, HxK ↦ f_HxK. This function may be pulled back along the projection G → H \ G / K which sends x to the double coset HxK. This results in a function x ↦ f_HxK. By the way in which this function was constructed, it is left invariant under H and right invariant under K. The corresponding element of the group algebra Z[G] is

${\displaystyle \sum _{x\in G}f_{HxK}\cdot [x],}$

and this element is invariant under left multiplication by H and right multiplication by K. Conceptually, this element is obtained by replacing HxK by the elements it contains, and the finiteness of G ensures that the sum is still finite. Conversely, every element of Z[G] which is left invariant under H and right invariant under K is the pullback of a function on Z[H \ G / K]. Parallel statements are true for Z[K \ G / L] and Z[H \ G / L].

When elements of Z[H \ G / K], Z[K \ G / L], and Z[H \ G / L] are interpreted as invariant elements of Z[G], then the product whose existence was asserted above is precisely the multiplication in Z[G]. Indeed, it is trivial to check that the product of a left-H-invariant element and a right-L-invariant element continues to be left-H-invariant and right-L-invariant. The bilinearity of the product follows immediately from the bilinearity of multiplication in Z[G]. It also follows that if M is a fourth subgroup of G, then the product of (H, K)-, (K, L)-, and (L, M)-double cosets is associative. Because the product in Z[G] corresponds to convolution of functions on G, this product is sometimes called the convolution product.

An important special case is when H = K = L. In this case, the product is a bilinear function

${\displaystyle \mathbf {Z} [H\backslash G/H]\times \mathbf {Z} [H\backslash G/H]\to \mathbf {Z} [H\backslash G/H].}$

This product turns Z[H \ G / H] into an associative ring whose identity element is the class of the trivial double coset [H]. In general, this ring is non-commutative. For example, if H = {1}, then the ring is the group algebra Z[G], and a group algebra is a commutative ring if and only if the underlying group is abelian.

If H is normal, so that the H-double cosets are the same as the elements of the quotient group G / H, then the product on Z[H \ G / H] is the product in the group algebra Z[G / H]. In particular, it is the usual convolution of functions on G / H. In this case, the ring is commutative if and only if G / H is abelian, or equivalently, if and only if H contains the commutator subgroup of G.

If H is not normal, then Z[H \ G / H] may be commutative even if G is non-abelian. A classical example is the product of two Hecke operators. This is the product in the Hecke algebra, which is commutative even though the group G is the modular group, which is non-abelian, and the subgroup is an arithmetic subgroup and in particular does not contain the commutator subgroup. Commutativity of the convolution product is closely tied to Gelfand pairs.

When the group G is a topological group, it is possible to weaken the assumption that the number of left and right cosets in each double coset is finite. The group algebra Z[G] is replaced by an algebra of functions such as L²(G) or C^∞(G), and the sums are replaced by integrals. The product still corresponds to convolution. For instance, this happens for the Hecke algebra of a locally compact group.

When a group ${\displaystyle G}$ has a transitive group action on a set ${\displaystyle S}$, computing certain double coset decompositions of ${\displaystyle G}$ reveals extra information about structure of the action of ${\displaystyle G}$ on ${\displaystyle S}$. Specifically, if ${\displaystyle H}$ is the stabilizer subgroup of some element ${\displaystyle s\in S}$, then ${\displaystyle G}$ decomposes as exactly two double cosets of ${\displaystyle (H,H)}$ if and only if ${\displaystyle G}$ acts transitively on the set of distinct pairs of ${\displaystyle S}$. See 2-transitive groups for more information about this action.

Double cosets are important in connection with representation theory, when a representation of H is used to construct an induced representation of G, which is then restricted to K. The corresponding double coset structure carries information about how the resulting representation decomposes. In the case of finite groups, this is Mackey's decomposition theorem.

They are also important in functional analysis, where in some important cases functions left-invariant and right-invariant by a subgroup K can form a commutative ring under convolution: see Gelfand pair.

In geometry, a Clifford–Klein form is a double coset space Γ\G/H, where G is a reductive Lie group, H is a closed subgroup, and Γ is a discrete subgroup (of G) that acts properly discontinuously on the homogeneous space G/H.

In number theory, the Hecke algebra corresponding to a congruence subgroup Γ of the modular group is spanned by elements of the double coset space ${\displaystyle \Gamma \backslash \mathrm {GL} _{2}^{+}(\mathbb {Q} )/\Gamma }$; the algebra structure is that acquired from the multiplication of double cosets described above. Of particular importance are the Hecke operators ${\displaystyle T_{m}}$ corresponding to the double cosets ${\displaystyle \Gamma _{0}(N)g\Gamma _{0}(N)}$ or ${\displaystyle \Gamma _{1}(N)g\Gamma _{1}(N)}$, where ${\displaystyle g=\left({\begin{smallmatrix}1&0\\0&m\end{smallmatrix}}\right)}$ (these have different properties depending on whether m and N are coprime or not), and the diamond operators ${\displaystyle \langle d\rangle }$ given by the double cosets ${\displaystyle \Gamma _{1}(N)\left({\begin{smallmatrix}a&b\\c&d\end{smallmatrix}}\right)\Gamma _{1}(N)}$ where ${\displaystyle d\in (\mathbb {Z} /N\mathbb {Z} )^{\times }}$ and we require ${\displaystyle \left({\begin{smallmatrix}a&b\\c&d\end{smallmatrix}}\right)\in \Gamma _{0}(N)}$ (the choice of a, b, c does not affect the answer).