In representation theory, the category of representations of some algebraic structure A has the representations of A as objects and equivariant maps as morphisms between them. One of the basic thrusts of representation theory is to understand the conditions under which this category is semisimple; i.e., whether an object decomposes into simple objects (see Maschke's theorem for the case of finite groups).

The Tannakian formalism gives conditions under which a group G may be recovered from the category of representations of it together with the forgetful functor to the category of vector spaces.^[1]

The Grothendieck ring of the category of finite-dimensional representations of a group G is called the representation ring of G.

Depending on the types of the representations one wants to consider, it is typical to use slightly different definitions.

For a finite group G and a field F, the category of representations of G over F has

• Objects: Pairs (V, f) of vector spaces V over F and representations f of G on that vector space
• Morphisms: Equivariant maps
• Composition: The composition of equivariant maps
• Identities: The identity function (which is an equivariant map).

The category is denoted by ${\displaystyle \operatorname {Rep} _{F}(G)}$ or ${\displaystyle \operatorname {Rep} (G)}$.

For a Lie group, one typically requires the representations to be smooth or admissible. For the case of a Lie algebra, see Lie algebra representation. See also: category O.

There is an isomorphism of categories between the category of representations of a group G over a field F (described above) and the category of modules over the group ring F[G], denoted F[G]-Mod.

Every group G can be viewed as a category with a single object, where morphisms in this category are the elements of G and composition is given by the group operation; so G is the automorphism group of the unique object. Given an arbitrary category C, a representation of G in C is a functor from G to C. Such a functor sends the unique object to an object say X in C and induces a group homomorphism ${\displaystyle G\to \operatorname {Aut} (X)}$; see Automorphism group#In category theory for more. For example, a G-set is equivalent to a functor from G to Set, the category of sets, and a linear representation is equivalent to a functor to Vect_F, the category of vector spaces over a field F.^[2]

In this setting, the category of linear representations of G over F is the functor category G → Vect_F, which has natural transformations as its morphisms.

This section needs expansion. You can help by adding to it. (November 2017)

The category of linear representations of a group has a monoidal structure given by the tensor product of representations, which is an important ingredient in Tannaka-Krein duality (see below).

Maschke's theorem states that when the characteristic of F doesn't divide the order of G, the category of representations of G over F is semisimple.

Given a group G with a subgroup H, there are two fundamental functors between the categories of representations of G and H (over a fixed field): one is a forgetful functor called the restriction functor

${\displaystyle {\begin{aligned}\operatorname {Res} _{H}^{G}:\operatorname {Rep} (G)&\longrightarrow \operatorname {Rep} (H)\\\pi &\longmapsto \pi |_{H}\end{aligned}}}$

and the other, the induction functor

${\displaystyle \operatorname {Ind} _{H}^{G}:\operatorname {Rep} (H)\to \operatorname {Rep} (G)}$.

When G and H are finite groups, they are adjoint to each other

${\displaystyle \operatorname {Hom} _{G}(\operatorname {Ind} _{H}^{G}W,U)\cong \operatorname {Hom} _{H}(W,\operatorname {Res} _{H}^{G}U)}$,

a theorem called Frobenius reciprocity.

The basic question is whether the decomposition into irreducible representations (simple objects of the category) behaves under restriction or induction. The question may be attacked for instance by the Mackey theory.

Tannaka–Krein duality concerns the interaction of a compact topological group and its category of linear representations. Tannaka's theorem describes the converse passage from the category of finite-dimensional representations of a group G back to the group G, allowing one to recover the group from its category of representations. Krein's theorem in effect completely characterizes all categories that can arise from a group in this fashion. These concepts can be applied to representations of several different structures, see the main article for details.

• https://ncatlab.org/nlab/show/category+of+representations