In mathematics, the indefinite orthogonal group, O(p, q) is the Lie group of all linear transformations of an n-dimensional real vector space that leave invariant a nondegenerate, symmetric bilinear form of signature (p, q), where n = p + q. It is also called the pseudo-orthogonal group^[1] or generalized orthogonal group.^[2] The dimension of the group is n(n − 1)/2.

The indefinite special orthogonal group, SO(p, q) is the subgroup of O(p, q) consisting of all elements with determinant 1. Unlike in the definite case, SO(p, q) is not connected – it has 2 components – and there are two additional finite index subgroups, namely the connected SO⁺(p, q) and O⁺(p, q), which has 2 components – see § Topology for definition and discussion.

The signature of the form determines the group up to isomorphism; interchanging p with q amounts to replacing the metric by its negative, and so gives the same group. If either p or q equals zero, then the group is isomorphic to the ordinary orthogonal group O(n). We assume in what follows that both p and q are positive.

The group O(p, q) is defined for vector spaces over the reals. For complex spaces, all groups O(p, q; C) are isomorphic to the usual orthogonal group O(p + q; C), since the transform ${\displaystyle z_{j}\mapsto iz_{j}}$ changes the signature of a form. This should not be confused with the indefinite unitary group U(p, q) which preserves a sesquilinear form of signature (p, q).

In even dimension n = 2p, O(p, p) is known as the split orthogonal group.

The basic example is the squeeze mappings, which is the group SO⁺(1, 1) of (the identity component of) linear transforms preserving the unit hyperbola. Concretely, these are the matrices ${\displaystyle \left[{\begin{smallmatrix}\cosh(\alpha )&\sinh(\alpha )\\\sinh(\alpha )&\cosh(\alpha )\end{smallmatrix}}\right],}$ and can be interpreted as hyperbolic rotations, just as the group SO(2) can be interpreted as circular rotations.

In physics, the Lorentz group O(1,3) is of central importance, being the setting for electromagnetism and special relativity. (Some texts use O(3,1) for the Lorentz group; however, O(1,3) is prevalent in quantum field theory because the geometric properties of the Dirac equation are more natural in O(1,3).)

One can define O(p, q) as a group of matrices, just as for the classical orthogonal group O(n). Consider the ${\displaystyle (p+q)\times (p+q)}$ diagonal matrix ${\displaystyle g}$ given by

${\displaystyle g=\mathrm {diag} (\underbrace {1,\ldots ,1} _{p},\underbrace {-1,\ldots ,-1} _{q}).}$

Then we may define a symmetric bilinear form ${\displaystyle [\cdot ,\cdot ]_{p,q}}$ on ${\displaystyle \mathbb {R} ^{p+q}}$ by the formula

${\displaystyle [x,y]_{p,q}=\langle x,gy\rangle =x_{1}y_{1}+\cdots +x_{p}y_{p}-x_{p+1}y_{p+1}-\cdots -x_{p+q}y_{p+q}}$,

where ${\displaystyle \langle \cdot ,\cdot \rangle }$ is the standard inner product on ${\displaystyle \mathbb {R} ^{p+q}}$.

We then define ${\displaystyle \mathrm {O} (p,q)}$ to be the group of ${\displaystyle (p+q)\times (p+q)}$ matrices that preserve this bilinear form:^[3]

${\displaystyle \mathrm {O} (p,q)=\{A\in M_{p+q}(\mathbb {R} )|[Ax,Ay]_{p,q}=[x,y]_{p,q}\,\forall x,y\in \mathbb {R} ^{p+q}\}}$.

More explicitly, ${\displaystyle \mathrm {O} (p,q)}$ consists of matrices ${\displaystyle A}$ such that^[4]

${\displaystyle gA^{\mathrm {tr} }g=A^{-1}}$,

where ${\displaystyle A^{\mathrm {tr} }}$ is the transpose of ${\displaystyle A}$.

One obtains an isomorphic group (indeed, a conjugate subgroup of GL(p + q)) by replacing g with any symmetric matrix with p positive eigenvalues and q negative ones. Diagonalizing this matrix gives a conjugation of this group with the standard group O(p, q).

The group SO⁺(p, q) and related subgroups of O(p, q) can be described algebraically. Partition a matrix L in O(p, q) as a block matrix:

${\displaystyle L={\begin{pmatrix}A&B\\C&D\end{pmatrix}}}$

where A, B, C, and D are p×p, p×q, q×p, and q×q blocks, respectively. It can be shown that the set of matrices in O(p, q) whose upper-left p×p block A has positive determinant is a subgroup. Or, to put it another way, if

${\displaystyle L={\begin{pmatrix}A&B\\C&D\end{pmatrix}}\;\mathrm {and} \;M={\begin{pmatrix}W&X\\Y&Z\end{pmatrix}}}$

are in O(p, q), then

${\displaystyle (\operatorname {sgn} \det A)(\operatorname {sgn} \det W)=\operatorname {sgn} \det(AW+BY).}$

The analogous result for the bottom-right q×q block also holds. The subgroup SO⁺(p, q) consists of matrices L such that det A and det D are both positive.^[5][6]

For all matrices L in O(p, q), the determinants of A and D have the property that ${\textstyle {\frac {\det A}{\det D}}=\det L}$ and that ${\displaystyle |{\det A}|=|{\det D}|\geq 1.}$^[7] In particular, the subgroup SO(p, q) consists of matrices L such that det A and det D have the same sign.^[5]

Assuming both p and q are positive, neither of the groups O(p, q) nor SO(p, q) are connected, having four and two components respectively. π₀(O(p, q)) ≅ C₂ × C₂ is the Klein four-group, with each factor being whether an element preserves or reverses the respective orientations on the p and q dimensional subspaces on which the form is definite; note that reversing orientation on only one of these subspaces reverses orientation on the whole space. The special orthogonal group has components π₀(SO(p, q)) = {(1, 1), (−1, −1)}, each of which either preserves both orientations or reverses both orientations, in either case preserving the overall orientation.^{[clarification needed]}

The identity component of O(p, q) is often denoted SO⁺(p, q) and can be identified with the set of elements in SO(p, q) that preserve both orientations. This notation is related to the notation O⁺(1, 3) for the orthochronous Lorentz group, where the + refers to preserving the orientation on the first (temporal) dimension.

The group O(p, q) is also not compact, but contains the compact subgroups O(p) and O(q) acting on the subspaces on which the form is definite. In fact, O(p) × O(q) is a maximal compact subgroup of O(p, q), while S(O(p) × O(q)) is a maximal compact subgroup of SO(p, q). Likewise, SO(p) × SO(q) is a maximal compact subgroup of SO⁺(p, q). Thus, the spaces are homotopy equivalent to products of (special) orthogonal groups, from which algebro-topological invariants can be computed. (See Maximal compact subgroup.)

In particular, the fundamental group of SO⁺(p, q) is the product of the fundamental groups of the components, π₁(SO⁺(p, q)) = π₁(SO(p)) × π₁(SO(q)), and is given by:

π1(SO+(p, q))	p = 1	p = 2	p ≥ 3
q = 1	C1	Z	C2
q = 2	Z	Z × Z	Z × C2
q ≥ 3	C2	C2 × Z	C2 × C2

In even dimensions, the middle group O(n, n) is known as the split orthogonal group, and is of particular interest, as it occurs as the group of T-duality transformations in string theory, for example. It is the split Lie group corresponding to the complex Lie algebra so_2n (the Lie group of the split real form of the Lie algebra); more precisely, the identity component is the split Lie group, as non-identity components cannot be reconstructed from the Lie algebra. In this sense it is opposite to the definite orthogonal group O(n) := O(n, 0) = O(0, n), which is the compact real form of the complex Lie algebra.

The group SO(1, 1) may be identified with the group of unit split-complex numbers.

In terms of being a group of Lie type – i.e., construction of an algebraic group from a Lie algebra – split orthogonal groups are Chevalley groups, while the non-split orthogonal groups require a slightly more complicated construction, and are Steinberg groups.

Split orthogonal groups are used to construct the generalized flag variety over non-algebraically closed fields.

This section needs expansion. You can help by adding to it. (March 2011)

• Orthogonal group
• Lorentz group
• Poincaré group
• Symmetric bilinear form