For RC5P, analysis was conducted modulo 3. It was observed that the operations in the cipher (rotation and addition, both on 32-bit words) were somewhat biased over congruence classes mod 3. To illustrate the approach, consider left rotation by a single bit:
Then, because
it follows that
Thus left rotation by a single bit has a simple description modulo 3. Analysis of other operations (data dependent rotation and modular addition) reveals similar, notable biases. Although there are some theoretical problems analysing the operations in combination, the bias can be detected experimentally for the entire cipher. In (Kelsey et al., 1999), experiments were conducted up to seven rounds, and based on this they conjecture that as many as 19 or 20 rounds of RC5P can be distinguished from random using this attack. There is also a corresponding method for recovering the secret key.
Against M6 there are attacks mod 5 and mod 257 that are even more effective.
Toshio Tokita; Tsutomu Matsumoto. "On Applicability of Differential Cryptanalysis, Linear Cryptanalysis and Mod n Cryptanalysis to an Encryption Algorithm M8 (ISO9979-20)". Ipsj Journal. 42 (8).