In ethology, the nasty neighbour effect describes the phenomenon whereby territory-holding animals behave more strongly toward familiar conspecific neighbours than to unfamiliar conspecifics. This phenomenon may be generally advantageous to an animal because the heightened response reduces the likelihood of a nearby intruder entering the territory and taking the resources it contains whereas an unfamiliar or distant territory-holder poses less of a threat. This reduced response minimises the time, energy, and risk of injury incurred during territorial encounters with animals which are less of a threat to the territory holder. The nasty neighbour effect is the converse of the dear enemy effect in which some species are less aggressive toward their neighbours than toward unfamiliar strangers.

The four-striped grass mouse (Rhabdomys pumilio) is group living with one single breeding male and up to four communally breeding females per group. Groups typically contain several philopatric adult sons (and daughters) that are believed not to breed in their natal group and all group members participate in territorial defence. When aggression in wild group-living male breeders was tested in a neutral test arena, they were nearly five times more aggressive toward their neighbours than toward strangers, leading to the prediction that neighbours are the most important competitors for paternity. Using a molecular parentage analysis it was shown that 28% of offspring are sired by neighbouring males and only 7% by strangers.^[1]

Colonies of the weaver ant (Oecophylla smaragdina) are able to recognize a greater proportion of workers from neighbouring colonies as non-colony members. When recognized as non-colony members, more aggression is exhibited toward neighbours than non-neighbours.^[2]

Banded mongoose (Mungos mungo) groups vocalize more and inspect more scent samples in response to olfactory cues of neighbours than strangers.^[3] It has been suggested that increased aggression toward neighbours is more common in social species with intense competition between neighbours, as opposed to reduced aggression toward neighbours typical for most solitary species. Furthermore, animals may respond in this way when encounters with intruders from non-neighbouring colonies are rare and of little consequence.

Female New Zealand bellbirds (Anthornis melanura) are more aggressive toward the songs of neighbouring females, indicating that neighbouring females pose a greater threat than strangers in this species.^[4]

Female hen harrier (Circus cyaneus) responses toward neighbours are more intense, mostly flights rather than calls, than responses toward female floaters (individuals without territories), which in turn were more intense than responses toward male floaters.^[5]

Guinea baboon (Papio papio) males which live in social groups called "gangs" do not differ in their response behaviour toward neighbouring and stranger males, and largely ignore any non-gang member, irrespective of familiarity; that is, they neither show a "dear enemy" nor "nasty neighbour" effect.^[6]