Most of the village is high upon a ridge overlooking Cwmtwrch and most of the houses are over 900 feet (270 m) above sea level. The majority of the houses date from the 1930s and were built by the local authority, then Glamorgan, now Neath Port Talbot council to house coal miners for a local colliery that has since been closed and returned to countryside. The village has been described as "a giant family".
Rhiwfawr is home to many Welsh language-speakers, mainly due to the Welsh-speaking school in its heart - Ysgol Gynradd Gymraeg Draddodiadol Rhiwfawr (English: Rhiwfawr Traditional Welsh Primary School). The school is attended by 30 - 40 children. The school celebrated its centenary in 2006. School now closed and is now residential.
