In 1988 he joined Leicester Tigers from Nuneaton and made his first-team debut against Oxford University in 1990. He was given the nickname Fruit Pastille Man early in his career, owing to sharing a surname with the confectionery brand that manufactures the well known sweet. For much of that time he was in harness with the famous ‘ABC club’ alongside Richard Cockerill and Darren Garforth. At Leicester Rowntree enjoyed great domestic success, and started both the 2001[2] and 2002[3] Heineken Cup finals.
After the 1999 World Cup Graham was not capped for almost 2 years until a series of fine performances for his club forced him back into international contention. He was prominent throughout the pre-2003 Rugby World Cup years. He was selected for the England squad to tour Canada and the US in 2001, participated in all that season's Autumn internationals — being named as man of the match in England's 21–15 Cook Cup victory over Australia — and started in each of the 2002 Six Nations games. He started in England's 15–13 win over New Zealand in Wellington in 2003 and put in a particularly memorable performance against the All Blacks when England's pack was reduced to just 6 men. Despite playing in the 2003 pre-World Cup trial match in France, Rowntree was omitted from Clive Woodward's squad, which won the tournament. Clive Woodward admitted that leaving Rowntree behind was one of the hardest decisions he had to make in his time as England head coach.
Rowntree retired from rugby in 2007 after 17 years playing the game, and he joined the Tigers coaching team where he made a rapid rise up the coaching ranks, becoming the forwards/scrum coach for the English national team ahead of the 2008 Six Nations Championship. He toured South Africa with the British & Irish Lions in 2009, acting as scrum coach, before being named forwards coach for the winning test series against Australia in 2013 and scrum coach for the drawn series with New Zealand in 2017.
Following the resignation of head coach Stuart Lancaster on 11 November 2015, newly appointed head coach Eddie Jones sacked the whole coaching team, with Rowntree leaving his post with England after eight years in December 2015. At the end of the 2015–16 English Premiership season, it was announced that Harlequins had appointed Rowntree as their new forwards coach.[4] Rowntree joined the Georgian national team in 2018, but left after the 2019 Rugby World Cup to join Irish province Munster as their new forwards coach.[5][6]
Rowntree signed a two-year contract extension with Munster in January 2022,[7] and, in April 2022, the province confirmed that Rowntree would replace Johann van Graan as head coach from the 2022–23 season.[8] Munster won silverware in Rowntree's first season as head coach, ending a twelve year trophy drought when they defeated South African side the Stormers 19–14 away from home in the final of the 2022–23 United Rugby Championship on 27 May 2023.[9] Munster confirmed a two-year contract extension with Rowntree in September 2023.[10]
On 29 October 2024, Munster announced the departure of Rowntree by mutual agreement after four losses in six games in the 2024-25 United Rugby Championship.[11]