In round 23 of the 2004 NRL season Smith made his NRL debut for the Melbourne Storm against the Canberra Raiders. He scored a try on debut. Smith didn't play first-grade in 2005 but became a regular first-grader in 2006. The Storm reached the 2006 NRL Grand Final and Smith was selected to play from the interchange bench in the loss to the Broncos. He played in the 2007 NRL Grand Final win over the Sea Eagles, again selected off the interchange. In 2010, the premiership that Smith won with Melbourne was stripped by the NRL due to the club's major and deliberate breaches of the salary cap.[6]
After winning the 2007 NRL Grand Final with the Melbourne Storm, Smith was selected for the 2007 All Golds tour. Not to be confused with compatriot and namesake Jeremy James Smith, both Jeremy Smiths were named in the rugby league Centenary clash of 14 October 2007, Jeremy James Smith in the number 7 jersey and Jeremy Jon Oscar Smith in the number 13 jersey.[7][8]
In January 2009, Smith was named the 2008 New Zealand Rugby League Player of the Year.[15] In his first year at the Dragons, the team won the 2009 Minor Premiership after finishing first in the regular season and again won the Minor Premiership in 2010.
In August 2010, Smith signed a 3-year contract with the Cronulla-Sutherland Sharks starting in 2011.[16] Smith played at lock in the Dragons' team that won the 2010 NRL Grand Final, defeating the Sydney Roosters. This was Smith's first valid premiership. After that win, he had played in more grand finals than any other New Zealander.[17] After winning the 2010 NRL Grand Final with the St. George Illawarra Dragons, Smith was a part of New Zealand's successful 2010 Four Nations campaign.
Cronulla-Sutherland Sharks
Despite having a disappointing year with the Cronulla-Sutherland Sharks in 2011, Smith was again chosen to represent New Zealand in the 2011 ANZAC Test and 2011 Four Nations. After a successful start to year with the Sharks, Smith was selected in the 2012 ANZAC Test. On 26 September 2012, Smith signed a three-year contract with the Newcastle Knights starting in 2013 to rejoin his former Dragons coach, Wayne Bennett.[18] He was released from the final year of his Sharks contract for personal reasons.[19] He was named the New Zealand Rugby League's 2012 Player of the Year.[20]
On 26 February 2016, Smith was announced as one of three co-captains for the Knights alongside Trent Hodkinson and Tariq Sims.[24] On 13 May 2016, he announced his retirement from rugby league at the end of the 2016 season.[25]