Harry Smith first played senior-level hockey with the Ottawa Aberdeens of the CAHL-Intermediate league in 1901. He played for Arnprior of the Upper Ottawa Valley Hockey League before joining Smiths Falls of the Southern Ontario Hockey Association. He played two seasons with Smiths Falls before returning home to play in Ottawa with the Ottawa Senators, already the Stanley Cup champion. He played two seasons with the Silver Seven with his brother Alf who was playing-coach. In the 1905–06 season, Alf, Harry and brother Tommy all played for the Silver Seven, occasionally playing together on one line.
Smith was known as a rough player. In 1907 Smith and teammates Alf Smith and Charles Spittal were charged with assault after beating Montreal Wanderers players Hod Stuart, Ernie "Moose" Johnson and Cecil Blachford with their sticks. Harry Smith was acquitted while Spittal and Alf Smith were each fined $20.[3] When Smith played in an exhibition game with the Winnipeg Maple Leafs against the Winnipeg Hockey Club on December 19, 1907, the Winnipeg Hockey Club players refused to carry on the contest after it had degenerated into a number of violent displays, the last of which involved blows between Harry Smith and Winnipeg Hockey Club defenseman Percy Browne.[4] He was subsequently expelled from the MHA along with Maple Leafs teammate Joe Hall.[5]
Smith was also known to have one of the better shots in the game during his era, which helped him score many goals.[6]
He became a professional player in 1907–08 with the Pittsburgh Bankers of the Western Pennsylvania Hockey League (WPHL). On February 15, 1908, he was involved in a riot with spectators in a game against the Pittsburgh Athletic Club at the Duquesne Gardens where he went into the stands swinging his stick after he had been hit on the ice by a chair thrown by a spectator.[7] In 1908–09 he was a 'for hire' player playing for the Bankers,[8] Toronto Pros (OPHL), Haileybury Comets (TPHL) and Montreal Wanderers (ECHA), where he was member of the Wanderers 1909 Stanley Cup challenge championship squad, although the team did not hold the Cup at the end of the season.
In December 1909, prior to the 1909–10 season, Smith was badly scalded on one if his legs by the bursting of a valve in a steam engine while working for his father Henry in Fort William. He was laid up in hospital for several days, and it was first thought that he would not be able to play any hockey during the season, but he did play later on for both the Cobalt Silver Kings and the Haileybury Comets in the National Hockey Association.[9]