Recruited from St Patrick's College, Ballarat, he played 76 games for St Kilda from 1902–1906, and kicked 122 goals, including four of which came in his first VFL game; at the time a club record. He represented Victoria in cricket from 1903-1905 and at football before an injury ended his career.
Baker won the Leading Goalkicker Medal in 1902, a season in which St Kilda kicked only 64 goals for the year (of which Baker kicked 46.9%) and finished last without winning a match, a VFL/AFL record seven matches behind second-last placed Geelong. Baker's proportion of his team's goals is also an VFL/AFL record.