Penny played for his local club Withernsea[5] before joining Hull City's youth system aged 11.[6] Despite being offered an extension to his scholarship in April 2015,[7] he signed for Southern League Premier Division club Bedworth United in September.[8] Having made five appearances,[9] he signed for Northern Premier League Premier Division club Stourbridge later that month.[10] In November 2015, Penny signed for Midland League Division One club Hinckley,[11] after making two appearances for Stourbridge.[12] He made 17 appearances and scored two goals for Hinckley in the 2015–16 season,[13] the team finishing fifth in the Midland League Division One.[14]
On 19 July 2017, Penny signed for League One club Peterborough United on a three-year contract after a successful trial.[20] The undisclosed fee set a new record for a transfer fee received by Nuneaton.[21] On 5 August 2017, he made his Football League debut as an 89th-minute substitute in Peterborough's 2–1 home win over Plymouth Argyle, on the opening day of 2017–18.[22] He made his first start for the club three days later, playing the first 68 minutes of a 3–1 home defeat to Barnet in the EFL Cup.[22]
Penny signed for National League North club Kidderminster Harriers in February 2020 on a two-year contract.[25] On 5 October 2021, Penny scored his first Kidderminster Harriers goal in a 3–0 home win against Ware FC in the FA Cup 3rd Qualifying round. On 5 February 2022, he scored the only Kidderminster goal against West Ham in the FA Cup fourth round[26] and went on to extend his deal to the summer of 2023 in April 2022.[27] Penny was subsequently named in the 2021-22 National League North Team of the Season.[28] Following promotion back to the National league during the 2022/23 season, Penny signed a new one-year deal with Kidderminster running through till 2023/24 season[29]