A small forward, Vardy started his career at Adelaide where he was an AFL Rising Star nominee in 1997. A key member on the forward flank for the team, he broke his collarbone in the semi-final against Geelong, but managed to kick an extremely important goal in the process while riding a rough collision. This meant he was unable to participate in the last two finals of the Crows' first premiership. The following season he enjoyed another stellar year, accumulating 44 goals and being involved a premiership as the Crows won another premiership, kicking two goals from the wing in the grand final.
Vardy established himself as one of the league's most dangerous running forwards with regular consistent performances, however he fell out of favour with then-Adelaide coach Gary Ayres and at the end of 2001 was traded to Melbourne for draft pick number 56.