He then joined the Penn State coaching staff from 1982 to 1983 as an assistant under offensive line coach Dick Anderson. At Penn State, he helped the Nittany Lions win the 1982 National Championship with a Sugar Bowl victory over Georgia. When Anderson left to become head coach at Rutgers prior to the 1984 season, Flaherty joined him on the Scarlet Knights' staff, where he remained through the 1991 season.
Flaherty became the tight ends coach of the Washington Redskins under the head coach at the time, Norv Turner, in 2000. However, after just 1 season, Flaherty was relieved of his duties due to Norv Turner being fired midway through the season. In 2004, Flaherty became the offensive line coach of the New York Giants. He was hired to a new staff under new head coach Tom Coughlin, and received two Super Bowl rings during this time. He would remain in this position until 2015 when Tom Coughlin stepped down, thus beginning a coaching staff overhaul. On January 22, 2016, Flaherty was hired by new head coach Chip Kelly to become the offensive line coach of the San Francisco 49ers. On January 14, 2017, Flaherty was hired by new head coach Doug Marrone to become the offensive line coach of the Jacksonville Jaguars. On December 31, 2018, following a disappointing 5-11 campaign, Flaherty was one of three assistant coaches that were fired at the end of the season.
On February 8, 2019, the Miami Dolphins announced they had hired Flaherty as their offensive line coach.[2] On July 29, 2019, the Dolphins head coach, Brian Flores, fired Flaherty just four days into training camp and replaced him with team analyst Dave DeGuglielmo.