He played most of his career with the Pittsburgh Pirates (1954; 1956–63), signing with them in 1951. Skinner spent his last 3+1⁄2 years as a pinch hitter and backup outfielder for the Cincinnati Reds (1963–64) and St. Louis Cardinals (1964–66). During his best season, 1962 with the Pirates, he batted .302 and hit 20 home runs. Over his 12-year career, he batted .277 with 1,198 hits, including 197 doubles, 58 triples and 103 homers. He played for two World Series champions in two tries. In the 1960 World Series with Pittsburgh, he was injured in game 1 and unable to return until game 7. He had 1 hit in 5 at bats plus a walk, a hit-by-pitch and a stolen base. As a pinch hitter for St. Louis during the 1964 World Series, Skinner hit safely in two of three at-bats for a .667 average.
In 1967, Skinner retired from playing and became manager of his hometown team, the San Diego Padres of the Triple-APacific Coast League, the top farm club of the Philadelphia Phillies. He led San Diego to an 85–63 record and the 1967 PCL championship, winning Minor League Manager of the Year honors from The Sporting News. In 1968, he began the year at San Diego but on June 16 he was called to the Phillies to replace Gene Mauch as manager with the Phils in fifth place with a record of 27–27. It was a disastrous move for the Phils; under Skinner, the team plunged to eighth place, with a 48–59 record, and when they performed even worse in 1969, at 44–64, and in fifth place in the new NL East Division, Skinner was replaced by his third-base coach, George Myatt, on August 6.