In his first two seasons, Barry Word and Christian Okoye consistently beat Williams out for playing time. In both seasons, he finished behind Word and Okoye in rushing yards. By 1993, Marcus Allen joined the team and became the starter.
In 1994, Williams moved on to the Los Angeles Raiders. He finally received a chance to be a starter, and responded with two good seasons. He rushed for 983 yards in 1994. He followed up this season with his only 1,000-yard rushing season, rushing for 1,114 yards and 9 touchdowns in 1995.