The Lenox Avenue Gang was started in the early 1900s by Horowitz as an independent group of around twenty members. It consisted mostly of pickpockets and burglars, under Jack Zelig's Eastman Gang. Mainly operating around 125th Street in Harlem, then a predominantly Jewish neighborhood, the gang generally committed muggings and robberies, but Zelig occasionally hired them for murder.
Under Horowitz's leadership, the gang produced many of the top criminals of the early century, including Jacob Seidenschner, Louis Rosenberg, and Francesco Cirofici. Prominent New York police detective Val O'Farrell ranked Cirofici as "one of the toughest men in the world", and police suspected Cirofici of at least six homicides. However, they were unable to charge him due to a lack of evidence. An associate of the gang, Cirofici's girlfriend Dutch Sadie, was known to carry a butcher knife in her clothing, and she assisted Cirofici in several muggings.
While successful in its early years, the gang was brought down after they were hired to murder gambler Herman Rosenthal, a suspected police informant who had complained to the press about police extorting too much of his profits. Horowitz, Seidenshner, Rosenberg, and Cirofici drove to the Metropole Hotel in the early morning of July 16, 1912. After Rosenthal left the hotel about 2 a.m., they shot him several times and fled the scene. The gangsters were seen by dozens of witnesses and were quickly arrested. When questioned, they told police that they were hired by New York Police Department Lieutenant Charles Becker, who had ordered Rosenthal's death after he had informed on Becker.
Gyp the Blood, Jacob Seidenschner, Lefty Louis Rosenberg, and Francesco Cirofici were charged with the murder of Rosenthal. They were each convicted and sentenced to death in the electric chair. They were executed at Sing Sing on April 13, 1914. With the loss of these leaders, the Lenox Avenue Gang disbanded entirely within several months. Becker was also convicted on charges of murder and executed in 1915.