Baldwin was the 12th and posthumously born child of George Pearce Baldwin and Sarah Chalkley Stanley. He started work at the Wilden Iron and Tin Plate Company at Wilden (near Stourport, Worcestershire), which, in 1840, had been taken over by his uncle Enoch Baldwin, who ran the business with his nephews, Pearce and William, trading as E.P.& W. Baldwin of Wilden. In 1879, Alfred and his brothers dissolved the partnership. Alfred then moved into Wilden House and took over the Wilden Works, changing the name of the business to Baldwins Ltd.
As well as being an ironmaster, Alfred also became a director and chairman of the Great Western Railway.
He married Louisa Macdonald, daughter of Reverend George Browne Macdonald, a Wesleyan minister, and Hannah Jones, on 9 August 1866.
Through his wife, Baldwin was connected to three famous English artists and an even more famous writer. The eldest sister, Alice, married the art teacher John Kipling in 1865; they became engaged at Rudyard Lake, near Leek in Staffordshire—hence, the name of their son, the writer Rudyard. In a joint wedding with her, her sister Agnes married the painter Sir Edward John Poynter.
Another sister, Georgiana, married the painter Edward Coley Burne-Jones in 1860.
Alfred Baldwin paid for the construction of All Saints Church, Wilden, which was consecrated in 1880. The original windows were replaced with 14 designs by Burne-Jones between 1902 and 1914. They are mostly dedicated to members of the MacDonald, Baldwin and Burne-Jones families; one shows Stanley Baldwin setting out on life's journey accompanied by a guardian angel.
