The course of Khlong Bangkok Yai was originally a meandering part of the Chao Phraya River, which used to be longer than in the present day. Those who travel by boat must cruise along the river, which took more than one day. In the reign of King Chairachathirat (1534–46) of the Ayutthaya Kingdom, he ordered the construction of a canal bypassing a loop of the Chao Phraya River, known as Khlong Lat Bangkok (คลองลัดบางกอก; 'Bangkok Short-Cut Canal'), thus reducing travel times. The Chao Phraya then changed course along the new canal, and the old course became what is known today as Khlong Bangkok Yai (lower section) and Khlong Bangkok Noi (upper section).
In the reign of King Taksin (1767–82) of the Thonburi Kingdom, he founded the new capital, Thonburi, on the west side of the Chao Phraya River. The banks of Khlong Bangkok Yai became the residence of many people, including much of the nobility, giving rise to the names Khlong Bang Luang or Khlong Bang Kha Luang (คลองบางหลวง, คลองบางข้าหลวง; 'Canal of Nobles'). Khlong Bangkok Yai was also the first canal in Thailand to have lampposts installed.