According to Anne Arundel County Land Records, in 1863 William H. Butler, "a free person of color" purchased 148 Duke of Gloucester Street, a recently built, stylish example of the urban Italianate townhouse, one of the few in the city. A builder and landowner, Butler was one of the wealthiest free blacks in the city in 1860. He served on the Annapolis City Council from 1873 to 1875, becoming the first African-American in Maryland to be elected to public office. He continued to purchase and develop property into the 1880s, particularly on Market Street. Between 1885 and 1890, he built five frame row houses as rental properties, providing a legacy for his six children. Two lots, now 121-123 Market, he sold to the Maryland Colored Baptist Congregation for a church, part of which is incorporated in today's existing duplex. William H. Butler, Sr. died a wealthy man in 1892. His wife Sarah remained in the house, and it stayed in the Butler family until 1922. Their son William H. Butler, Jr., also served on the Annapolis City Council from 1893 to 1897.