This house was the first to occupy this lot. Thomas Franklin Waters wrote:
“Adjoining the Ezekiel Rogers property on East St. was an acre of meadow land, called the “Clay Pitts Meadow,” which John Baker owned and sold to Thomas Harris in 1665 (Ips. Deeds 3: 23). Harris owned land on the other side of the street and had his dwelling there. This meadow descended from father to son. It was owned by John Harris, then by John & William, sons of John 1, who sold his part of it to John Jr., Dec. 22, 1742 (82: 252). John Harris’s heirs owned the lot in 1832. His widow, Mary, sold the lot, or part of it, to Joseph Wait. Wait sold a section of the lot, measuring eight rods ten links, on East St., to Elizabeth Caldwell, widow of Thomas Caldwell, May 6, 1829 (504: 291). She removed a dwelling from another site to this spot. The property was still owned owned by her heirs at the end of the 19th century.”
The 1856 Ipswich map shows the owner of the house as “Mrs. Caldwell”. The rear two-story wing is believed to be the older house, joined together when the house was moved. Structural evidence suggests construction dates of about 1740 to 1775 for the two sections.

