Harold Bowen wrote that this building was constructed from bricks that were saved from the house owned by undertaker and upholsterer George Haskell where the parking lot is now for the Ipswich Inn, opposite N. Main Street.
This photo from the collection of William J. Barton appears to show a group of Ipswich Female Seminary students posing for Ipswich photographer Edward L. Darling. The house behind them stood at 1 High St., directly across from North Main St. The location is now the parking lot for the Ipswich Inn. The house at 5 High Street served at that time as one of the dormitories for the Ipswich Female Seminary. The bricks from this house are said to have been used to build 19 Brown Square.
Historic Ipswich 