The James Brown house at 50 North Main Street in Ipswich is believed to have been constructed about 1700 and extended in the 1720s. In 1721, Stephen Perkins, a shopkeeper, and James Brown, a yeoman, bought from Thomas Lovell a house on a large lot that extended along North Main Street (39:61). Perkins and Brown divided the land, and Brown retained the house and land on North Main St.
There had been a dwelling on the larger lot before 1654 when Thomas Lovell acquired the property (1:152), but the oak frame with 1″ chamfers in the southern portion of the present house is of the type constructed in the late First period. The northern section appears to date from the 1720s and was probably added by Brown after he had acquired the house. The house transferred in 1721 remains at the core of the present house.
At some time before the end of the 18th century, the house was extended even further south. This 3/4 acre property had houses and came to be owned by two families at the same time, Thomas Kimball and Ephraim Kendall. In 1773, Timothy Thornton, husband of Eunice Brown, petitioned the town to divide the property. In 1845, Thomas Morley bought the most southerly part of the property (355:298) and rotated the middle portion 90° to form a separate dwelling at 48 North Main.
James Damon took down the remainder of the old dwelling and built the fine house at 46 North Main. (In 1886, Harry K. Dodge took down the home of the widow Margaret Kendall and built the house at 44 North Main St.)
In Tales of Olde Ipswich, Volume III, Harold Bowen wrote that Albert Hills operated a grocery store in this building in the early 20th Century. Albert P. Hills was a Civil War veteran, having enlisted at age 18 and served as a drummer in Company I (Captain Hobbs) of the 23rd Massachusetts Regiment. It was the home of long-time selectman Brianard Wallace in the 1940s.
Sources:
Historic Ipswich 