The house began as a small one-over-one structure built one of the lot's four 17th-century owners. Corp. John Andrews' White Horse Inn makes a good story, and that's how the house got its name.
John Sparks was granted a license to open an inn and sell beer "at a penny a quart." The tavern became a popular stopping place on the Bay Road, where men of business would meet for food and drink. The Court met at this location, where it heard and dismissed charges of witchcraft.