In the early 1700s, Elkridge, Maryland was a bustling and prosperous port located south of Baltimore on the Patapsco River. Caleb Dorsey owned several iron forges on the creeks that fed into the river. His father gave him the land that Belmont sits on as a wedding present and Caleb built a Manor House, an unpretentious example of brick and stucco Georgian architecture. The house sat on a knoll at the end of a long drive overlooking the rolling meadows that surrounded the property.

When Caleb died, the property was passed down through his son to his daughter, Priscilla who named the home Belmont in 1815. Priscilla was married to Alexander Hanson, who died at the early age of 33. Beset by financial problems, Priscilla sold off parcels of land and eventually the property was sold at auction for $25. Fortunately, the auction was contested and Belmont reverted back to the Hanson family who owned it until 1917.

In 1917, the property passed to Mary Bowdoin Bruce, a descendant of the original builder Caleb Dorsey. Mary and her husband, Howard, restored the estate to its original beauty. Although Belmont remained as a working farm, Howard Bruce's interest was in raising and breeding thoroughbred horses. He bought Billy Barton, who eventually won numerous races, including the Grand National Handicaps and the prestigious Maryland Hunt Cup.

In 1961, after Howard Bruce's death, the property passed to his cousin, David Bruce, the former ambassador to Britain, who gave Belmont to the Smithsonian Institution. In 1983, The American Chemical Society purchased the property to use for its own meetings and to serve the local community. During this period, the Joint Chiefs of Staff, led by Colin Powell, used Belmont for planning part of the Desert Storm War. Most recently, the committee that put together the Baltimore/Washington Bid for the 2012 Olympics met initially at Belmont to announce their alliance. Over the past 20 years, Belmont has been the host to Nobel Laureates, Fortune 500 executives and influential political leaders- who have gathered in the past to share their knowledge, views and experience.

In 2004, Howard Community College purchased Belmont to expand its hospitality program and provide students with practical experience in working in a hospitality environment alongside professionals in the field.