Swayne Hall was built as Baptist men's college in 1857. After the American Civil War it became a part of Talladega College. It is historically Alabama’s oldest college and is a private black liberal arts college. In the post-Civil War era it was originally constructed with slave labor. The building was named Union Army General Wager Swayne.