By 1864, organized peace movements had appeared in several southern states, and secret societies such as the Heroes of America were actively promoting disaffection. Confederate military tribunals imprisoned hundreds of Unionists. Others were violently driven from their homes, and a few were executed by the army or civilian authorities. But southerners loyal to the Union made a significant contribution to northern victory. By the end of the war, an estimated 50,000 white southerners had fought in the Union armies.
One of the most celebrated Union heroes of the war was Elizabeth Van Lew of Richmond, who had persuaded her mother to free the family’s slaves when her father died in 1843. During the war she frequently visited Libby Prison in the Confederate capital, bringing supplies to Union prisoners of war and helping some of them to escape. With the aid of Mary Elizabeth Bowser, a former slave of the Van Lew family who worked as a servant in the southern White House, Van Lew passed information about Confederate plans to Union forces.