A police officer at the vigil tried to stop a man from draping a black sheet and tying a rope around the monument's neck, while members of the crowd chanted “this is speech” at the officer.
Several speakers recounted their emotional experiences in Charlottesville, and explained why they believe white supremacy is unacceptable.
Nancy Verson, who was watching the speakers from the crowd, said it’s important to communicate freely, although she believes certain views and speech can't be tolerated.
"I think racism is alive and well," Verson said. "And I think our current administration, because of the vitriolic tone, has really emboldened and empowered people that have racist views to be able to speak up and act out or express themselves with hate."
Cathy Meerbergen, who recently moved to the South, said it has been eye-opening to see how racism differs in the South. Universities, she said, can have an element of white privilege.
“I've encountered racism of other kinds in the country, but this sense of entitlement based on history, and this strange liaison to the Civil War still on the outcome of the Civil War, I just find incredible,” Meerbergen said.
In addition to the Chapel Hill vigil, protesters gathered across the state in Durham, Charlotte, Raleigh and other cities Sunday night.
Counter-protesters clashed with white nationalists, neo-Nazis and members of the Ku Klux Klan at Saturday's "Unite the Right" rally in Charlottesville, Va. The right-wing groups gathered on the University of Virginia's campus to protest the removal of a statue of Confederate general Robert E. Lee.
One person was killed and 19 others were injured when a car drove into counter-protesters walking down a street in downtown Charlottesville. Police arrested the suspected driver, 20-year-old James Alex Field Jr. that afternoon.
Two state patrol troopers died when their helicopter crashed near Charlottesville after monitoring Saturday's events.
Virginia Gov. Terry McAuliffe declared a state of emergency Saturday.
"I am disgusted by the hatred, bigotry and violence these protesters have brought to our state over the past 24 hours," McAuliffe said in a statement. "The actions I have taken are intended to assist local government and restore public safety."
President Donald Trump called for an end to all violence on Saturday from his golf resort in Bedminster, N.J.
"We condemn in the strongest possible terms this egregious display of hatred, bigotry and violence, on many sides," Trump said. "On many sides. It's been going on for a long time in our country. Not Donald Trump, not Barack Obama. This has been going on for a long, long time."
Many criticized the president for not condemning white supremacy and racism.
N.C. Rep. Graig Meyer, D-Orange, questioned the term "many sides."
"I know which side I'm on. Justice. Love. Peace," Meyer tweeted. "Any moral equivalency about sides is false."