Evewright's Walkway of Memories at the Port of Tilbury, Essex, features more than 130 people on the walkway where passengers disembarked in 1948.
Several of the 432 panes of glass where the artwork is featured have been broken.
The artist said it had "empowered" him to continue his work.
Essex-based artist Evewright's art and sound installation was installed on 72 windows on the 55m (180ft) walkway and includes 130 images and 35 audio recordings.
The panes of glass are collaged with photographs, documents, boat passenger tickets and memorabilia and more than 160 minutes of audio stories can be heard in ten-minute segments across 22 listening windows.
The artist said vandals had "indiscriminately targeted the bridge".
"This ain't going to stop me, actually it empowers me even more to keep working as an artist to try and represent the Black British experience," he said.
"We should all try and fight against it, stand up against it and speak out when we're attacked.
"To me, it was a personal attack because I am from that generation, I'm first-born... it's almost like it's a hate crime."
He said the port had acted very swiftly to protect the bridge, putting extra cameras in place.
The exhibition remains open to the public but with the damage still visible.
"For me as an artist, I felt it was important to keep those strikes on the bridge... it's to let everyone know that this hate is still there in our society," he said.
The BBC has approached Essex Police and the Port of Tilbury for comment.