Sir John Cass's Foundation Primary School will change its name to The Aldgate School for the new school year.
A bust and statue of Sir John Cass will also be removed.
It comes as the City of London Corporation launched a consultation to review landmarks with historic links to racism and slavery.
Sir John Cass's Foundation was set up as an educational charity in 1748 and funds work across London, concentrating on projects for young people.
On the foundation's website Sir John Cass is described as "a merchant and politician, whose wealth posthumously was used to create the Foundation to deliver educational benefits to disadvantaged children".
But he was also a major figure in The Royal African Company, which was heavily involved in the Atlantic slave economy.
A statement from the school's governing board said it was "incompatible" with the school's values to be "celebrating" Sir John Cass.
"Our school is one in which every child is given full opportunity to flourish, irrespective of their background.
"Governors further proposed that the new school name should be one which is geographical rather than biographical, should retain a sense of the long history of the school and its links to the Sir John Cass's Foundation and the Church of England, and should celebrate its unique position as the only state-maintained school in the City of London."
The school said the new name will be adopted as soon as legal approval is granted by the City of London Corporation.
The corporation, which looks after the Square Mile in the capital, said there are a number of statues and road names with links to the slave trade and the public are being asked to consult on renaming them.
The online consultation will run until 24 November.
Believe those who are seeking the truth. Doubt those who find it.