The Army said one stand gave way just before 11:00 BST and part of a second stand collapsed a short time later.
London Ambulance Service (LAS) said two patients were taken to hospital for treatment as a priority.
Three others were treated at the scene with one later transferred to hospital.
Spectators were evacuated from the stands by the Army and police so that safety checks could be carried out.
A "partial structural failure" was to blame, the Army said. LAS has not confirmed the severity of the injuries that were sustained.
Marcus Dell, who was at the event, said he heard screams.
"Everyone had stood for the national anthem and there was a bit of a commotion. The first we heard was that one of floorboards had broken and someone had fallen through and hurt themselves.
"There was a lot of chatter in the stands and a few people did leave."
He said a floorboard had cracked behind him and then about 25 minutes later, there was a similar incident in the stand opposite.
When the area was evacuated as a safety precaution there was confusion, Mr Dell said, although the "police did a good job of making sure there wasn't any panic".
Paul Rogers, who was nearby, said: "We heard a loud bang and a scream and a person shouting 'Help! Help!'
"A number of people climbed over the back wall and went below the section of stand that had broken. It lasted about 20 minutes.
"It just seemed like a small section of the stand went," Mr Rogers said.
Another witness said the stands were emptied one by one, adding that the evacuation was "very well organised, not chaos as quoted by some".
* It's a parade which has marked the official birthday of the British sovereign
* It has been held for more than 260 years
* Last year's parade was in the grounds of Windsor Castle, rather than in central London
* It is normally held on the Queen's official birthday which is marked on the second Saturday of June each year
Up to 1,450 soldiers from the Household Division and the King's Troop Royal Horse Artillery were taking part in the parade, alongside about 400 musicians from the Massed Bands.
It is very similar to the Trooping the Colour event reviewed annually by the Queen - she is due to do so on 2 June as part of the Jubilee celebrations. Prince William is to review the troops next Saturday.
Earlier, the troops were being reviewed by Maj Gen Christopher Ghika of the Household Division.
An LAS spokesperson said: "We sent a tactical response unit, an ambulance crew, two cycle responders and an incident response officer.
"We worked alongside St John Ambulance volunteers to treat five people at the scene. We took one patient to a major trauma centre as a priority. St John Ambulance took a second patient to a major trauma centre as a priority. We discharged three patients at the scene. We later arranged transport to hospital for one of these patients."
In a statement, the Army said: "Safety is our number-one priority and we are urgently working with our partners and relevant organisations to understand what happened and make sure it doesn't happen again."