She will instead watch the showpiece horse racing event - which is part of her Platinum Jubilee celebrations - on TV at Windsor Castle, the palace said.
Until the pandemic, the Queen had only missed Derby Day twice in her reign.
The 96-year-old earlier pulled out of Friday's Jubilee thanksgiving service after experiencing "discomfort" during Thursday's events.
The Queen has previously spoken of mobility problems and the palace confirmed ahead of the Jubilee the monarch would consider the events she felt able to attend during the four-day celebration.
The Queen's daughter, the Princess Royal, is expected to officially represent the monarch at Saturday's Derby.
Earlier, the Queen's love of horse racing was referenced at a packed thanksgiving service at St Paul's Cathedral.
Archbishop of York Stephen Cottrell praised her for "staying the course" and said she was "still in the saddle", even though she was not able to attend.
Derby Day will see the Queen's unprecedented 70 years on the throne celebrated by 40 retired and current jockeys who have ridden for the monarch, and will form a guard of honour.
The jockeys will dress in her distinctive purple, gold and red racing silks, and racecourse owners the Jockey Club say the Queen's Stand will be permanently renamed the Queen Elizabeth II Stand.
Later on Saturday, the Prince of Wales and the Duke of Cambridge will pay tribute to the Queen during a special Jubilee concert outside Buckingham Palace.
The two future kings will address a live audience of 22,000 towards the end of the BBC's two-and-half-hour open-air show.
Diana Ross, Queen + Adam Lambert, Alicia Keys, Hans Zimmer, George Ezra, and Eurovision 2022 runner-up Sam Ryder are among the stars performing at the Platinum Party at the Palace.
Sir David Attenborough, David Beckham and Stephen Fry will also feature, and there will be a specially recorded performance from Sir Elton John.
Speaking ahead of the concert, disco legend Nile Rodgers told the BBC he was expecting an "incredibly festive" atmosphere.
"I've been playing in the UK since the early 1970s and I don't think I've ever seen the country this happy and zhuzhed up and decorated. I expect the concert to be completely out of control," he said.
And Italian operatic tenor Andrea Bocelli said: "The psychological condition when you do one of these performances... is like when a football player has to kick the last penalty - I just hope that after many years of my career, I can manage the pressure!"
Singer-songwriter Mabel told the BBC it was an "honour" to perform, and Ella Eyre - who was at Buckingham Palace for Prince William and Catherine's wedding - said it was "surreal to be on the other side", adding that the production was "phenomenal".
Saturday is also the first birthday of Prince Harry and Meghan's daughter Lilibet, who travelled with her parents and older brother Archie for her first visit to the UK.
By John Hunt, BBC Radio 5 Live racing commentator, at Epsom
The Queen loves nothing more than a day out at the Derby, and of course a day out or two at Royal Ascot which is only two weeks away.
There are hopeful whispers around the racing community that Her Majesty, even though she can't make the races tomorrow, might be able to attend at least one day at Royal Ascot.
The Queen is of course a huge supporter of horse racing - she's absolutely been involved from the word go.
Four days after her coronation in 1953, she came to Epsom to see her horse very nearly win the Derby.
She's been locked in from that day onwards - and has been to most of the Derbys ever since.
Sadly not to be this year - but I'm sure she'll be watching on, deeply interested.
The Queen's hopes of having a winner on this year's Derby Day ended earlier as her only runner was pulled out of a race.
Just Fine had been entered for the penultimate race but was removed at Thursday's 48-hour declaration stage. Another of her horses Reach For The Moon, who was among the early favourites for the Derby itself, was withdrawn in May along with two others.
Thoroughbreds owned by the Queen have won four out of the five flat racing classics, with only the Derby eluding her.
Meanwhile, a Jubilee attendee said she was told the Queen was "fine" by the Duchess of Cambridge during an event at London's Guildhall on Friday.
Gill Smallwood told PA news agency she had asked after the Queen during a conversation with the duchess, who replied that the monarch had found the first day of Jubilee celebrations "very tiring", but had a "lovely time".