Campaign group Fair Play for Women argued it unlawfully allowed "self-identification" as male or female.
The guidance said people can use the sex listed on their passport - which can be changed without a legal process.
The Office for National Statistics said it was asking the same question on sex it had done since 1801.
After a court hearing on Tuesday, Mr Justice Swift ruled that the guidance should be changed to say individuals should only respond with the answer on their birth certificate or gender recognition certificate - a document which allows trans people to change their legal gender.
A full judicial review will take place next week before the census day, but the judge made the interim order saying he was satisfied the campaign group was "more likely than not to succeed" on its case regarding the legal definition of "sex".
Taking place in England, Wales and Northern Ireland on 21 March, the census aims to provide a snapshot of the population of the country which can then be used to make decisions about services - and which this year will be used to understand the impact of the pandemic.
Scotland's census has been postponed for a year because of the coronavirus crisis.
For the first time, it will include a question about gender identity as well as the one about a person's legally registered sex. The decision had been welcomed by some trans people as a "step in the right direction".
The legal action comes after ministers abandoned plans last year to reform the Gender Recognition Act to allow people to change their birth certificates without a medical diagnosis of gender dysphoria - a process known as "self-identification".
Fair Play for Women, which seeks to defend "the sex-based rights of women and girls", launched a crowdfunded legal action arguing that the census guidance on the question, "What is your sex?" was unlawful. The group called it "sex self-identification through the back door".
The guidance, published in the online census only, said: "If you are considering how to answer, use the sex recorded on one of your legal documents such as a birth certificate, gender recognition certificate, or passport.
"If you are aged 16 years or over, there is a later voluntary question on gender identity. This asks if the gender you identify with is different from your sex registered at birth.
"If it is different, you can then record your gender identity."
They argued this could potentially have the effect of "distorting" the data gathered in the census.
The Office for National Statistics argued that the question on sex could refer to a person's sex as recorded in any document provided by the state.
Speaking before the hearing, Dr Nicola Williams, director of Fair Play For Women, said: "If we don't have good data on sex we can't monitor inequalities due to sex, and if we can't measure it, we can't make good policies to remedy it."
The Office for National Statistics said most people did not need help to answer the question on sex but it would update the guidance to advise people to use the sex recorded on their birth certificate or gender recognition certificate.
A spokesperson for the ONS said: "We are continuing to ask a binary choice, female or male, sex question on the census. This approach is unchanged since 1801."