The author says Beira's Place will provide free support and advocacy for women in Edinburgh who have experienced abuse at any time in their lives.
She said the service would meet an "unmet need" in response to demands from female survivors of abuse.
Ms Rowling said it was important that survivors had the option of women-centred and women-delivered care.
Rape Crisis Scotland said support should also be available to trans and non-binary people.
The Harry Potter creator has held a longstanding interest in women's and children's issues and has been a vocal critic of the Scottish government's reform of the Gender Recognition Act.
Other critics of the reform are among those on the board of directors for Beira's Place.
As well as Ms Rowling, they include former prison governor Rhona Hotchkiss, previous Scottish Labour leader Johann Lamont, GP Margaret McCartney, and director of For Women Scotland, Susan Smith.
The new venture is being funded by the author, who lives in Edinburgh. It is not a charity and will not rely on donations.
Ms Rowling said: "I founded Beira's Place to provide what I believe is currently an unmet need for women in the Lothians area.
"As a survivor of sexual assault myself, I know how important it is that survivors have the option of women-centred and women-delivered care at such a vulnerable time.
"Beira's Place will offer an increase in capacity for services in the area and will, I hope, enable more women to process and recover from their trauma."
Named after Beira, the Scottish goddess of winter, it will be run by a paid staff of experienced support workers and headed by chief executive Isabelle Kerr and deputy chief executive Susan Domminney.
They previously ran the Glasgow and Clyde Rape Crisis.
Ms Kerr said: "Violence against women and girls is an issue that crosses all cultures, classes, and religions.
"These are gendered crimes that are overwhelmingly perpetrated by men and disproportionately experienced by women.
"Beira's Place recognises that effective sexual violence services must be independent, needs-led, and provide responsive, women-centred services so that they are free from the pressure of current political agendas.
"We are committed to ensuring that our service is free, confidential, and accessible to women survivors who may need it."
The establishment of Beira's Place comes two months after Ms Rowling supported a rally protesting against the Gender Recognition Reform (Scotland) Bill.
She wore a T-shirt calling First Minister Nicola Sturgeon a "destroyer of women's rights".
The proposed new law will make it easier for people in Scotland to change their legally recognised gender, and is due its final vote at Holyrood next week.
It will remove the requirement for a medical diagnosis of gender dysphoria as a condition of acquiring a gender recognition certificate.
Critics have raised concerns about the impact it could have on single sex spaces such as women's refuges and prisons.
The Scottish government has denied this, saying it will streamline a process which is currently invasive and distressing for an already marginalised group.
Rape Crisis Scotland - which has 17 centres across Scotland - said it welcomed any new service to support survivors of sexual violence.
It added: "It is crucial that the life-saving support offered by rape crisis centres is available to trans and non-binary people.
"All rape crisis services in Scotland offer support to trans women and have done so for 15 years. There has not been a single incident of anyone abusing this.
"We continue to see the paths to equality for women and trans people are being deeply interconnected and dependent on shared efforts to dismantle systems of discrimination."
Writer Suzanne Moore, who interviewed JK Rowling on the opening of Beira's Place, told BBC Radio 4 Woman's Hour it was a major undertaking for the author.
"A project like this is years in the making," she said. "She has gathered a team of experts, people who have worked in rape crisis centres and what she really wants to ensure is this is not a charity that is subject to the whims of power and that its funding is never under threat.
"It's really simple and in some ways it has not got a lot to do with the whole trans issue and the complex politics of Scotland. JK is doing something to improve the lives of some of the most vulnerable people."