The Court of Appeal refused Mr Assange's appeal for the detention order issued in 2010 to be revoked.
Mr Assange, who denies the allegations, has sought refuge in the Ecuadorean embassy in London to avoid extradition.
If he is sent to Sweden, he says he fears charges in the US over the leaking of secret government documents.
Two women in Sweden accuse him of sexual assault.
'No reason' Thursday's court decision ruled on an appeal against a similar decision by a lower court.
"There is no reason to set aside the detention solely because Julian Assange is in an embassy and the detention order cannot be enforced at present for that reason," the Svea Court of Appeal in Stockholm said in a statement.
"The reasons for detention still outweigh the reasons to the contrary since Julian Assange is suspected of crimes of a relatively serious nature and there is a great risk that he will evade legal proceedings or punishment if the detention order is set aside."
The Ecuadorean government granted asylum to the Wikileaks founder in 2012 after the UK Supreme Court refused to reopen his appeal against extradition.
Mr Assange has not been formally indicted in Sweden, but prosecutors want to question him over allegations of sexual misconduct and rape involving two women he met during a visit to the Scandinavian country in 2010.
Mr Assange denies the allegations and has said they are part of a smear campaign against him.
He fears that, if he were extradited to Sweden, he would be extradited again to the United States, where he could face charges over the release of thousands of secret documents by Wikileaks.
Chelsea Manning, an American soldier formerly known as Bradley Manning, was sentenced to 35 years in prison in the US for passing documents to Wikileaks.
The leaks caused intense embarrassment of the US and other governments.