25 people and wounded dozens more during holiday prayers.
Sanaa, seized by the Iran-backed Huthi insurgents a year ago, has been shaken by a string of bombings in recent months by IS, a radical Sunni organisation which considers Shiites to be heretics.
Thursday's blast ripped through the Balili mosque, located near a police academy, where the rebels and their supporters go to pray, witnesses said.
It came as Muslims marked Eid al-Adha, the Feast of the Sacrifice, the most important holiday of the Islamic calendar.
Witnesses said that after a first blast inside the mosque, a suicide bomber detonated an explosives belt at the entrance as worshippers rushed out.
"We found a shoe bomb and explosives hidden in underwear and abandoned in the toilet," he said, adding that two devices had failed to detonate while a third had exploded inside the shrine, causing panic.
"As the crowd rushed to leave the prayer room, a suicide bomber tried to force his way into the mosque," said Khaled. "He was stopped at the entrance by a security officer and blew himself up."
Blood stains and debris were seen on the floor of the mosque after the blast, while Huthi rebels inspected the damage.
In an online statement, IS said one of its members wearing a suicide belt had struck "Huthi infidels".
IS, which controls swathes of territory in Iraq and Syria, has repeatedly targeted Shiites in Yemen's capital and elsewhere.
Bomb attacks targeting several Shiite mosques in Sanaa on March 21 killed 142 people, with IS also claiming attacks on mosques in Kuwait and Saudi Arabia.
The Huthis seized the capital of Sunni-majority Yemen in September 2014 and expanded their grip to other parts of the country.
After six months in exile in neighbouring Saudi Arabia, President Abedrabbo Mansour Hadi returned to Aden Tuesday with a vow to liberate the country from the Huthis.In a speech to mark Eid, Hadi said the Huthis had "suffered defeat after defeat" and that "Yemen will soon be freed".
The Arab coalition launched air strikes against the rebels on March 26, and started a ground operation in July.
Hadi loyalists began an all-out offensive against the Huthis in oil-rich Marib province east of Sanaa earlier this month, aiming to retake its capital.
The United Nations says around 5,000 people have been killed in Yemen and 25,000 wounded, many of them civilians, since late March.
Yemen descended into chaos after the 2012 ouster of longtime strongman Ali Abdullah Saleh, and security has broken down since Huthi militiamen swept into the capital unopposed.
IS and the Yemen-based branch of its jihadist rival Al-Qaeda have exploited the turmoil to boost their activities in the impoverished country on the tip of the Arabian Peninsula.
Al-Qaeda has long been the dominant jihadist force in Yemen, located next to oil-flush Saudi Arabia and key shipping lanes, but experts say IS is seeking to supplant its extremist rival.
Al-Qaeda in the Arabian Peninsula (AQAP) controls parts of the vast southeastern province of Hadramawt, including the provincial capital Mukalla, which it is seized in April.
It has distanced itself from IS's tactics, saying that it avoids targeting mosques to protect "innocent Muslims".
The United States has waged a longstanding drone war against AQAP which it regards as the jihadist network's most dangerous branch.