military source and a witness said on Friday.
The Islamist group has been driven out of much of the huge swathe of territory they controlled at the start of the year, thanks to a concerted push by troops from Nigeria and neighbours Chad, Niger and Cameroon.
The town of Gwoza, in mountainous terrain, was one of the last places to fall, on March 27, and there remain pockets of Boko Haram activity in the area, security sources say.
"Just as troops were trying to evacuate some civilians from the hills so as to safeguard them from a planned air strike ... some Boko Haram attacked them and slit the throats of 12 people," a military source said of Wednesday's attack.A witness, Jonas Musa, told Reuters his parents were both among the victims. He said soldiers had moved one wave of people from the hills around Gwoza, but before they could go back for the second, the attackers struck.
Failure to crush Boko Haram or protect civilians was one reason President Goodluck Jonathan lost an election on March 28 to Muhammadu Buhari. Boko Haram, fighting to establish an Islamic state, has killed thousands and kidnapped hundreds during its six-year-old insurgency in Africa's biggest economy and top oil producer.
Buhari has pledged to spare no effort in crushing the militants after he is sworn in on May 29. He said on Tuesday he would do everything he can to rescue more than 200 girls abducted by the group a year ago from a school in the village of Chibok, but that he could not promise to find them.