The former vice-president’s dismissal earlier this month led the ruling Zanu-PF party and the army to intervene and force Mr Mugabe to quit.
Mr Mnangagwa, who had fled the country, returned from exile on Wednesday.
The opposition is urging Mr Mnangagwa, who has been part of the ruling elite, to end the “culture of corruption”.
Although Mr Mnangagwa has unseated Zimbabwe’s long-time ruler, he is still associated by many with some of the worst atrocities committed under the ruling Zanu-PF party since the country gained independence in 1980.
He was the country’s spymaster during the 1980s civil conflict, in which thousands of civilians were killed. But he has denied any role in the massacres, blaming the army.
After being sacked as part of a power struggle over who would succeed Mr Mugabe as president, Mr Mnangagwa fled to South Africa two weeks ago – only to return home on Wednesday to a hero’s welcome.
Tens of thousands of people packed the National Sports Stadium at Harare to witness the inauguration. Pop singer Jah Prayzer provided the entertainment and, as people in the crowd danced, the atmosphere was closer to that of a concert.
Dignitaries, including leaders from various African countries filed in to cheers.
Opposition leaders Morgan Tsvangirai and Joice Mujuru – who have both also had their sights on the presidency at various times – were there.
Mr Mnangagwa was led in the oath of office by Chief Justice Luke Malaba, saying he would “be faithful to Zimbabwe”, “protect and promote the rights and people of Zimbabwe” and discharge his duties to the best of his abilities.
Mr Mnangagwa was accompanied by his wife Auxilia and gave her a kiss after the green presidential sash was placed around his neck.
Source: BBC World Service