He had announced on Tuesday that the annual event would be banned but hours later he was overruled by Interior Minister Jeje Odong.
Uganda's Ethics Minister Simon Lokodo has said the devil had beaten him after his failed attempt to ban this year's Nyege Nyege music festival, which he alleged promoted open sex, gay relations and drug abuse.
"I tried my best to block it, but the devil has a strong hand. I had to backtrack.”
He had announced on Tuesday that the annual event would be banned but hours later he was overruled by Interior Minister Jeje Odong.
The government later gave the event, which started on Thursday, the go-ahead.
The festival has been taking place annually since 2015 on the banks of the River Nile in eastern Uganda, and is popular with young people from urban areas.
Artists from the US, Europe and other parts of Africa have been lined up to perform at the four-day festival, whose sponsors include the British Council, South African telecom giant MTN and Coca-Cola.
Organisers expect to draw 8,000 people, up from 6,000 last year.
Mr Lokodo, a former priest who is known to promote conservative social values, said on Tuesday that the event was "close to devil worshipping and not acceptable".
"This thing [Nyege Nyege] is not the best at all. I have dug into it and established that it’s not innocent. It’s an instrument being used by our brothers and sisters of Western world to introduce to us a culture, a behaviour and attitude that is foreign and not in our laws, culture and religion," he told the Daily Monitor.
Event organisers have been sharing the festival highlights on Instagram.