Zuma told reporters that Cooperative Governance Minister Sicelo Shiceka and Public Works Minister Gwen Mahlangu Nkabinde will leave cabinet.
Shiceka is suspected of spending R600 000 on unsanctioned luxury travel, including overseas visits to a girlfriend imprisoned in Switzerland for drug smuggling.
Nkabinde and police commissioner Bheki Cele, who is not in the cabinet, were named in a government-sanctioned report for suspected illegal conduct in property rental deals.
Mahlangu-Nkabinde would be replaced by Deputy Minister of Rural Development and Land Reform Thulas Nxesi. Current Public Works Minister Richard Baloyi would take up Shiceka's post, Zuma said.
Inquiry
"In August, I informed the national commissioner, General Bheki Cele, of my intention to institute a board of inquiry to look into the allegations of misconduct, in relation to the procurement of office accommodation for the South African Police Service, as per the findings and recommendations of the public protector," Zuma told reporters in Pretoria.
The board of inquiry will be led by retired Justice Yvonne Mokgoro, alongside advocates Terry Motau and Anthea Platt.
"I have also decided to suspend the national commissioner from duty with immediate effect, pending the outcome of the inquiry, in terms of section 8 (3) (a) read with Section 9 (1) of the South African Police Service Act.
"He will, during the period of suspension, be entitled to his full salary, allowances, privileges and benefits, in terms of Section 8(3)(b) of the act."
Major-General Nhlanhla Mkhwanazi will act in Cele's position, Zuma said.
Cabinet reshuffle
As part of Zuma's cabinet reshuffle, current Deputy Minister for Performance Monitoring and Evaluation Dina Pule has been appointed minister of communications.There willl no longer be a deputy minister of public works.
Minister of Communications Roy Padayachie has been moved to the public service and administration portfolio.
Obed Bapela, the Deputy Minister of Communications, has been appointed deputy minister in the presidency for performance monitoring and evaluation.
Hendrietta Bogopane-Zulu, the Deputy Minister of Public Works, will be the deputy minister for women, children and persons with disability.
Lechesa Tsenoli, chairperson of the portfolio committee on cooperative governance and traditional affairs, has been appointed as deputy minister for rural development and land reform.
ANC member of parliament Thembisa Stella Ndabeni will become the deputy minister of communications.