"There are many diverse voices within businesses... one of the key challenges you have is... to bring these voices into some kind of synchronisation," he told Business Unity SA's (Busa's) annual general meeting in Johannesburg.
Black business groups including the Black Management Forum (BMF), resurrected the Black Business Council in September, suspended their participation in Busa.
The BMF said Busa's structure is fundamentally flawed, with the voice of black business "permanently outnumbered and suppressed".
On Monday, Gordhan said business is a key player in the economy and in deciding South Africa's growth path, and called for "better synergies between business and government".
He called on business to reconsider remuneration in the light of the huge inequalities in society.
"There is not enough coming from business that indicates sufficient recognition on your side that huge bonuses to managers is going to have a damaging effect."