The Baal Shem’s problem was with the word chacham, or rather with a particular interpretation of it. He said that when God tells you what to do you should comply without being a chacham.
In one sense this is a very strange argument. How can you not bring brains and wisdom (chochmah) to the task God sets you?
But remember that a chacham (especially if you pronounce it in the Yiddish way, a chochem) is often too clever for his own good. He can’t see things in a simple, straightforward way. He has to theorise and analyse and intellectualise. He has to make everything complicated. If he belongs to a committee he has to formulate motions and propose amendments and set up sub-committees and refer things to the experts. He is so clever that the job is constantly postponed and never gets done.
The Baal Shem had no time for verbal shenanigans. God explained what He wanted, and the people had to get on with it.