One of the questions asked by the Netziv, Rabbi Naftali Zvi Yehudah Berlin, is why the whole people had to be punished when it is impossible to imagine that every single one of them supported the sin.
In a classroom situation, l’havdil, it often happens that one pupil misbehaves and the whole class is punished, even the quiet ones who played no part in the mischief. There are always quite legitimate mutters, “It isn’t fair!”
In the wilderness the Israelites must also have felt it wasn’t fair, and Moses must have had some sympathy with the silent group who may have been perfectly innocent.
Perhaps we can argue that the ones who were not themselves guilty should have had the courage to stand up and speak up and either prevent the sin or at least protest against it.
Perhaps the whole episode was a means of teaching the people that if a rasha gets into power the result can be terrible damage, so the people have to ensure that r’sha’im are kept out of office.