In the sanctuary nothing shoddy could be tolerated, neither in the procedures of sacrifice and worship, nor in the garb and appearance of those charged with the responsibilities of leadership and office.
Clothing mattered with the priest: it also matters with the rabbi. A scholar must not have even a speck on his garments: it would be an insult to God and the Torah. Shabbat and festival clothing must be even better and more dignified than weekday clothing.
Concessions might be made in extreme climatic conditions, and in very hot weather relatively casual dressing may be acceptable – but even then one must be tidy and neat.
In a British synagogue where I once served, the dress code was very British and a prospective chazan was ruled out because he was wearing brown shoes. In my synagogue in Australia no-one even noticed if people came in sandals, though I once gently remonstrated with a member who came on Shabbat in shorts.
But in Israel no-one seemed bothered when a young man came to synagogue on Kol Nidrei night wearing shorts.
The moral? Take account of the environment, certainly, but don’t say, “Who cares what you wear?” Your garments say something about you. The Torah says that the kohen’s clothing should be “for honour and glory” (Ex. 28:2).
Since we are meant to be “a kingdom of priests” (Ex. 19:6), we should ensure that our clothes proclaim honour to ourselves and the glory of God.