Yes thats exactly what Im thinking too. Other villages are bound to know of Konoha's defeat at the hands of a single Akatsuki member - the Akatsuki leader. So they'll have an idea of his power. Then they'll learn that Naruto was the man who singlehandedly defeated that Akatsuki leader. So they'll have an estimate of his power. If Naruto leaves the village, they'll know that there's no one strong enough in the already severely weakened village to survive another onslaught and protect Konoha through or against it....and even if thats not the case, it'll still leave Konoha an open target for attacks from all other countries or atleast one. The way the current events are panning out, Dagoro's prediction seems to be the best possible outcome, but If Danzou and the other's have any sense, they'll do all they can to keep Naruto in the village.

On the other hand, Akatsuki's leader has been defeated but Akatsuki is not finished...and the elders in Konoha might still perceive other possible threats to Naruto, who might want to use him to obtain the Kyubi's power "to unleash it again on Konoha." So they wont let him leave the village again, even if they know that he's more than capable of taking care of himself against enemy.