To get an idea of why the Greeks thought foreigners were all barbarians speaking nonsense, try listening to some videos of people speaking languages you don’t understand. Here, for example, is someone speaking Greek.
Speaking modern Greek
On the other hand, here is someone speaking Farsi – Persian. Can you tell the languages apart?
Also check out these videos of people speaking various African languages – Xhosa, Swahili, Yoruba, and !Kung.
More about Greek views of barbarians
Bibliography and further reading about Greek ideas about “barbarians”:
Eyewitness: Ancient Greece , by Anne Pearson.
Greeks and Barbarians, by Thomas Harrison (2001).
Greeks, Romans and Barbarians: Spheres of Interaction, by Barry Cunliffe.