That's the whole point of Unversioned media items. Unversioned means that there are no separate language versions. It's one shared.
If you set one in English, it will be same for any other language. If you change something in Spanish, it will be applied to all languages, including English.
If you want language specific media items, use Versioned template.
It's a bit confusing as when it comes to fields it's Shared checkbox which states whether field should have only single value shared for all languages, and Unversioned field checkbox has another meaning. Naming Unversioned for media items suggests something else. But still, this is how it works.