I have uninstalled built-in music player (with orange icon on the pic below), but when playing music through another player it's icon and controls somehow appear back! This makes me feel uncomfortable, it spoils perfectness of OS. How could I remove this non-installed player controls?
These controls are not of the default Music player but the default audio controls of the elementary OS. They look like the controls of the already uninstalled player because the design of both is made the same. Every time you start audio from the browser for example these controls will appear.
I do not know how to remove them without problems for the os, but if the design of the icon is what botters you, you can change it using custom icon pack or just replace the icons with a name
/usr/share/icons/elementary/apps subfolders with a one that you choose.