Apple has not provided the option to delete full albums or artists through the device, instead users can only remove one song at a time or their entire library. Learn how to delete your entire library here.
Remove Songs Uploaded From iTunes
This is the standard procedure for removing a song.
Remove Songs Downloaded from iCloud (Purchased Music)
Use this method for songs you cannot get rid of using the normal left swipe method.
1. Remove the song by swiping left.
You will notice the song is still there and that swiping left will have no action for that specific song.
2. Go to Settings > Music > Show All Music – Make sure this option is enabled.
3. Go to iTunes Store > Purchased and re-download the songs that you want to delete.
4. Go to Settings > Music > Show All Music – Disable this option
5. Go to the music player and remove the song by swiping left. This will delete the song.