I purchased the song Bongo Cha Cha Cha by Caterina Valente & Werner Muller and His Orchestra from iTunes – or whatever it was called at the time – on July 21, 2019. The 1959 recording is featured in Spider-Man: Far From Home, which had been released a few weeks earlier, and it became a bit of a hit.

About a week ago I realized I hadn’t heard it in a long time, even though I had tagged it as a favorite and it should have been cycling through several of my smart playlists. I searched for the track listing in the (confusingly-named) Music macOS desktop app and it was in there, but when I tried to play it, I got an error that the file was “unable to be located.” That’s annoying. I knew I had to have the file somewhere. I routinely make multiple redundant backups of my music and photo libraries.

iTunes Sync SettingsBefore digging into my archives, though, first I checked my iPhone and discovered the track was there. That should never be the case; I don’t use the (confusingly-named) Music service and I don’t let the Music app and/or Finder manage synchronizing my music with my iPhone. (See image.) There should never be a music file that exists on my iPhone but isn’t in my library. (There’s tons of music in my library that isn’t on my iPhone, of course, but the reverse should never be the case.)

How could a file be on my iPhone if it didn’t come from my library? There is really only one way this could happen. I could have purchased the music track using the (confusingly-named) iTunes Store iOS app and not yet synchronized my iPhone with my MacBook Air. I absolutely might have purchased the track with the iTunes Store app on my phone instead of within the (confusingly-named) iTunes Store embedded in the Music app on my laptop, but I have synchronized my iPhone with my laptop hundreds of times since the summer of 2019. There’s no logical reason the file would be on my iPhone and not be in my library. I had to have accidentally deleted the actual physical file from the music library folder on my 256GB external music thumb drive. (I offloaded my music library folder to an external hard drive well over a decade ago to avoid consuming so much of my laptop’s hard drive.)

This should not be possible, though. The first time I synchronized my iPhone with my laptop after accidentally deleting the file from my hard drive, either Finder and/or the Music macOS app and/or the Music iOS app should have thrown an error to let me know there was an issue. Something like, “Hey! There is a file that exists on your iPhone – one that you purchased from us, by the way! – that seems to have been deleted from the music library to which you’re synchronizing it. Do you also want to delete it from your iPhone? Or do you want to restore the missing file to your music library using the copy on your iPhone?” For no reason should that mismatch simply be ignored. But that’s what happened. And that mismatch was ignored every one of the hundreds of times I synchronized my iPhone with my music library for more than half a decade.

Ignoring that egregious synchronization bug, now I can only listen to the song on my iPhone because that’s the only place the file is. But I often choose to listen to music using my laptop instead of my iPhone. More than that, though, having the song on my iPhone but not in my music library is just not good. If someone steals my phone (or I drop it into a boiling bowl of chocolate like in that very weird Verizon Christmas ad that aired incessantly over the holidays), it would mean I’d lose access to the song.

Classics with a Chaser“But wait just a minute,” you’re probably saying to yourself while reading this essay instead of doing something dramatically more important. “If you purchased the song from the iTunes Store, shouldn’t you be able to just re-download the track from the iTunes Store?”

You’d think that would be the case, wouldn’t you? But you cannot. If your local macOS Music library database thinks that you already have a downloaded copy of the file – even though that database “knows” the downloaded copy does not exist! – the iTunes Store won’t let you download it again. (It will, quite oddly, allow you to play the song, but only from within the iTunes Store interface, which makes no sense at all. Who in the world uses the iTunes Store interface as a music player?) To be fair, you can get around this bug by deleting the (missing) downloaded track from your Music library, at which point the iTunes Store will (ironically) realize you don’t have a copy of the file and enable the option to download it again. But doing that erases any metadata you had associated with the track, and if you curate your smart playlists based on any combination of play count, last played date, last skipped date, added to library date, or any other metadata, you’re going to lose all of that logic because now your smart playlists will think the new file has never been played or skipped and was just added to the library. After 30+ years of managing all this metadata, I’ve become more than a bit precious about it. So I really only like to use this option as a very last resort.

All of that is irrelevant, though, because of an even bigger problem: The track I purchased — Bongo Cha Cha Cha by Caterina Valente & Werner Muller and His Orchestra from the album Classics with a Chaser — no longer exists in the iTunes Store! This is perhaps the most terrible aspect of the demise of physical media. I’m now in a position where, if I wasn’t me, I’d have no way at all to ever again play this song that I legally purchased and, theoretically, owned. There are a dozen variations of this song in the iTunes Store right now, and several remixes released after the surge in popularity from its inclusion on the Spider-Man soundtrack. But the version I purchased, the version I like, is no longer available.
This makes perfect sense for mega-corporations like Apple and Amazon because it means when I die, my digital right of ownership of that file dies with me and I can’t bequeath my digital music library to my sons like I would a CD or LP collection. If they want to listen to any of my favorite songs, they’ll have to buy their own licenses for them. This is reason #7,432 why I do my best to remember to always rip purchased music to DRM-free mp3.

So now what are my options for getting the song back?

Because the song was playable from my iPhone, I knew a copy of the m4a file I’d originally downloaded from the iTunes Store was somewhere in my iPhone’s memory. Unfortunately Apple really, really doesn’t want you to think of your iPhone as an external hard drive, and – like Google – they do their best – for reasons I cannot comprehend – to obfuscate the very concept of files and folders. But one of the main features of the iPhone is, in fact, a very tiny, very gigantic, solid-state hard drive, full of folders full of files. I haven’t attempted to do it in a decade, but I knew there had to be some knucklehead like me out there who had written an application that would allow me to connect to an iPhone and examine its internal filesystem.

It took me one Google search to find one. I downloaded the macOS app iMazing and used that to trick my laptop into treating my iPhone as if it was an external hard drive, found the file, copied it to my desktop, and then copied it from my desktop to my music drive. Then I right-clicked the track listing in what I still refer to as iTunes (but which is now, as I’ve mentioned, confusingly named Music) and chose “Get Info” from the context menu. That failed, as I knew it would, because the file wasn’t where the library thought it should be. Conveniently when the interface does that, the Music app prompts you to search for the missing file. I knew where I’d put the file I ripped from my iPhone on my thumb drive, so I was able to locate it there and re-import it, which preserved all my metadata. Whew. And as a final trick, I converted the track from Apple’s m4a format to plain old mp3 and deleted the digitally-protected version so I shouldn’t ever have to worry about this again. At least for Bongo Cha Cha Cha.

I’m simplifying some of the details here that aren’t relevant to the issue. For example, I didn’t “accidentally delete” the file from my music library. It was lost (along with a dozen other tracks purchased from iTunes) when my external music hard drive got fried when I inadvertently plugged it into a powered USB port instead of a data USB port in August of 2019. I hadn’t made a routine backup in nearly a month, so when I restored the library by connecting it to one of my backup drives, the database entries for any tracks I’d added to the library since the previous backup were now pointing to file locations that weren’t on the backup drive. So I have gone through this exact same frustration about a dozen times now.

Another annoying bug in the Apple music iPhone to library synchronization process is that because the *.m4a file existed on my iPhone all this time, I did actually hear the song every now and then if I was listening to my entire iPhone library on shuffle. But in that case the iPhone’s internal metadata should have incremented the track’s play count each time it was played, and synchronized that play count with the play count metadata of my primary, local, macOS Music app database. But it wasn’t doing that. It was simply – again, in an inexcusable failure of any type of logical data synchronization rules – ignoring that data.