Algorithm Nation
I am a recommendations outlier.
Recommendations often fail to impress me, even when they are from someone I trust. There are people who I trust with my life. I trust their opinions and their taste, and yet, their recommendations frequently aren’t good for me. All media seems equally specific and yet obtuse.
I don’t only love obscure bands, and I don’t hate big bands or their records. I love Johnny Cash, Run The Jewels, and Appetite for Destruction; and I love Mudhoney, Kenny Hoopla, and Open Door Policy. What record do you recommend for someone who counts Rocket From The Crypt, Ghostface Killah, and PJ Harvey as three of their favorite recording artists? Would you have guessed it was Tierra Whack or John Darnielle that I was enjoying the most this month? I wouldn’t have guessed that I listened to Lifter Puller more than any band last year, and I like them more than The Hold Steady, with whom I started. Even I don’t know how to predict it.
There is no fault with these human recommendations, they simply can’t calculate all the factors that I automatically and unconsciously calculate when I consider something. And any algorithm you could write that recognized it would really only work for me (or for such a small number of people that it wouldn’t be worth your time to write it).
My best friend thought I might like American Horror Story. And you know what…I might. His recommendation just can’t override my own hesitation to dig into a horror story. But, you say, “You’ve watched so many other horror movies in the meantime.” You’re right, but that’s a show, and each episode is an hour long. ”But, you’ve watched a lot of crap that is certainly of lower quality than AHS that is an hour long. I mean, Jack Ryan? Come on.” You’re right, but that is mindless and I didn’t have to pay that close attention. “Yeah, but what about…”
Anyway, my point is that I really did watch many, many shows after I got the recommendation for AHS. There were many others that fulfilled or escaped some minuscule condition that I formulated on the spot, remote in hand, as I scoured Netflix, Hulu, Prime, and HBOMax deciding what to watch. [I finally watched a couple episodes of AHS because my daughter mentioned her interest in it. Holy shit! is it gnarly for basic cable.]
I watch a lot of TV — too much, according to the viewing history I researched to maintain verisimilitude in that imaginary conversation — and yet, streaming services constantly recommend stuff to me that I have zero interest in watching. Not only that, but Netflix specifically made it harder on themselves to find something for me. The “like” button keeps a watched program in my recommendations. It is bonkers to me that they don’t know that I rarely watch things again. They even have a list for things I might like to watch again.
And yet, right now, on my Netflix homepage online there exists…hold on…let me count…twenty-one shows or movies that I have watched every bit of, most in the last three years, many in the last year. And this is just scrolling down to see the first five, not the total of each category. (And I didn’t count “Watch it Again.”)
I have been using Netflix since the DVD days, and I don’t want to take the time to download and count, but I imagine I’ve watched at least 1000 hours streaming, plus another 200–300 DVDs in that time. How can they not know that recommending reality* shows to me is futile? I have watched exactly two, and both of them were ones that I looked up to watch. To be fair, I still share an account (separate profiles) with my ex-wife, and her watch history has reality* shows, but for me, the algorithm is a failure.
I once got riled about this and messaged with Netflix support. I was in a cantankerous mood and must have had time to kill, because this is a quote from the transcript:
Jill (Netflix): Not a problem, basically you don’t want that Netflix offer you titles that you already watched.
Me: That is correct. I don’t want Netflix to keep offering shows or movies that I have completed.
Me: I thought we had established that this is not possible…My question is:
Me: Why is this not possible?
Jill (Netflix): I got it know, thanks for the heads up :)
Jill (Netflix): This is not possible at the time because Netflix has a matching system that link the movies and shows based on genres.
Me: I can’t keep asking the same question too many more times…It is starting to get frustrating. One last time: Why can’t I remove individual titles from recommendations?
Jill (Netflix): You can use the thumbs down options to hide those tiles at the moment, in that way, Netflix will not recommend them to you anymore. I’m sorry that we don’t have a way to hide individual titles from recommendations at the moment, but I will note your feedback so we can improve our service into our platform.
Note that my only way to stop the recommendations of things I’ve already seen is that I have is to rate the things I liked as though I dislike them. How much more broken could that recommendation system be? Or do they not even care what I click? It may be that everything is based on my viewing patterns and how much time I spend hovering on a link or reading about a show.
I sat and spent some time thinking about the recommendations that work and don’t work, and why. And then I thought about why they might work for others but not me. Then I thought about how they could work, and why they probably never will.
[Update: Upon re-reading this months after the original publication, and additional consideration has been bubbling up in my thoughts. I think that Netflix might be re-recommending so many shows because it signals that they have a lot of shows and that I enjoy those shows, making it more likely that I will continue my subscription.]
Why algorithmic recommendations don’t work for me personally.
I mean, very personally. Not why they don’t work for me in the philosophical sense, but in the actual, real-world, what-does-Josh-want sense. I think recommendations don’t work for me for several reasons.
I am very specific and particular about my entertainment.
I have a deep viewing history, and have cultivated a sense of expectation about what I want to watch. I was one of the few in Gen X who was raised without a daily diet of TV. I got my fair share of viewing in, but it was limited by the fact that from ages two to twelve there was no TV in the house I grew up in. And in the households that I visited that had a television, I had to share, or was limited in viewing time. This lack of freedom meant that each choice to watch something had more weight. When I was at my grandparents house, where I spent a good amount of time, I was limited to choosing one, or maybe two, shows per day to watch, which is plenty, but as a kid, during the summer, in the 1980s, it wasn’t much. When I was young, Dukes of Hazzard or Gilligan’s Island would suffice, but eventually, my taste developed.
I frequently make decisions based on what I know in advance of a particular piece of entertainment.
If I am interested in something, I commit to lowering my knowledge. I know whether or not I want to see John Wick or Get Out, so I don’t need a trailer or article.
There is no incentive for a media company to ever point you to any content other than what it has financial incentive to point at.
Entertainment Weekly*, for example, may maintain some respectable journalistic integrity in their reviews, but reviews are a relatively small part of the financial success of most media. Reviews, for me, can be a helpful guide to find what I want to spend my time on. Generally, not to find what I know I want to watch, but to find what I may have overlooked. Netflix recently debuted a movie called Triple Frontier (what a terrible name). I watched it the weekend it came out. If I’m doing something non-writing (paying bills, designing, posting, etc.), I like movies on that I can follow without too much attention paid. This movie is a fine example. I watched half the trailer, saw guns, fast vehicles, and unearned masculine arrogance, and knew that I would make it to the end of the movie. I still haven’t watched Bird Box or any of the top twenty movies on Vulture’s list of the best Netflix original movies. (Wait, I did watch The Apostle, #16. I liked it a lot. Audience score on Rotten Tomatoes: 54%.) I will watch several others, but none have moved ahead of what was already clogging up “My List” before they came out. Some of this is aspirational, but Roma, The Kindergarten Teacher, and Private Life, all deserve more attention than the half-assed time I gave to Triple Frontier, so I save them for uninterrupted movie watching time, which is rare. My point is that Netflix doesn’t seem to know the difference, because they keep suggesting Bird Box, even though I’ve never even paused on it for a second’s attention since watching the trailer.
I follow a distinct set of creators. Musicians, authors, writers, artists, actors, directors, even occasionally producers or publishers. I will give a shot to something that I might otherwise pass on if a specific person is involved. And they don’t have to be the star. If Melanie Lynskey, Regina King, Toby Huss, or Karyn Kusama are involved in any capacity, it will make my list. Yasiin Bey, John Brandon, Dave Chappelle, Dan Clowes, Dave Cooper, Craig Finn, Sage Francis, Zach Galifianakis, Ghostface Killah, Mike Patton, John Reis, Liz Suburbia, Jim Woodring, and on and on. They all get first looks. I will spend time with otherwise uninteresting art if they contribute. I have never watched a full Freddy, Jason, or Michael Meyers movie before, but I will probably watch the latest Halloween because Toby Huss appears.
Following creators is the lowest context way to get involved in an entertainment. Familiarity doesn’t make something good, or even enjoyable. I find new creators through association with familiar creators. I wanted to like an album named after my (our) hometown, Anderson .Paak’s Oxnard, but two run-throughs didn’t hook me. Maybe after some time I’ll be ready for it. I’m hearing that Ventura is better, but if that doesn’t hook me, .Paak will get a pass in the future.
This brings up a key point for music algorithms. The last time I tried Pandora (maybe 2013/2014) I got burnt out because it’s focused on individual songs. I rarely listen to individual songs. I like albums and playlists of individual artists/bands. I tend to even avoid greatest hits albums and soundtracks. There are few algorithms that focus on albums, and therefore they all overlook me. I get way more value out of the lists at the bottom that include common songwriters or performers than of anything on Apple Music’s “For You” tab. For example, I just looked at the “For You” and scrolled down to “Thursday’s Albums.” There are four recommended “since you’re into Raekwon,” and three of four are already in my library. The only album they came up with that’s not already in my library is Kendrick Lamar’s To Pimp A Butterfly, which I listened to several times two years ago, but it didn’t hook me. At least they didn’t recommend DAMN. since it’s already in my library.
The things I watch tend to be things I’m sure I want to watch.
This creates media “silos” like the news silos we find ourselves in, but having tightened my attention budget, I don’t take big risks often. Unless I intend to watch passively, like with Triple Frontier, I simply quit watching, or listening, or reading, if something can’t hold my attention. I discussed some reasons for this tangentially in “The Writer’s Age of a Visual Medium”. I listened to every minute of about 200 consecutive episodes of the podcast WTF with Marc Maron, but only listen to the interviews of about 60% now, and only about 10% of the front matter. The remaining interviews either are deleted immediately or get deleted if the first ten minutes don’t engage me. My attention has become more selective.
I actively play against algorithms.
The biggest challenge tech company’s algorithms have with me is that I am cautious about how I share information online, and often work against the tracking devices put in place. 90% of my searches take place via Duck Duck Go, I don’t sign in to Google or use Google Docs unless a collaborator wants to, I don’t post very often on Facebook, or Instagram, or Twitter, and when I sign on, I sometimes purposely counter program, “liking” both right- and left-wing political opinions that I disagree with, following many websites by traveling directly to the url, or using RSS, and not “liking” their posts on social media. My Apple News feed includes the Washington Post and Fox News, and I distrust their narratives equally. I should say, rather, that I trust their bias equally. And I never, ever, “like” or “heart” or “share” an article via Apple News or Facebook, lest they give it more weight somehow. I will occasionally “dislike” or “hide” things so that I see fewer headlines about celebrities and murders, but it doesn’t seem to work because I still find too many to tap on. That’s my own fault — they can still track my attention, as I am certainly susceptible to confirmation bias as well as hate- or shame-clicking. This is why I find Apple the lesser of two evils between them and Android; they seem to protect my information slightly better.
Why recommendations may work better for others.
Most folks consume entertainment without much thought.
I don’t intend this as a criticism. My dear sister, who I would never begrudge her media habits, consumes books, magazines, podcasts, and music. Why she likes them is rarely important to her; she likes them, that is what is important. I am simply much more obsessive about what these things mean, what the value of art and entertainment is, and why it’s important to choose based on even a capricious system of merit. I know I am in a small minority of the world population. And I don’t mean only because I am incredibly privileged. That is a factor, but additionally, I just think about this stuff more, which is neither good nor bad.
It can be very difficult to tell high from low quality.
I think of my own ability to judge, especially when I was younger. There are areas where I don’t have the experience to judge whether something is worth my time. There are albums I listened to fervently five or six years ago that now sound boring and dated. There are artists I loved to pore over who now bore me terribly. There are books I dug into and read series of that I couldn’t bear fifty pages of today. Considering that entertainment’s biggest consumer group is younger than I am in my mid-40s, at best their taste is less developed. And I’m reaching the point where I care less about good from bad. I feel less judgment today toward entertainment than I did even ten years ago. I spend plenty of time looking at Instagram, which outside of personal acquaintances consists mostly of skateboard videos and comics artists’ work-in-progress. I just want a diversion sometimes.
Most consumers have to be told what is available.
The way they are told is in the silo they, or the algorithm, have created for themselves. I aggressively curate my recommendation engines. This does create something a silo, but like I said, I want some connection with the recommendation. Today I find myself searching viewpoints that I don’t have a deep knowledge of, particularly the historically marginalized, and seeing what strikes me. My Twitter feed is loaded with people of color, queer people, women, and marginalized cultures, because I am always interested in underground artists.
The Sopranos Season One is mostly junk.
My taste is my taste, and I find that being told what to watch doesn’t often work.
I need to have some type of qualitative reference point or instinct that the media is of reasonable quality. Some media are easily identified as such. Substandard quality in comics is easy to tell at a glance. If a compelling trailer can’t be cut for even the most boring movie, then either no one cared enough so I don’t care to watch it, or the film is junk. Music is the most taste-specific in my consumption pattern, and I find it easy to tell if something is of interest to me within a vaguely attendant listening of an album.
Television is the hardest, because even my favorite shows have weak first seasons. Personally, I think The Sopranos Season One is mostly junk, and that Mad Men, Veep, Breaking Bad, and even The Wire start off on the wrong foot. Deadwood seems to be the exception that tests the rule. Maybe it’s just that these shows get so much better as they hit their stride, but I’m sure there are good shows that I don’t watch because I lose interest during the first season. Conversely, there are a handful of shows that start so strong, I sometimes wish I stopped after two seasons. Hannibal being my strongest argument.
I acknowledge that free will is much rarer than I’d like to think.
I am a product of my conditioning and my brain function in ways I definitely do not control.
For most media, the talent are the main consideration.
Pop culture is defined by what’s popular. I don’t understand why I seek new creators, but I also don’t understand why others don’t. I already explained that I am obsessive about the people who make the movies, TV, comics, and music that I like, but I speak to people about things all the time and they can’t tell me who made it. I recently had a conversation with someone I trust to recommend quality. While we discussed the Amy Adams, I admitted I hadn’t seen The Fighter, and they said it was great. I mentioned that it was also Darren Aronofsky, and they said, “Oh, I don’t really pay attention to directors.”
Nostalgia seems to be a strong draw.
I have little interest in nostalgia content. I loved The Dukes of Hazzard and Gilligan’s Island, but that doesn’t mean I have any interest in revisiting them to see if my nine-year-old taste holds up. I don’t understand why anyone is happy when they are watching a movie simply because it makes a reference to an older movie. That’s cool that you found an easter egg, but if it doesn’t deepen the meaning or doesn’t support an actual joke, I don’t know why anyone cares that Tobias Funke appears in Avengers: Infinity War? That may be fun, but is there an actual joke there? Did we cheer because the Millennium Falcon showed up in a Star Wars sequel? I don’t get the excitement. I think that there may be some actual value in setting the tone for Episode VII, but what exactly are we cheering? “Yeah! I see that thing I know!” They’re talking down to us and we’re celebrating it.
Celebrity is the same type of bullshit. Dana Carvey (I think, I couldn’t find the source) once gave an analogy in an interview that I’m paraphrasing here:
If they put a cantaloupe on television for five minutes every night for a week, then they took that cantaloupe to the mall and set it on a pedestal, people would walk by and say, “Hey, that’s the cantaloupe from TV.”
I’ve never heard a more cogent criticism for how powerful familiarity is in entertainment. Whether we think about celebrity, nostalgia, memes, or fan service, we are not talking about quality.
Sometimes we just want to feel good about our decisions.
Having worked in retail for decades, there are numerous times when people have already made a decision about what they are buying and at the point of being rung up they ask, “What do you think about this?” It is simply an opportunity to confirm that they made a good decision. Saying, “That’s a great choice,” is always a successful approach.
How recommendations could work for me, but likely never will.
Recommendations need to be divorced from sales incentives.
Netflix is a new kind of media company in a key sense. They use cold, hard, data to determine what to buy, what to create, what to present to whom, and how to advertise. They are missing the art of it all, but that’s not what their business is built on. To make a recommendation truly worthwhile it has to separate itself from financial incentives, especially short term incentives. HBO seems to be a strong case for this. They gambled that they could make a hit out of the most expensive television show ever, Game of Thrones, and it paid off.
Major props, by the way, to Netflix for this tweet. This is the kind of thing that builds long term customer appreciation from me.
Ask the customer what’s important, then reprogram the algorithm.
Very successful companies traditionally focus on things that most companies would sacrifice for increased revenue or profit.
Hobby Lobby is still closed on Sundays, even when most retail businesses find it to be one of the biggest days of the week. When I take my kids to Michaels on Sunday, it is packed (and I live in suburban Texas, where church attendance is high).
In-N-Out keeps selling those terrible french fries—when people would enjoy the taste of frozen re-fried fries better—because they are “never frozen.”
When it comes to algorithms, they need to ask better questions. When I signed up for Apple Music (which thankfully makes it easy to listen to full albums), they started with a recommendation engine in which I had to choose artists I like. The problem with this is that not one of my favorites was on there. Sure I like Guns ’n’ Roses and Metallica better than Def Leppard and Avenged Sevenfold, and I like Jay-Z and Kanye better than Snoop Dogg and Dr. Dre, but I like Conor Oberst and Anthrax and Del Tha Funkee Homosapien better than all of them. So, all Apple could find out was what bands rank 50–100 instead of 1–50. I know I am an outlier, but wouldn’t a system that takes that into account make everyone’s algorithm better?
Free recommendations from advertising.
Co-op is co-opted. It’s not a coincidence that Heinz is at eye level on the ketchup/catsup shelves. It’s no coincidence which television you see first at Best Buy or Walmart; which result comes first on Google or Amazon; that Zynga apps get recommended on Android and the Apple App Store. That Medium posts focus on recommending “member-only” content. I once paid for advertising (membership) on Medium, I am not overly anti-capitalist, but advertising does not make an algorithm better.
Tie criteria from one medium/platform into another.
Caveat: I know that privacy concerns make this nearly impossible, and that even if I could consent to this, I frequently wouldn’t.
It seems to me that Netflix would benefit from knowing what music I like. And Apple Music would benefit from knowing what movies I watch. I do not want content silos to get stronger and more siloed. But it could make the recommendations better.
So why did I even write this article?
I don’t really care to help a multi-billion dollar company make their algorithm better. I spent all this time thinking and writing about this because I love small businesses. The best recommendations came from the times I walked into a record store or comic shop or video store and they knew me personally. I like working with these retailers to make their business stronger, and their brains are the algorithm that builds their business. It’s something tech companies could learn from.
Footnotes
”Reality” is a misnomer for me. Without an extended tangent about the philosophical impossibility of objective truth, suffice it to say that there is nothing real about anything that anyone publishes in any format. “History is just a lie you read about.” — David Milch
Entertainment Weekly is owned by Time Warner (or whatever name they currently use). Even if we exclude direct financial benefit — covers featuring Game of Thrones or Harry Potter; articles on advertiser’s content; studio support with “exclusive” previews and cover photos; etc. — there is little arguing that having 59 covers for 22 movies is focused on financial gain. Stars sell magazines. There is a reason that EW doesn’t feature Ta-Nehisi Coates or Roxanne Gay as their cover articles. And it’s not racism, because Jonathan Franzen and Dave Eggers don’t make it either. A quick, lazy internet search reveals Rowling, King, and Grisham as those who made the cut. I have no beef with any of them, but you can’t tell me it’s focused on anything but sales. And to be clear, I’m not making the case that a magazine should focus on anything but sales, if that’s what they want. Just that financial incentive is the motivation, not making quality recommendations.
Cutting Room Floor
I am patient in deciding, to the point that deciding is often enough time spent on entertainment and I don’t continue past the deciding. I spend a lot of time thinking about why I consume what I do. I am skeptical of popular media, and skeptical of ignoring popular media. I was excited to see Star Wars The Last Jedi (Rian Johnson helped), and have never seen Avatar, even though I love Aliens and enjoyed T2 and True Lies quite a bit. I was recently thinking I may give Waterworld a try when I need a brainless background movie. At least Dennis Hopper is in it, and he’s interesting. [for the record, I made it 30 minutes into Waterworld, then gave up.]