Breach of Close

Wolfgang Tillmans, Anders pulling splinter from his foot, 2004

Not being an industrial spy or a corporate lawyer, I don’t ordinarily find reading patent applications terribly interesting. But I recently spent an entire weekend happily poring over one particular patent, despite the fact that reading such documents, like eating an artichoke, is hard work and offers only delayed gratification. The heart of the invention—the summary description—lies deep within layers of nested dependent claims of apparatuses and methods, each of which, however, offers its own tiny bit of additional information about the invention.

The patent I was studying, application no. 20060242139, was filed by Yahoo, Inc. for an “interestingness ranking of media objects.” A chunk of it covers the algorithm that flickr has developed to select the 500 photographs for its “Explore” page. (An interactive mosaic wall of thumbnail images of the daily 500 can be seen at http://interes.tingness.com/)

I had first heard about the algorithm from my friend Nikolas, who is one of the most interesting people I know and a remarkable photographer in his own right. I was immediately intrigued, which is just a fancy way of saying I was interested. How do you measure something as nebulous as interestingness, and indeed, from the point of view not of the individual but of the community? Although the set of 500 photos varies from day to day, on any one day they are the same for all users, regardless of individual members’ own particular interests. You personally may be fascinated by deadpan photos of industrial buildings or elaborately staged compositions of domestic life gone awry, but clicking on flickr’s Explore button will deliver you the same mix of cityscapes, furry creatures and soulful women that it does for your virtual neighbors. That said, the interestingness algorithm does work on more targeted subsets of images. You can sort your flickr search results for cast-iron buildings or water droplets on the basis of interestingness.

The real power of the metric lies in its potential to query and retrieve images of interest to the individual. The patent itself suggests some ways in which this metric can be fine-tuned. One such tweak involves using location. A photo of a bridge might be more “interesting” if it’s close to where I live or where my friends live or a place associated with other images that I’ve already tagged as favorites. This is where the big bucks lie and the ultimate reason for developing the algorithm in the first place: a better way of googling for images, or rather I should say, flickering for images and getting relevant, user-targeted results.

The success of the metric for retrieving individualized search results depends in part on the size of the set of images over which the algorithm queries and the degree to which the images themselves are tagged with richly informative metadata. But with over five billion images and a community of users who have done much of the work of tagging the photos they upload, much of the foundation has been laid.

Google, of course, has an exponentially larger set of images, but not having an army of avid taggers it must rely on other cues to determine relevance, such as keywords in the text in which the image is embedded. When I searched for “empty chairs” Google returned 3.16 million results. But after the first 10 pages, the results became less consistent, contaminated by images that had nothing to do with the object itself (an album cover of Don McLean’s Legendary Songs) though they were linguistically relevant (the image of the cover comes from a text containing the lyrics for the song, “Empty Chairs”). At about page 55, fewer than one in ten images actually contained an empty chair. Flickr, on the other hand, returns only 32,000 items from its “Empty Seats” group. I couldn’t muster the interest to browse to the end of the set but there were still only chairs when I got to page 101. And Flickr search results can be sorted not only by relevance and recency but also by interestingness. Which brings us back to our initial question: what makes an image interesting to the community.

Curiosity, Pleasure and Investment

Interest is born of curiosity. Like a stranger’s smile at the far end of a bar, something about a  book cover, the snatch of a melody, or the composition and texture of an image invites us to tarry. It beckons us to explore, to leaf through the pages of the book, to look more carefully at the picture, to stop what we’re doing and listen to the rest of the song.

It is the mind pricking up its ears. Flickr captures this “desire for more” by measuring the rate at which users click on a particular thumbnail image to see the photograph in greater detail and to read whatever information the photographer has provided about the image. This in itself doesn’t really say all that much. On its own, a click is nothing more than a stop at a painting along the path of an exhibition. (Sadly, the average museum visitor spends only 8 seconds at each painting. Lest you think this is merely a case of rushing-through-the-Louvre museum fatigue, you might want to look a visitor study that was conducted at the Musée Cantini in Marseilles. The research team observed visitors to an exhibition of twelve masterpieces by Picasso and Matisse that were on loan from the Orangerie Museum in Paris. There were just twelve paintings. Twelve. By Picasso and Matisse for God’s sake! The average time of observation of any given painting was still only 12.5 seconds.)

Interest is more than mere curiosity. A click-through rate doesn’t reveal anything about our experience once we’ve spent some time with the object that has initially aroused our curiosity. The first paragraphs of the book can be riddled with clichés, the stranger who smiled might have bad breath and the photo could turn out to be trite upon closer observation.

If interest is to be more than fleeting, exploring the object of our curiosity must give us something—giggles or goose bumps, a tingle of pleasure or a prick of doubt, an occasion to think and make connections, something in any event that quickens us. Flickr approximates this positive feedback in its algorithm by counting the number of users who have marked a particular photo as a ‘favorite’.

However, interest that remains at the level of curiosity, even when it is a source of pleasure, is shallow, quick to evaporate in the heat of competing priorities.  If interest is to endure, it must be cultivated, and this requires some investment on our part—of attention, time, and often money, though money is perhaps the least important. Buying a book is not the same as reading it and much less recommending it, and best seller lists are not necessarily the lists of the most interesting books. Since there’s no real cost in clicking on an image, and not much more in marking an image your favorite, there’s very little if no user investment.

Apart from your time, the only other resource your can commit on flickr is the universal currency of all social networks: your reputation. Some of this is already wagered in tagging favorites, which function then as a mark of identification; you are what you like. But the majority of investments in time and reputation are recorded and measured when users comment on photographs they have seen on flickr. In the language of the patent, the interestingness rank is determined in part by “the quantity of user-entered metadata concerning the media object, the number of users who have assigned metadata to the media object.” (Some bloggers have noted that the metadata captured in the ranking also concern what the uploader and not just the visitor has provided; untagged photos are generally ranked much lower than tagged ones).

Most interests have a relatively short half-life: they flag and sag and finally pass. Only a few really last. Flickr factors in the decay of interestingness, admittedly somewhat arbitrarily. It decrements the ranking a photo will accumulate over time by a factor of about 2% per day, starting from the day the image was posted. Ingenuously, this decay factor prevents a highly ranked photo from going viral, fed by an endless positive feedback loop whereby the image, precisely because of its interestingness rank, is seen by more people, who in turn are more likely to click it, mark it as a favorite and comment on it.

Explorers, Sentries, Advocates and Curators

There are comments and comments, of course.  “Wow” is not the same as “The contrasting red and blue make for a dramatic image… and that sun-star is a great accent mark”. Some people’s comments are more carefully considered and more informative than others, and in the end worth more.

Length alone is not always a reliable clue to the value or relevance of a comment, and since flickr cannot parse these texts, it gives comments from members whose own photographs have been commented on—and who are thus likely to be better photographers—greater weight in the interestingness metric than those whose images have not. This may explain in part why the photographs in the 500 are technically quite accomplished even when the subject matter is not too adventurous. Flowers and sunsets figure prominently, as do beautiful women. There are cars and cats and on one day oddly enough several coffee cups.  But mostly they are “ah” pictures, things like dramatic bridge shots, far vistas and macro close-ups of insects.

The taste of the crowd, however, may not necessarily be mine. Sometimes they coincide, as in my enthusiasm for the highly rated tv series The Wire and Glee, but often they don’t. I’m much more interested in the wisdom of the niche and the taste of the tail. I’ll readily follow people on last.fm who listen to drog_a_tek (the album a gift from Nikolas, but more on that later) in order to explore what else they’re listening to.

I would never have discovered Giacinto Scelsi’s hauntingly beautiful Uaxuctum (1966) had it not been for NewMusicXX, a remarkable youtube channel of twentieth century music maintained by Max Ridgway.  Or György Kurtág’s ghostly Kurtágonals (2009) if not for Thierry, an ardent and very knowledgeable salesclerk at the Brussels fnac.

No, it takes more than the great mass of consumers for us to stumble upon or indeed be led to truly interesting creative works. We need the explorers, purposeful researchers like Thierry who seek out and track down the works of artists whose interestingness has not yet reached a critical mass. We need the sentries, the bots and feeds and aggregators like Digg that simply scout out whatever appears to be on the threshold of going viral, like epidemiologists on the lookout for the first signs of an outbreak of disease. We need the advocates and propagandists (in the best sense of the word) like Ridgway.

We need the curators, too. People like Maria Popova, who styles herself an “interestingness curator” dedicated to “picking culture’s collective brain for tidbits of stuff that inspires, revolutionizes, or simply makes us think”.  Her site Brain Pickings is an ever-changing treasure-chest of intriguing ideas and innovations across an astonishingly broad range of disciplines. A morning with brainpickings turned me on to at least a dozen new ideas/books/sites/events I wanted to explore further. The story of how the New York Subway came to use Helvetica type, Steven Pinker’s talk on TED, the Holstee manifesto on better living. Some of these will surely be fleeting interests, intentions that flared up in the enthusiasm of the moment but rapidly petered out, like the all too brief euphoric high of nitrous oxide, but others may last.

David Rowan, in his Wired piece on the interestingness curators of social news cautions that following the interests of like-minded people risks missing out on “serendipitous discoveries”. But there’s really no one just like me (thankfully, because that would make for very boring conversation). My friends may broadly share some of my interests, but never in the same degree or with the same depth of knowledge. I continually learn from my friends. While I do make such serendipitous discoveries on my own, I am grateful to Nikolas and Jörg and Dieter and Natalie who have led me to discover Nan Goldin, negonis, Six Feet Under and the best chocolate cake I’ve ever tasted. And a host of other wonderful things.

Brendan Fernandes, New Primitivism 2

Natalie gave one of her fabulous dinner parties again. I still don’t know how she manages to make it all look so effortless. I know that a lot went on in the kitchen so that we could sit down to a yummy gingered carrot-and-coconut cream soup and a perfectly roasted wild turkey with a cranberry rice stuffing. But Natalie is so cool about everything you’d think she was one of the guests. Maybe it’s because she’s truly more interested in her guests than in the food, however scrumptious her cooking is.

She always manages to assemble the most interesting people around her. Or people who can talk interestingly about a lot of interesting things. Not sure if it’s the same thing. In any event Natalie’s parties make me feel smarter than I really am. And I always have the sense that the day after the party I’ll have something I hadn’t had before; it’s like the morning after a sex party in a Berlin fetish club: you discover you’ve got phone numbers in  your pocket or an invitation for a weekend in the Brandenburg countryside or Chlamydia. In the case of Natalie’s parties it’s ideas and stories, or just things that stick in your mind.

I drank more than I should have. I’m never fully conscious of how much I drink at Natalie’s. Someone always seems to be filling my glass, though of course that begs the question why my glass keeps emptying itself in the first place. The wine at Natalie’s is as good as the conversation, and both run like a coursing stream. It feels a bit like Piaf’s song, Let Mots d’Amour, how the last words of one line start the next, and the song starts to cascade along like white-water rapids. I don’t know which is more intoxicating, the talk or the wine. Ok, I do know which, but I still got swept up in both.

With the pending passage of the most anti-labor labor legislation in post-war memory and a 2011 budget that foresees even higher taxes and deeper wage cuts, it was inevitable that the conversation would turn at some point to politics. There had been another general strike that Wednesday, which degenerated into the usual vandalism along the main commercial avenues: store windows smashed, garbage dumpsters set afire, bus stop shelters torn down. We agreed this was clearly misdirected anger.  Timothy said it needs to be rechanneled to a more productive target and suggested tax-evading doctors, a proposal we all seconded.

I think Natalie was probably ready that evening to take charge of the guerrilla action herself. “All the social institutions that could do something—the media, the courts, the police—they’ve all been so corroded and corrupted that we can’t expect them to do shit. It’s like they don’t exist,” she said. Since we’re living in an ever more primitive society, why not embrace it, she said, and act like primitives, but with a purpose. And it was clear from what Natalie was saying that this purpose would certainly include trashing the plush offices of doctors who declared less income that an assembly-line worker.

That night we called it neo-Primitivism. We must have drunk too much to remember that neo-Primitivism already exists (existed) as a movement (it’s as good thing we didn’t waste time putting together a manifesto). And at least twice. First as a post-revolutionary Russian art movement (Malevich and Chagall were proponents), that sought to fuse Futurism and Cubism with Russian folk motifs. And in modern times as a sociological position that argues that with the erosion of modern institutions people are organizing themselves on a tribal basis, where tribe means things like “brand” identity, social networking communities and the like.

But the idea of tribes fascinated me. (It was one of those phone-number-in-my-back-packet ideas).

I thought about the “taste tribes” at Hunch, iconic communities organized around common likes and dislikes. I had read about this website in the paper. It claims to provide targeted recommendations for books, music, DVDs, blogs, even restaurants, classical musicians, and jeans—hundreds of things actually—all on the basis of a “taste profile” it compiles from the responses you give to 20 simple questions. The questions have nothing to do with the books you like to read or the music you usually listen to but instead things like whether you like your sandwich cut down the middle or on the diagonal? Or if  you have a basement in your house.

I thought, come on, this is bullshit. 20 questions and they know who I am? But I figured, it was thought up by a group of young people mostly from MIT and apparently some big players (on-line retailers, booksellers, etc.)  are very interested in it, so I decided to give it a shot.

I answered the 20 questions and it brought me to my “results page” with recommendations for DVD boxed TV series and magazines. It freaked me out — all the recommendations were exactly on target!The magazines Hunch recommended– Economist, New York Review of Books, the New Yorker – are all ones I read more or less regularly, and the TV series suggestions (The Wire, Six Feet Under) were shows I had already raving to friends about. Granted, The Wire has received stellar reviews from practically everywhere, and it could have been a safe choice but it’s not just a matter of popularity I think.  Desperate Housewives is popular but that didn’t come up.  Spooky.

The hitch to Hunch is that you have to give it the right to access your Twitter or Facebook accounts whenever it wants. Natalie didn’t like the idea of a file being started on her. Or was it, another file being started on her? I can’t remember. In a witty play on words in the equivalent Greek expression she calls it being “enveloped”—in the double sense of being surrounded by the watchful eyes of online retailers and surveillance cameras, and the records of your commercial, social and medical transactions all being stuffed inside a large A4 envelope.

I tend to side with Natalie on the privacy issue. I don’t even have a Facebook account. I logged on to Hunch through a Twitter account I have that’s essentially moribund – just a log on name, no followers, no tweets. Yes, It felt like I was cheating, getting something for nothing. To make up for my thievery, I’ve made a practice of clicking the “like” or “dislike” buttons on Hunch when it presents me with recommendations.

The service is a brilliant example of the collective intelligence of the web; the uncanny “taste profile” it builds is a function not only of its amazing algorithms but also of the data it has access to.  And obviously, the more people allow Hunch access to their data, the more intelligent the crowd. But of course the crowd no longer owns its own intelligence, if it ever could have. And I think, if people started sharing their medical histories, workout results (which they already have started to do), food logs (ditto)  online on a massive scale—tens of millions of cases—we’d see unprecedented advances in medical science. Natalie would say, but what if the data wound up in the hands of insurance companies? Frankly, I’m willing to take the chance.

I later realized that of course there aren’t just 20 questions that everyone answers. And if even if they were only twenty, if each question has two answer choices (some have three, but let’s make it simple), that would theoretically make for  1,048,576 combinations of answers (2 to the power of 20) or a million taste profiles. So even if we’re being pigeon-holed, there are a lot of holes to go around.

A friend had asked me what I answered to the question, “will you go out of your way to step on a crunchy leaf?”. But this wasn’t one of the questions that I had answered. And then I realized, like the computer-adaptive GMAT test, Hunch uses a branching algorithm. Each subsequent question is chosen on the basis of your answer to the previous one. So it wasn’t the twenty questions that led Hunch to create such an uncannily on-target taste profile but rather the particular set of twenty questions that were selected just for me. So the set of possible questions and answers is astronomically large. So the taste profile could actually be unique to me!

In the end, it’s more about probabilistic relations than categorizing people. Like people who find department store Santa Clauses creepy are less likely to give their kids names with the same initial. And since a lot of people on Hunch share their likes and dislikes generously, there are a lot of data to correlate.

Hunch always seems to be asking questions. I thought I had answered about 50, but I just checked my profile for the first time and saw that it’s 263. Ok, answering the questions is addicting. They intrigue me. I want to understand what a question like whether I like or don’t like sprinkles on my ice cream could possibly correlate with. Or whether I prefer Plato or Aristotle.  Maybe it’s also addicting because Hunch is so tirelessly interested in you. Whether you think Israelis and Palestinians will reach a peace settlement, which comedians you find funny, if there’s anyone who I’d step in front of a bullet for?  No one is ever that interested in your views. But it’s the kind of questioning that get’s boring fast, because there’s never any follow up.

After answering a couple of sets of questions I checked back on the DVD recommendations and was shocked to find The Simpsons on the list. How did that sneak in there? Maybe because I’d said I don’t drink white wine (What I meant was that I usually drink red wine, but I doubt if Hunch could surmise that). It was right with my favorite soft drink (water), color in clothes (black), and jeans (skinny and straight leg). I don’t read much off the NYT best-seller list, but if I did I would probably read the ones recommended (The Omnivore’s Dilemma, Outliers) or had read them (Kitchen Confidential, Freakonomics, though neither would be in my top 20 books of the year). Mixed results for classical music (yes for Glenn Gould and Grieg but Mahler?). Gay vacations were also uneven:  San Francisco and Russian River were rightly there but Berlin should have been first not fifth, and Mykonos should have never made the list. At all.

I imagine I occupy a particular point in Hunch’s vast taste-scape, in a tribe of other users whose profile approximates mine. When I read user reviews of the recommendations Hunch makes, apart from their moniker and tags, I see how closely related they are to me.  Whether they belong to my tribe or not. I still haven’t figured out, though, how I actually go about talking to any of them. It’s nothing like the online gay cruising sites, which in many cases are also very much organized on tribal lines—muscle queens, radical queers, skinheads, bears, twinks, Log Cabin Republicans, rubber fetishists, down to the more tail-end niche triblets like MA1 gearheads and vacuum pumpers.

One of the tasks Natalie had given me that evening was to help her figure out whether Emilios, who unbeknownst to him was at the party as Natalie’s unofficial not-so-blind date, was gay or not. My gaydar picked up just the faintest of blips. He was perfectly groomed and manicured; there wasn’t a stray hair anywhere, and his eyebrows were overly symmetrical for a face with dimensions that often diverged from the golden mean. I sensed he probably owns a clothes brush. Oh, when we confessed to our obsessions that evening, he said his plants. But apart from that, zilch.

Next time I’ll ask him twenty questions.