Do recommendations scale?

"The Music Recommendation System is an automated system that provides music recommendations specifically tailored to each user to find new music that they might like. This system, designed by students at the University of Illinois (Champaign-Urbana), operates by taking ratings from your own iTunes playlists and comparing them against other users who have used the recommendation system."

Sounds good, eh? So I went there, downloaded, and then submitted the few hundred tracks I've rated in my iTunes database. Then I get this:

"Your estimated time remaining for results is 2.23 day(s)."

And that was with only 7% of my "catalogue" hand-rated. Ouch. Waiting 2.23 days for results points to a question of scale, methinks.

I don't mean to knock a bit of work which is clearly in prototype stages, but perhaps this is an exhibit B backing up Clay Shirky's intriguing argument about situated software. This looks like a case of 'Web School' software design; something built for mass use, trying to scale and failing. As I write, there are 2727 users in their database, 527494 songs, approx 322 songs per user, and by actually proudly displaying these kind of stats the project team seem to be broadcasting the notion that more recommendations here would necessarily be better (due to collaborative filtering's requirement for 'scale'). More is more. Their software isn't scaling though, and I wonder whether their recommendations are really going to deliver too. Even when something scales extremely well - say Amazon - I don't get particularly interesting recommendations.

Better to create a situated recommendations service built around my friends and colleagues perhaps? No need to scale there. Amongst those I know, there are some I'd turn to for music recommendations - and indeed I do, all the time, always doing a near-subconscious, complex level of filtering, reputation modelling, and drawing on history, context, intent, and all the richness of pattern in close, often physical, human interaction. In the complex cultural field of music recommendation, less may be more. I guess a Shirkian model would be a 'local' application enabling and nurturing that level of recommending amongst these <150-strong social groups. Basically, amongst me and my mates. Even then, reputation within a knowledge domain would be handled in more sophisticated fashion than simply "my default online social group", as I have good friends, online a lot, whose music I wouldn't go near with a barge-pole. So something a lot closer to the (culturally, physically) localised applications described in Clay's piece make a lot more sense.

"Situated software isn't a technological strategy so much as an attitude about closeness of fit between software and its group of users, and a refusal to embrace scale, generality or completeness as unqualified virtues." [Clay Shirky]

Compared with the alternative of staring one of the biggest 'hourglasses' I've ever encountered, it might be worth pursuing. Moreover, I'm not convinced that the eventual recommendations will be necessarily superior to the ones I get from my social set anyway. So there could well be a better 'closeness of fit' to explore here. But I remain open-minded, and will let you know.

University of Illinois: Music Recommendation System
Clay Shirky: Situated Software

Comments

[hans]

Although my >8k rated track count appears to be too large to even upload, I look forward to being able to use this service, specifically because it is webwide rather than situated. While an automated means to learn recommendations from my friends might be interesting, this is something different. The 713,654 tracks currently in their database dwarf those known to my circle, so I'm likely to be turned on to music I enjoy but might never have encountered otherwise. I'll gladly wait several days for results.

Of course, it's all moot until it scales well enough for me to even use it.

[Dan]

OK, so finally, over a week later(!), here they are. And while they're not as bad as I thought they'd be, they're still not that useful. Certainly not as useful as the recommendations I get from the trusted set around me. There are the usual predictable choices (Jan Garbarek, Elvis Costello, Neil Young), the tracks I already have (er, the same as the predictable choices), the 'good guesses that I'm not actually that interested in but could've been' (Balanescu Quartet. Bill Withers), the anomalies (Tindersticks? Not really), and then the just plain wrong (Jackson Browne? Mike Oldfield?! Jesus ...) The choice of particular tracks from these artists is a bit odd too - Southern Man and Like A Hurricane by Neil Young. But why not, say, For The Turnstiles or Cinnamon Girl by the same artist? Even if there is a system there, those choices seem so random as to be effectively misleading rather than useful.

Interestingly though, there's a huge batch of music for films (or cinematic sounding stuff), Scandinavian in origin. I haven't heard of most of these artists, and I'll check a couple out. I like a lot of Scandinavian stuff and I like soundtracks, so I can see the logic here. It seems rather too prevalent though, so I suspect the results are being skewed by the dataset that the system happens to have at this point. Generously assuming that isn't the case though, these are more interesting suggestions than most by Amazon.

However, it's still not so good that I'd use it again. It's not as good as my weekly Boomkat newsletter. It's not as good as the tips from the few music-orientated blogs I read. It's certainly not as good as the tips from those friends I really turn to for this stuff.

But anyway, here's the whole list, if you're interested.

Lives in the Balance
Jackson Browne
The Next Voice You Hear - The Best of Jackson Browne

Incantations - Part Four (Excerpt)
Mike Oldfield
Elements

From Hank To Hendrix
Neil Young
Unplugged 1993

Mr. Soul
Neil Young
Unplugged 1993

Tiny Tears
Tindersticks
Tindersticks [II]

Mistakes
Tindersticks
Tindersticks [II]

You Don't Have To Say
Anders Widmark
Anders Widmark Featuring Sara Isaksson

Ronja Rövardotter (Rid Över Skog)
Björn Isfält
Filmmusik

Mitt Liv Som Hund (Mot Småland)
Björn Isfält
Filmmusik

Hang Hang
Joshua Bell and Edgar Meyer
Short Trip Home

Moonlight Sonata
Fläskkvartetten
Pärlor Från Svin 2

9. Wen you dance i can really
Neil Young
After the Goldrush

4. Southern man
Neil Young
After the Goldrush

7. Don't let it bring You down
Neil Young
After the Goldrush

Vitabergspredikan
Stefan Sundström
Latlåtar 91-00

Tubular Bells (Opening Theme)
Mike Oldfield (opening theme)
Elements

She's Gone
Tindersticks
Tindersticks [II]

For Other Us
Brodsky Quartet & Elvis Costello
The Juliet Letters

Expert Rites
Brodsky Quartet & Elvis Costello
The Juliet Letters

I Almost Had A Weakness
Brodsky Quartet & Elvis Costello
The Juliet Letters

Who Do You Thing You Are?
Brodsky Quartet & Elvis Costello
The Juliet Letters

Etude
Mike Oldfield
Elements

Like A Hurricane
Neil Young
Unplugged 1993

Sleepy Song
Tindersticks
Tindersticks [II]

The fan
Erik Borelius
What is success

Manhandle
Erik Borelius
What is success

Resan Till Melonia (Ariel In Över Melonia)
Björn Isfält
Filmmusik

Gilbert Grape (Piano Tema)
Björn Isfält
Filmmusik

Synd
Fläskkvartetten
Pärlor Från Svin 2

Lamento
Fläskkvartetten
Pärlor Från Svin 2

Detta Blir Min Död
Peter LeMarc
Det Som Håller Oss Vid Liv

Teo
Fläskkvartetten
Pärlor Från Svin 2

Storm
Fläskkvartetten
Pärlor Från Svin 2

MacVolta
Fläskkvartetten
Pärlor Från Svin 2

Så Långt Mina Armar Räcker
Peter LeMarc
Det Som Håller Oss Vid Liv

Sanctus
Jan Garbarek
Jan Garbarek - Hilliard Ensemble - Officium

Du Är Bra
Peter LeMarc
Det Som Håller Oss Vid Liv

Matty drawing
Balanescu Quartet
Angels & Insects

His logic was bizarre
Erik Borelius
What is success

To love and be loved
Erik Borelius
What is success

Där Blått Möter Blått
Peter LeMarc
Det Som Håller Oss Vid Liv

Giliap (Lilla DJävulen)
Björn Isfält
Filmmusik

Alla frågar efter Alice
Eldkvarn
Brott lönar sig alltid

Änglagård, Andra Sommaren (Råttor)
Björn Isfält
Filmmusik

Giliap (Sommarens Sista Dans)
Björn Isfält
Filmmusik

Mil efter mil
Eldkvarn
Brott lönar sig alltid

Deliver Us
Brodsky Quartet & Elvis Costello
The Juliet Letters

Trubbel
Eldkvarn
Brott lönar sig alltid

01 Good to see You
Neil Young
Silver and Gold

World Keeps Going Around
Bill Withers
Live at Carnegie Hall

[Carl-Martin Schönning]

Do you know where I can buy or download something with Björn Isfält? The only thing I have found so far here in Sweden is the wonderful soundtrack to En kärlekshistoria (A swedish lovestory). I love his music but it seems impossible to find!

[Dan]

Trackbacks sent to this post at the time (before I turned trackbacks off due to spam):

» On iTunes and music recommendations and ratings... from plasticbag.org
At the moment there's a lot of discussion about music-related recommendations and ratings functionality roaming around webloggia. I wrote about MediaUnbound the other day (On MediaUnbound and Recommendations Engines), and I've linked to the (currently ... [Read More]

» The New Musical Functionality... from plasticbag.org
Over the last few months webloggia has been full of discussions about the new musical functionality that's starting to emerge around the web. I wasn't immune from this trend - I wrote about MediaUnbound (On MediaUnbound and Recommendations Engines) and... [Read More]