Posts Tagged ‘movies’

Books and movies

Posted: 16 August 2009 in Uncategorized
Tags: , , , ,

This post contains NO spoilers.

I saw The Time Traveler’s Wife with my wife today.  I had read the book about a year ago, and had been looking forward to the movie.  I wasn’t disappointed — I thought the movie was very moving and captured the spirit of the book, even if it didn’t capture everything.  It ignored some dynamics that the book elaborated on and some scenes and details were slightly different.

One thing I was concerned about while watching the movie was just how much I was liking it because I knew all the background in the book, or how much came from the movie.  If the former was true, then the movie wasn’t going to be that great an experience for someone who had read it.  If the latter was true, then it was a damn good movie.  I don’t have the answer to that.

Another concern is how it’s a cultural norm in our society to bash movies based on books, and yet to relentlessly watch them to the point that Hollywood feels compelled to turn every book that sells a few copies into one.  Douglas Adams once made the point that he changed the story of the Hitchhiker’s Guide to the Galaxy to match the medium he was writing it for.  A story that plays well on the radio can take advantage of completely different things when it is translated to book or movie form.  I don’t have the exact quote and searching for that kind of thing is damn near impossible on Google (let me know if you find it).

But that’s an observation I have long taken to heart when watching movies translated from books.  Obviously you can’t fit an entire book into 2 hours and still have a story that tells like anything worth watching.  You can’t capture the full power of every scene, every nuance, nor every subtlety that a book can.  That’s not what the silver screen does well.  What it does well (when it is done right) is making you feel in touch with characters and the story.  Books do that too, but movies actually put the images before your eyes.

That said, I have never been able to bring myself to read a book based on a movie.  I just can’t do it.

R.I.P. Movie Rating Prediction

R.I.P. Movie Rating Prediction

There is no longer any reason to bother researching new ways of predicting the ratings users will give to movies.  It’s time to move on to more interesting things.  But seriously, given the fact that the last few miles of the Netflix competition were hard-fought by combining hundreds of different algorithms, is there much value in trying to improve recommender systems in this way, anymore?

I expect that the Netflix Prize data set, if left open to the public, will still be useful for a number of machine learning tasks where the goal is not necessarily improving recommender systems.  So predicting movie ratings may never be really dead.  But it is my hope that that as a goal for research will diminish and the focus will start moving towards other aspects of recommender systems still greatly lacking.  Like building systems that facilitate discovery of new items.

Factoring in the temporal dimension was a big deal in the latter part of the competition.  Sometimes you’re just in the mood for something gloomy.  Or something funny.  Or something ridiculous.  The same movie may totally turn you off a week later.  No machine (biological or mechanical) can predict these swings of emotions in the near future, so why bother?  Flip that around and let’s find ways of improving the search for items matching our mood at the time.

A system that interactively elicits your mood and guides you to matching items would be incredibly useful, don’t you think?

Image representing Netflix as depicted in Crun...
Image via CrunchBase

It looks like some of the top players in the Netflix Prize competition have teamed up and finally broke the 10% improvement barrier.  I know I’m a few days late on this, though not because I didn’t see when it happened.  I’ve been battling an ear infection all week and it has left me dizzy, in pain, and with no energy when I get home from work.  I hesitated before even posting anything about this, since there is little I can add at this point that hasn’t already been said. I’ll just share a few thoughts and experiences for posterity and leave it at that.  I’m also going to eventually make the point that recommender systems are operating under a false assumption, if you read this all the way through. :)

I competed for the prize for a bit, trying out a few ideas with support vector machines and maximum margin matrix factorization [pdf] that never panned out.  We were getting about a 4% improvement over Cinematch, which put us way down the list.  Going further would mean investing a lot of effort into implementing other algorithms, working out the ensemble, etc., unless we came up with some novel algorithm that bridged the gap.  That didn’t seem likely, so I stopped working on it just after leaving school.  I learned a lot about machine learning, matrix factorization, and scaling thanks to the competition, so it was hardly a net loss for me.

The one thing I regret is that the prize encouraged me and my advisor to spend more effort on the competition than we should have, which in turn meant we didn’t spend more time working on something tangibly productive for research.  Bluntly put, I think if we hadn’t wasted so much time on the competition, we could have worked on a different research problem more likely to produce a paper.  The lack of published research on my CV was the main reason I didn’t move on to get my PhD at CMU (at least, that’s what I was told by those close to the decision).  Hindsight is 20/20, and at the time, the shining glory of winning a million bucks and fame was delicious.  It also seemed like we had ideas that “maybe kinda sorta” were going somewhere.  That turned out to not be the case, but when admissions committees look at research experience, negative results = no results.

Many people have lauded the competition by saying that it has encouraged research in collaborative filtering and brought public attention to the field.  I was one of those people.  Others have criticized it for not focusing more on what people actually care about when using recommender systems — getting something useful and having a good experience!  And yes, Daniel Lemire, I’m thinking of you. :)  But I’m convinced that Daniel is right.  I remember reading in the literature that a 10% improvement is about what’s needed for someone to actually be able to notice a difference in recommender systems.  So maybe people will notice a slight improvement in the Netflix recommendations if these ideas are ever implemented.  Which is another problem — most of the stuff that led to winning the prize is so computationally expensive, it’s not really feasible for production.  Netflix recently released some improvements, and I didn’t notice a damned thing.  They still recommended me Daft Punk’s Electroma, which was a mind-numbing screen-turd.  And I must have seen every good sci-fi movie ever made, because there are no more recommendations for me in that category.  I have trouble believing that.

The point of a recommender system really shouldn’t be just to guess what I might happen to rate something at a given time.  The fact that introducing time makes such a big difference in improving performance in the competition seems like a ginormous red flag to me.  Sure I can look back in time and say “on day X, people liked movies about killing terrorists.”  The qualifying set in the competition asked you to predict the rating for a movie by a user on a given date in the past.  Remember what I said about hindsight being 20/20?  How about you predict what I will rate a movie this coming weekend.  See the problem?

I will sound the HCIR trumpets and say that what recommender systems should really be looking at is improving exploration.  When I go looking for a movie to a watch, or a pair of shoes to buy, I already know what I like in general.  Let me pick a starting point and then show me useful ways of narrowing down my search to the cool thing I really want.  Clerk dogs is a good first step on this path, though I think we’re going to have to move away from curated knowledge before this is going to catch fire.

Maybe I have this all wrong.  Maybe we need to discard the notion of recommender systems, since they are operating under the wrong premise.  We don’t need a machine to recommend something it thinks we’ll like.  We need a machine that will help us discover something we’ll like.  We need to be making discovery engines.  (Replace recommender system with search engine in most of what I just said and you’ll find that I have really been sounding the HCIR trumpets.)

Reblog this post [with Zemanta]

Movie idea #7210

Posted: 24 January 2009 in Uncategorized
Tags: , , , , , ,

The main character is a geeky computer programmer playing around with genetic algorithms.  He comes up with a representation that lets him evolve intelligent agents for market trading that become wildly successful.  He makes over a billion dollars in a very short period of time, like say, a week.  He shuts them down, in nervousness and excitement and faces a dilemma.

Do I destroy the agents and all record of their existence, my notes, etc, so that no one else can use this power for evil?


Do I open source these bad boys and bring down the whole damn system?

Of course, either decision will make him the target of criminal organizations, big businesses, and the American government (but I repeat myself).  Feel free to throw in chase scenes, a plucky old hacker who helps him disappear off the grid, and a grizzled retired policeman who saves his life.  Oh and some hot chick who never paid him any attention before he was a billionaire, but really cared about him all along and he was too shy to see it.

Film studios, feel free to write this up and make your millions.  Just don’t forget to use my name, since my blog is licensed under Creative Commons Attribution 3.0.

I happened on clerk dogs, a new movie recommender, the other day.  They are still in beta and are missing data in many key areas of film, but they are definitely worth checking out.  Like Pandora, clerk dogs uses human editors to classify movies along several dimensions.  Indeed, the founder Stuart Skorman (also founder of calls it the movie genome project.  Of course, another movie recommender (also, still in beta) is using that term.  Stuart goes on to say:

We have designed this innovative search engine for the movie buffs who have seen so many movies that they’re having a hard time finding new ones (or old ones) that they will really love. I hope you find hundreds of great movies!



This is a problem I’ve been noticing with Netflix lately.  I would be pretty sure I’ve seen every sci-fi movie worth seeing that has been released if all I had to go on was Netflix’s recommendations.  I gave clerk dogs a shot, starting with my favorite movie.  They seem to have done a decent job with classifying Brazil and a number of the similar movies they have listed are indeed similar in many ways to it.  When I first visited the site, they showed the similar movies on a grid and said whether it was “more dark”, “less disturbing”, “more violent”, and so on.  If that functionality still exists, I can’t find it.

However, you can “Mash it” to find movies that fit your mood.  Pick your base movie and mash it.  Then change the sliding scale to decide what sort of differences you are looking for.  Can you say kickass?

I applaud clerk dogs for a job well done.  I’ve already found a number of movies that Netflix was hiding from me.  I added them to my Netflix queue though so I guess they are still benefitting.

Top 48 Sci-Fi Adaptations

Posted: 26 August 2008 in Uncategorized
Tags: , , , ,

I just saw this meme on Pat’s Fantasy Hotlist and since he didn’t tag anyone, and I’ve never done this sort of thing before, I figured what the heck.  Just as he tagged no one, nor will I.  This is for my fleeting amusement and as an escape from the joys of moving.  The chain-letter-like aspect of “tagging” people is somewhat repulsive to me.

From Box Office Mojo’s list of Top 48 Sci-Fi Films Based on a Book (or Story) (1980- present). Some of the titles on this list look suspicious. (Was Cocoon really based on a piece of written fiction? There’s a difference between an adaptation and a novelization.)

Here are the rules.

– Copy the list below.
– Mark in bold the movie titles for which you read the book.
– Italicize the movie titles for which you started the book but didn’t finish it.
– Tag 5 people to perpetuate the meme. (You may of course play along anyway.)

And now, the list…

1. Jurassic Park
2. War of the Worlds
3. The Lost World: Jurassic Park
4. I, Robot
5. Contact
6. Congo
7. Cocoon
8. The Stepford Wives
9. The Time Machine
10. Starship Troopers
11. The Hitchhiker’s Guide to the Galaxy
12. K-PAX
13. 2010
14. The Running Man
15. Sphere
16. The Mothman Prophecies
17. Dreamcatcher
18. Blade Runner(Do Androids Dream of Electric Sheep?)
19. Dune
20. The Island of Dr. Moreau
21. Invasion of the Body Snatchers
22. The Iron Giant(The Iron Man)
23. Battlefield Earth
24. The Incredible Shrinking Woman
25. Fire in the Sky
26. Altered States
27. Timeline
28. The Postman
29. Freejack(Immortality, Inc.)
30. Solaris
31. Memoirs of an Invisible Man
32. The Thing(Who Goes There?)
33. The Thirteenth Floor
34. Lifeforce(Space Vampires)
35. Deadly Friend
36. The Puppet Masters
37. 1984
38. A Scanner Darkly
39. Creator
40. Monkey Shines
41. Solo(Weapon)
42. The Handmaid’s Tale
43. Communion
44. Carnosaur
45. From Beyond
46. Nightflyers
47. Watchers
48. Body Snatchers

I’ve read most of the books on that list that I’d consider worth reading.  Any recommendations for the ones I haven’t?

So I spent the last weekend in Boston.  I stayed at the very enjoyable Inn at Harvard.  I was reminded several times by the women staying at the hotel of the female players in the public radio game show Says You! That was show actually one of my early influences in choosing the field of computational linguistics.  At least it was the first place I heard the term.  On my job search, I have been asked the question many times:  “Why NLP/computational linguistics?”  I try to give a short version, because I don’t want to bore anyway and I tend to ramble when my stories get too long.  So here is the dirt.

I was going to school for my BS in computer science and one of the requirements was a social science elective.  I thought anthropology sounded cool, or at least cooler than the sociology and psychology offerings.  In the course of the class we hit on the topic of anthropological linguistics and the Sapir-Whorf hypothesis.  In a very succinct and incomplete nutshell, the SW hypothesis states that language shapes the sorts of things we normally think about.  Not necessarily preventing us from thinking about certain things, but just changing the things that we devote the most mental resources to.  I thought this was a really cool idea and decided to take some linguistics classes.  After several such classes, I knew I was in love with linguistics but didn’t see a way to actually support a family on it.  And by family, I mean two anorexic chipmunks through a short winter.

It was already becoming clear to me in my linguistics class that there was a lot of this stuff you could handle with a computer program.  The brain is just a different sort of hardware, one possibly hardwired for language use.  I figured there had to be a combination of computer science and linguistics.  Right around the same time I was having that thought, I heard a letter from a computational linguist being read on Says You! (or was it a caller?).  In any case, it was like a bomb went off in my head and I knew what I wanted to do.  I started looking for grad schools, the best I got into was CMU, and the rest is history.

I didn’t go to any breweries in Boston, unfortunately, aside from the John Harvard’s Brew House in Harvard Square.  That was decent, but nothing to write home about.  Oddly, though, I did write home about it, texting and tweeting friends, my sisters, wife, and mother with pictures of the beer.  Sitting alone in a bar in a strange city, what can I say.  I had an offer for a tour earlier in the day, but there was the threat of rain and I fell asleep for many hours, screwing that opportunity up.  The bed in that hotel had a soporific effect, which I guess should make some bedmaker somewhere happy.  I also saw The Dark Knight and Wanted.  The former was infrickincredible.  I was blown away by the nonstop goodness of this movie.  I got to see Wanted for free because they screwed up the beginning of TDK and the movie was very very slightly tilted the whole time.  After a short while it wasn’t very noticeable to me.  That didn’t stop me from taking the free ticket.  Wanted was ok-ish.  What I wanted was a less predictable storyline.  The action was great, and even though it was fantastical (which I have no problem with), I once or twice couldn’t suspend disbelief.

And now for some pictures.


Posted: 20 June 2008 in Uncategorized
Tags: , , , ,

This post contains no spoilers.

I rewatched Primer this week. I had seen it a couple years ago as one of the first movies I got from Netflix the first time I signed up. It was a successful recommendation. Since I was a kid, I have been totally intrigued with time travel and time travel movies. Time travel movies rank among my favorite films, like 12 Monkeys, Time Bandits, the Butterfly Effect, etc. Time travel books are great too, like The Time Traveler’s Wife. Thinking about the implications of being able to change things — and what happens when you do — filled many teenage hours.  An important part of my fascination then is resolving the conflicts inherent in time travel.  What happens if you change something in the past?  What are the rules in the movie or book?  Does the movie/book adhere to its own rules or do they screw up?

Primer is a time travel movie in a league of its own.  I think it’s pretty much impossible to fully grasp the first time through.  It is probably the most confusing movie I have ever seen (that is not “absurd” anyway).  It’s been bumping around in my head for the past couple years, driving me to see it again.  Mike D’Angelo in Esquire said it’s like “following the path of one blade on a high-speed ceiling fan.”  That’s a fairly accurate description.


This post is spoiler free.

I finally got to see Juno tonight. It’s been sitting at the top of my Netflix queue for nearly two months with a long wait. What a great movie! One of my favorite parts was the soundtrack. There were several great songs by Kimya Dawson (of the Moldy Peaches) and then a performance by the two leads of the Moldy Peaches song “Anyone Else But You.” The version sung in the movie is missing a few stanzas. My favorite of the missing ones is below (sung by Kimya):

“Up up down down left right left right B A start
Just because we use cheats
Doesn’t mean we’re not smart
I don’t see what anyone can see in anyone else
But you…”

Go geek references (and Thundercats)! And speaking of cheats, trying using that cheat code in Google Reader (minus the start button at the end of course).

And returning to Netflix: they are removing individual profiles from accounts as of September 1st. What a boneheaded, retardafreakin’ idea. Supposedly it will help them make the website better. I hope it’s a lot better since this change has me pissed.

Is that all there is?

Posted: 19 May 2008 in Uncategorized
Tags: , , , , , ,

My taste in music is definitely in flux.  Five years ago I would have found this intolerable, but now I can’t stop listening to it.  I blame Pandora.  The musical journeys it takes you on can be transformational.

Unfortunately the video stops before the song is over, but YouTube offers several full length suggestions immediately after.  The videos themselves are all insane, so I didn’t want to endorse any.  I just listen to the sound track in another tab and don’t watch them.

This question was a central theme in the movie The Nines, which I recommend.  It also came up in Revolver, which I just watched tonight, though it wasn’t asked explicitly.  Instead, the question is who is your worst enemy?  The movie’s position is that it is not external, but internal.  I think I can say that without spoiling anything.  The trick is to avoid the lie that your perception is infallible.  Pulling that off is a different matter altogether, though it is a helpful trait for a good scientist.