KDD Cup and Workshop 2007

Co-organized by ACM SIGKDD and Netflix

For KDD-2007, San Jose, California, Aug 12, 2007

Call-for-Participation

• All four of the top teams on the leaderboard (as of end-July, 2007) of the Netflix Prize competition will present their techniques.
• Winners of the KDD Cup competition will also present their algorithms and analyses.

Workshop Program

Workshop Proceedings

Talk slides of Netflix Prize papers

Talk slides of KDD Cup papers

---

Winners of KDD Cup 2007 and task answer files

• Task 1 - Who Rated What - Answer file
  • First place: Miklos Kurucz, Andras A. Benczur, Tamas Kiss, Istvan Nagy, Adrienn Szabo, Balazs Torma (Hungarian Academy of Sciences)
  • First runner up: Advanced Analytical Solutions Team of Neo Metrics directed by Jorge Sueiras (Neo Metrics)
  • Second runner up: Yan Liu, Zhenzhen Kou (IBM Research)
• Task 2 - How Many Ratings - Answer file
  • First place: Saharon Rosset, Claudia Perlich, Yan Liu (IBM Research)
  • First runner up: Advanced Analytical Solutions Team of Neo Metrics directed by Jorge Sueiras (Neo Metrics)
  • Second runner up: James Malaugh (Team Lead), Sachin Gangaputra, Nikhil Rastogi, Rahul Shankar, Sandeep Gupta, Kushagra Gupta, Neha Gupta, Gaurav Lal (Inductis)

Introduction

KDD Cup is the first and the oldest data mining competition, and is an integral part of the annual ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). This year's KDD Cup will be related to (but different from) the current Netflix Prize competition. There will also be a workshop at the KDD-2007 conference, where the participants of both the KDD Cup and the current Netflix Prize competition will present their papers and exchange ideas. We are looking forward to an interesting competition and your participation. We particularly encourage the participation of students.

There are 2 different parallel options for participating:

1. The KDD Cup competition (open to all)
2. Workshop paper submissions (open to Netflix prize participants only).

KDD Cup Competition Tasks and Rules

1. Task Descriptions

This year's tasks employ the Netflix Prize training data set. This data set consists of more than 100 million ratings from over 480 thousand randomly-chosen, anonymous customers on nearly 18 thousand movie titles. The data were collected between October, 1998 and December, 2005 and reflect the distribution of all ratings received by Netflix during this period. The ratings are on a scale from 1 to 5 (integral) stars. (See below for details on downloading this data set.)

This year's competition consists of two tasks. Each team can participate in the competition of any one task or both tasks.

1. Task 1 (Who Rated What in 2006): Your task is to predict which users rated which movies in 2006. We will provide a list of 100,000 (user_id, movie_id) pairs where the users and movies are drawn from the Netflix Prize training data set. None of the pairs were rated in the training set. Your task is to predict the probability that each pair was rated in 2006 (i.e., the probability that user_id rated movie_id in 2006). (The actual rating is irrelevant; we just want whether the movie was rated by that user sometime in 2006. The date in 2006 when the rating was given is also irrelevant.)
   
   
2. Task 2 (How Many Ratings in 2006): Your task is to predict the number of additional ratings the users from the Netflix Prize training dataset gave to a subset of the movies in the training dataset. We provide a list of 8863 movie_ids drawn from the Netflix Prize training dataset. You need to predict the number of additional ratings that all users in the Netflix Prize training dataset provided in 2006 for each of those movie titles. (Again the actual rating given by each user is irrelevant; we just want the number of times that the movie was rated in 2006. The date in 2006 when the rating was given is also irrelevant.)

1.1. Winner Selection

Evaluation: Winners will be determined, for both tasks, by computing the root mean squared error (RMSE) between your individual predictions and the correct answers. That is, if your prediction for an item is Y, the correct answer for the item is X and we have n items, RMSE = sqrt((sum(for all items (X-Y)^2))/n). Entry with the smallest RMSE will be judged the winner; in case of a tie, the entry with the earliest submission date will be judged the winner.

• In the case of "Who Rated What in 2006", the correct answer is 1 if the movie is rated by that user, 0 otherwise.
• In the case of "How Many Ratings in 2006", the correct answer is the actual number of ratings received. However, RMSE is computed slightly differently from the first task. Assuming that the actual number of ratings received is X, in computing RMSE, we use ln(1+X), where "ln" is natural logarithm. This also applies to your predicted number. That is, assuming that your predicted number is Y, we use ln(1+Y).

Note: We reserve the right to use a different evaluation criterion if no team can achieve the baseline result for a task. For example, for task one, the baseline result is one that assigns each pair the probability of the base rate, which is the proportion of movies rated in the test set (unknown to contestants).

Following the award of the KDD Cup prizes, the answer sets will be made available at the KDD Cup website and the Netflix Prize website.

2. Agreement

2.1. Eligibility

2.2. Integrity

2.4. Registration

You must register before June 1, 2007 in order to participate. By registering, we mean two things; (1) sign up as an user of the site, and (2) create a submission (could be a dummy submission) for each task that you will participate (by clicking on the "Submit Paper" button). All the submitted information can be updated later, but no new submission can be made after June 1, 2007. Registration does not imply any commitment to participation. After registration, the system will give you a unique identifier (called Paper ID) for the submitted KDD Cup task. The ID will be used when submitting the corresponding results file. Separate registration is required on the Netflix Prize web site to download the training data set. We will keep your registration data private.

3. Obtaining the Training Dataset and the Qualifying Answer Sets

• The who_rated_what_2006.txt file consists of 100,000 lines, each containing a user_id and movie_id pair.
• The how_many_ratings_2006.txt file consists of 8863 lines, each containing a movie_id.

The user_ids and movie_ids are taken from the Netflix Prize training dataset.

4. Submission of Results and Papers

4.1. Submission of Task Results

Each team can submit ONLY one results file for a task, but the submission can be updated as many times as you want before the deadline. No feedback will be provided upon submission. Only your last submission/update for each task before the deadline will be evaluated and all other submissions are overwritten. The submission site is now open (see above). Consider submitting early to avoid any last-minute congestion at the site.

There should be as many lines in each file as there are in the corresponding qualifying answer set file for each task. Each line must end with a line feed ("\n") or a carriage return immediately followed by a line feed ("\r\n"). After a results file has been uploaded, it overwrites the previous version.

The submission files should be named and formatted as follows. The file name should be of the form:

id-lastname-firstname-type.txt

id is the paper ID that you are assigned when you register for the KDD Cup competition. Lastname and firstname must be those of the contact person. type should be WRW for the "Who Rated What in 2006" task and HMR for the "How Many Ratings in 2006" task. For example, 280-Liu-Bing-WRW.txt.

Each line of a "Who Rated What in 2006" file should consist of a (floating point) number representing the probability that the corresponding user_id rated movie_id in 2006. Each line of a "How Many Ratings in 2006" file should consist of an integer or a floating point number representing the predicted number of ratings that the users in the Netflix Prize dataset provided for the corresponding movie in 2006. You can download this [Perl script] to check the format of your submission(s) (see comments in the file for usage).

Please follow the file format and file name format requirements. Results submitted with incorrect format risk being incorrectly evaluated or rejected. Each results file must be a text file (.txt). Do not zip or compress your file.

4.2. Submission of Papers

The copyright block of the first page must use the following:

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
KDDCup.07, August 12, 2007, San Jose, California, USA.
Copyright 2007 ACM 978-1-59593-834-3/07/0008.$5.00.

You can download a sample MS Word doc file from here, and a sample pdf file from here.

5. Frequently Asked Questions (FAQ)

---

Maintained by: Bing Liu

SEARCH:

Related searches:
netflix training answer dataset workshop

gipoco.com is neither affiliated with the authors of this page or responsible
for its contents. This is a safe-cache copy of the original web site.

gipoco.com is neither affiliated with the authors of this page nor responsible for its contents. This is a safe-cache copy of the original web site.