ICPR - HARL 2012

RESULTS

	28.09.2012 Results are available!
	01.09.2012 The submission deadline has passed, we are currently evaluating the submissions.
	15.7.2011 The test dataset has been published.
	02.12.2011 Kinect calibration details and correspondence software have been published.
	01.12.2011 The ground truth annotation has been published.

The goal of the HARL 2012 competition is recognition of complex human activities. In contrast to previous competitions and existing datasets, the proposed tasks focus on complex human behavior involving several people in the video at the same time, on actions involving several interacting people and on human-object interactions. The goal is not only to classify activities, but also to detect and to localize them. The dataset is shot with two different cameras: a moving camera mounted on a mobile robot delivering grayscale videos in VGA resolution and depth images from a consumer depth camera (Primesense/MS Kinect); and a consumer camcorder delivering color videos in DVD resolution.

The dataset used for the competition is the LIRIS human activities dataset, which consists of 10 classes. Each of classes can be a normal activity, a human-human interaction or a human-object interaction, or a combination of the latter two types:

More information on the dataset, file formats, and downloads can be found on the LIRIS human activities dataset site.

Competition schedule

A valid submission of results (due on Sep. 1st 2012) will contain:

• Detection results for all videos of the test set in the XML format described below.
• A command line executable which takes a video as an input argument and which produces a detection result in XML format as output. These executables will be used to check samples of the XML results submitted by the participants. Supported platforms are Windows, Linux and MacOS.

Rankings

Each participant may chose whether to submit results using:

• set D1 only (grayscale + depth)
• set D2 only (color)
• both sets D1+D2, which means that your algorithms can use grayscale data only, the least common denominator between D1 and D2

For each of these 3 possible settings, it is possible to return:

• full detection+recognition+localization results, i.e. an XML file in the same format as the ones we provide as groundtruth
• detection+recognition results only _without_ localization, i.e. information on which action is performed in which video how many times. In this case Precision and Recall figures will be established by simple check of action presence.

All combinations are possible, at the end of the competition we will publish 6 different rankings.

Evaluation & metrics

The goal of the competition is

• to detect relevant human behavior in midst of irrelevant additional motion, e.g. other people walking in the environment or performing irrelevant actions;
• to recognize the detected actions among the given action classes
• to localize the actions temporally and spatially

Different actions may happen in parallel in the same video at the same time. The ground truth data will therefore be annotated by marking labeled bounding boxes for each frame of each action (Details on the annotation can be found on the dataset page).

The ground truth annotation will be segmented into action occurrences regrouping all frames and bounding boxes of the same action. This makes it possible to provide more meaningful recall and precision values - indeed, a recall of 90% is easier to interpret if it tells us that 90% of the actions have been correctly detected, than if its says that, e.g. 90% of the action bounding boxes have been correctly detected on 100% of the activities.

Participants will report results in the same format - this means that the detection results need to be segmented in the same way: each detected action consists of a list of bounding boxes, where each bounding box corresponds to a frame. Each action must consist of consecutive frames, no holes are allowed in the sequence.

Detailed information on the evaluation metrics is found on a dedicated page on evaluation.

Registration and contact information

Registration is now closed. Questions on this competition can be sent to the following address :

Committee and contact information

HARL 2012 is organized by the following people: