ITOP Dataset

Summary

The ITOP dataset (Invariant Top View) contains 100K depth images from side and top views of a person in a scene. For each image, the location of 15 human body parts are labeled with 3-dimensional (x,y,z) coordinates, relative to the sensor's position. Read the full paper for more context [pdf].

Getting Started

Download then decompress the h5.gz file.

gunzip ITOP_side_test_depth_map.h5.gz

Using Python and h5py (pip install h5py or conda install h5py), we can load the contents:

import h5py
import numpy as np

f = h5py.File('ITOP_side_test_depth_map.h5', 'r')
data, ids = f.get('data'), f.get('id')
data, ids = np.asarray(data), np.asarray(ids)

print(data.shape, ids.shape)
# (10501, 240, 320) (10501,)

Note: For any of the *_images.h5.gz files, the underlying file is a tar file and not a h5 file. Please rename the file extension from h5.gz to tar.gz before opening. The following commands will work:

mv ITOP_side_test_images.h5.gz ITOP_side_test_images.tar.gz
tar xf ITOP_side_test_images.tar.gz

Metadata

File sizes for images, depth maps, point clouds, and labels refer to the uncompressed size.

+-------+--------+---------+---------+----------+------------+--------------+---------+
| View | Split | Frames | People | Images | Depth Map | Point Cloud | Labels |
+-------+--------+---------+---------+----------+------------+--------------+---------+
| Side | Train | 39,795 | 16 | 1.1 GiB | 5.7 GiB | 18 GiB | 2.9 GiB |
| Side | Test | 10,501 | 4 | 276 MiB | 1.6 GiB | 4.6 GiB | 771 MiB |
| Top | Train | 39,795 | 16 | 974 MiB | 5.7 GiB | 18 GiB | 2.9 GiB |
| Top | Test | 10,501 | 4 | 261 MiB | 1.6 GiB | 4.6 GiB | 771 MiB |
+-------+--------+---------+---------+----------+------------+--------------+---------+

Data Schema

Each file contains several HDF5 datasets at the root level. Dimensions, attributes, and data types are listed below. The key refers to the (HDF5) dataset name. Let (n) denote the number of images.

Transformation

To convert from point clouds to a (240 times 320) image, the following transformations were used. Let (x_{textrmimg}) and (y_{textrmimg}) denote the ((x,y)) coordinate in the image plane. Using the raw point cloud ((x,y,z)) real world coo