How Deep Is Your Video?