@inproceedings{ Wisspeintner:Zant:Iocchi:Schiffer:2009:RoboCup:Home,
  author      = "Thomas Wisspeintner and Tijn van der Zant 
                 and Luca Iocchi and Stefan Schiffer",
  title       = "RoboCup@Home: Results in Benchmarking Domestic Service Robots",
  booktitle   = "Proceedings of the International RoboCup Symposium 2009 (RoboCup 2009)",
  pages       = "390--401",
  year        = "2009",
  month       = "June 30 -- July 3",
  location    = "Graz, Austria",
  publisher   = "Springer",
  series      = "LNCS",
  volume      = "5949",
  number      = "",
  isbn        = "978-3-642-11875-3",
  ISSN        = "0302-9743 (Print) 1611-3349 (Online)",
  doi         = "10.1007/978-3-642-11876-0_34",
  note        = "",
  abstract    = " Benchmarking robotic technologies is of utmost
                  importance for actual deployment of robotic
                  applications in industrial and every-day
                  environments, therefore many efforts have recently
                  focused on this problem. Among the many different
                  ways of benchmarking robotic systems, scientific
                  competitions are recognized as one of the most
                  effective ways of rapid development of scientific
                  progress in a field.  The RoboCup@Home league
                  targets the development and deployment of autonomous
                  service and assistive robot technology, being
                  essential for future personal domestic applications,
                  and offers an important approach to benchmarking
                  domestic and service robots.  In this paper we
                  present the new methodology for benchmarking DSR
                  adopted in RoboCup@Home, that includes the
                  definition of multiple benchmarks (tests) and of
                  performance metrics based on the relationships
                  between key abilities required to the robots and the
                  tests. We also discuss the results of our
                  benchmarking approach over the past years and
                  provide an outlook on short- and mid-term goals of
                  @Home and of DSR in general.",
}