The Evaluation Competition seeks to motivate the participation of undergraduate students, graduate students and teachers in HCI, contributing to the training of these students. The Evaluation Competition has an essentially practical nature: participants make the evaluation of a computer system and thus apply their theoretical knowledge related to HCI evaluation methods.

Competition Theme

The theme of IHC 2018 is 'Interaction, Culture and Creativity'. Nowadays, with the extensive use of information technology, it is clear its cultural impact by changing behaviors, providing new forms of interaction, and also being an instrument which helps designers to you to give free rein to their creativity, in order to create applications that seek to get people's lives better.

In this year's Evaluation Competition, we will discuss the use of technology within the context of health area. Such area is an exponent in HCI projects to improve the quality of life of people, continuing medical education, and access to information by patients and users, in the medical, pharmaceutical, nutrition and other fields. Thus, the 2018 Evaluation Competition is focused on examining how technology can support people in daily care with their health. There are a number of systems and applications with this proposal, as those that monitor people's diet, sleep and physical activity; those that remind people to drink water and take medicine; and those that support caregivers for the elderly, among others. Finally, designers do not skimp on their creativity when developing solutions to help people adopt a healthy routine at different stages of their life.

Here are some links to get inspired:

• http://revistamelhor.com.br/10-novos-apps-que-vao-melhorar-sua-saude/
• https://play.google.com/store/apps/category/HEALTH_AND_FITNESS?hl=pt_BR
• https://veja.abril.com.br/saude/os-melhores-aplicativos-para-cuidar-da-saude/
• http://infograficos.estadao.com.br/focas/planeje-sua-vida/os-apps-que-facilitam-a-vida-dos-idosos

Evaluation

Object and focus

Evaluate the quality of use (usability, communicability, accessibility etc) or the user experience of an application that supports people in daily care with their health. Applications to be evaluated may be of individual use (single user) or collaborative. Proposals with evaluation of apps that enhance social collaboration in field of health are also welcome.

Methods

The teams are free to choose the methods they will use to evaluate the system.

Product

After the evaluation, each team must generate a report containing:

• Name(s) of employed evaluation method(s);
• Justification of the choice of method(s);
• Description of the chosen platform(s): brand, model, operating system;
• Description of the evaluation process (part of the interface under consideration, evaluated features, procedures involved);
• Evaluation results:
• Founded problems: description, location and context of occurrence with justification;
• Closing remarks: conclusion and other observations
• Team analysis of how the employed method(s) were able (or not) to identify aspects related to the experience of use of the application(s).

Submissions

Submissions of reports should be anonymous and have up to 10 pages following the template for full papers of ACM SIGCHI (2015) (https://sigchi.org/templates/). Authors should submit a PDF version of their paper through the   JEMS system, in PDF format.

Reports will be kept confidential during the review process and reports of the finalists will be kept confidential until the start of the event, when, during the presentations, the teams will be identified.

Team Requirements

Two kinds of teams are allowed to participate in the competition:

• Undergraduate students: 3 to 5 students.
• Graduate students: 2 to 4 students.

All teams need to have exactly one professor-advisor, associated to a higher education institution. Mixed teams, i.e. teams comprised of undergraduate and graduate students, are not allowed.

Not following these requirements results in automatic exclusion from the selection process.

Instructions for Participation

To participate in the Competition, each team should follow three steps:

1. Run the evaluation;
2. Submit the evaluation report;
3. If chosen, present the evaluation report during the Evaluation Competition session at IHC 2018, in Belém - PA.

Important Dates

• Deadline for submission of reports in JEMS: July 13, 2018 July 06, 2018
• Notification of finalists for presentation at IHC 2018: August 24, 2018 August 17, 2018
• Deadline for submission of final reports in JEMS: September 10, 2018 September 06, 2018
• Presentation at IHC 2018: October 22-26, 2018

Evaluation of submitted reports

Each report will be evaluated by reviewers with experience in HCI or evaluation of interactive systems. The following criteria will be considered:

• Adequacy of the system/application to the Evaluation Competition theme;
• Readability, organization and presentation of the evaluation report;
• Clear definition of the evaluation scope and purpose;
• Adequacy of the chosen method(s) and evaluation procedure(s);
• Quality of results considering the established scope and purpose;
• Consideration of ethical issues involved in conducting the evaluation (in the case of using methods involving users or people outside the team);
• Quality of the critical analysis of the ability of the method to disclose or not potential problems of use of the system;

Selection and presentation at IHC 2016

3 finalists of each category (undergraduate and graduate) will be selected for a short oral presentation, followed by questions from a group of evaluators, during the event in Belém, Brazil, between October 22 and 26 2018. The presence of at least one member of each finalist team is required.

Awards

Each finalist team who presents their work at the symposium will receive a Certificate of Recognition. The winning team will also receive a prize (to be defined by the Organization).

Support of the participation of finalists

The organization will provide a free registration for one member of each team selected for oral presentation.

Publication

The reports of the finalist teams will be published as the extended abstracts of the IHC 2018 proceedings.

Summary

• System to be evaluated: applications that support people in daily care with their health;
• Criteria to be evaluated: Quality of use or User experience;
• Methods to be used: selected by the teams;
• Team size: 1 professor-advisor + 3 to 5 undergraduate students OR 1 professor-advisor + 2 to 4 graduate students;

Organizers

• Carla Spinillo (UFPR) - cgspin@gmail.com
• Maria Lúcia Bento Villela (UFVJM) - smvillela@gmail.com