A Case Study of Trust Issues in Scientific Video Collections

In-situ video recording of underwater ecosystems is able to provide valuable information for biology research and natural resources management, e.g. changes in species abundance. Searching the videos manually, however, requires costly human effort. Our video analysis tool supports the key task of counting different species of fish, allowing marine biologists to query the video collection without watching the videos. To be suitable for scientific research on changes in species abundance, the video data must include data provenance information that reflects the potential biases introduced through the video processing.In order to trust the analyses made by the system, we need to provide expert users with sufficient information to allow them to interpret these potential biases. We conducted two user studies to design a user interface that includes data provenance information. Our qualitative analysis discusses the support for understanding the reliability of video analysis, and trusting the results it produces. Our main finding is that disclosing details about the video processing and provenance data allows biologists to compare the results with their traditional statistical methods, thus increasing their trust in the results.