Skip to main content

Advertisement

SpringerLink
Log in

A rule-based event detection system for real-life underwater domain

  • Special Issue Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Understanding and analyzing fish behaviour is a fundamental task for biologists that study marine ecosystems because the changes in animal behaviour reflect environmental conditions such as pollution and climate change. To support investigators in addressing these complex questions, underwater cameras have been recently used. They can continuously monitor marine life while having almost no influence on the environment under observation, which is not the case with observations made by divers for instance. However, the huge quantity of recorded data make the manual video analysis practically impossible. Thus machine vision approaches are needed to distill the information to be investigated. In this paper, we propose an automatic event detection system able to identify solitary and pairing behaviours of the most common fish species of the Taiwanese coral reef. More specifically, the proposed system employs robust low-level processing modules for fish detection, tracking and recognition that extract the raw data used in the event detection process. Then each fish trajectory is modeled and classified using hidden Markov models. The events of interest are detected by integrating end-user rules, specified through an ad hoc user interface, and the analysis of fish trajectories. The system was tested on 499 events of interest, divided into solitary and pairing events for each fish species. It achieved an average accuracy of 0.105, expressed in terms of normalized detection cost. The obtained results are promising, especially given the difficulties occurring in underwater environments. And moreover, it allows marine biologists to speed up the behaviour analysis process, and to reliably carry on their investigations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. http://ecogrid.nchc.org.tw/.

  2. www.fish4knowledge.eu.

  3. http://www.fishbase.org/summary/Dascyllus-reticulatus.html.

  4. http://homepages.inf.ed.ac.uk/rbf/Fish4Knowledge/DELIVERABLES/Del21.pdf.

  5. http://fishbase.org.

References

  1. Gkalelis, N., Mezaris, V., Kompatsiaris, I.: High-level event detection in video exploiting discriminant concepts. In: 9th International Workshop on Content-Based Multimedia Indexing (CBMI 2011). Madrid, Spain, 06/2011 (2011)

  2. Liao, M.-Y., Chen, D.-Y., Sua, C.-W., Tyan, H.-R.: Real-time event detection and its application to surveillance systems. In: Proceedings of 2006 IEEE International Symposium on Circuits and Systems, 2006. ISCAS 2006, vol. 4, p. 512 (2006)

  3. Ballan, L., Bertini, M., Bimbo, A.D., Seidenari, L., Serra, G.: Event detection and recognition for semantic annotation of video. Multimedia Tools Appl. 51, 279–302 (2011)

    Article  Google Scholar 

  4. Spampinato, C., Chen-Burger, Y.-H., Nadarajan, G., Fisher, R.: Detecting, tracking and counting fish in low quality unconstrained underwater videos. In: Proceedings of 3rd International Conference on Computer Vision Theory and Applications (VISAPP), vol. 2, pp. 514–519 (2008)

  5. Spampinato, C., Giordano, D., Di Salvo, R., Chen-Burger, Y.-H.J., Fisher, R.B., Nadarajan, G.: Automatic fish classification for underwater species behavior understanding. In: Proceedings of the first ACM international workshop on analysis and retrieval of tracked events and motion in imagery streams, pp. 45–50. ARTEMIS ’10, ACM, New York, NY, USA (2010)

  6. Spampinato, C., Palazzo, S., Giordano, D., Kavasidis, I., Lin, F.-P., Lin, Y.-T.: Covariance based fish tracking in real-life underwater environment. In: VISAPP (2), pp. 409–414 (2012)

  7. Rijnsdorp, A.D., Peck, M.A., Engelhard, G.H., Mšllmann, C., Pinnegar, J.K.: Resolving the effect of climate change on fish populations. ICES Journal of Marine Science: Journal du Conseil 66(7), 1570–1583 (2009)

    Article  Google Scholar 

  8. Scott, G.R., Sloman, K.A.: The effects of environmental pollutants on complex fish behaviour: integrating behavioural and physiological indicators of toxicity. Aquatic Toxicol 68(4), 369–392 (2004)

    Article  Google Scholar 

  9. Spampinato, C., Palazzo, S., Boom, B., van Ossenbruggen, J., Kavasidis, I., Di Salvo, R., Lin, F.-P., Giordano, D., Hardman, L., Fisher, R.: Understanding fish behavior during typhoon events in real-life underwater environments. Multimedia Tools Appl. pp. 1–38 (2012). doi:10.1007/s11042-012-1101-5

  10. Cupillard, F., Avanzi, A., Bremond, F., Thonnat, M.: Video understanding for metro surveillance. In: IEEE International Conference on Networking Sensing and Control, vol. 1, pp. 186–191, IEEE (2004)

  11. Ke, Y., Sukthankar, R., Hebert, M.: Event detection in crowded videos. In: IEEE 11th International Conference on Computer Vision, vol. 23, pp. 1–8 (2007)

  12. Zhang, Z., Huang, K., Tan, T., Wang, L.: Trajectory series analysis based event rule induction for visual surveillance. In: IEEE Conference on Computer Vision and, Pattern Recognition, pp. 1–8 (2007)

  13. Haering, N., Qian, R.J., Sezan, M,I.: A semantic event-detection approach and its application to detecting hunts in wildlife video (2000)

  14. Liao, M.-Y., Chen, D.-Y., Sua, C.-W., Tyan, H.-R.: Real-time event detection and its application to surveillance systems. In: Proceedings of the IEEE International Symposium on Circuits and Systems (2006)

  15. Li, B., Ibrahim Sezan, M.: Event detection and summarization in sports video. In: Proceedings IEEE Workshop on Content-Based Access of Image and Video Libraries CBAIVL 2001, pp. 132–138 (2001)

  16. Sadlier, D.A., O’Connor, N.E.: Event detection in field sports video using audio-visual features and a support vector Machine (2005)

  17. Medioni, G., Cohen, I., Bremond, F., Hongeng, S., Nevatia, R.: Event detection and analysis from video streams. IEEE Trans. Pattern Anal. Mach. Intell. 23(8), 873–889 (2001)

    Article  Google Scholar 

  18. Assfalg, J., Bertini, M., Colombo, C., Bimbo, A.D., Nunziati, W.: Highlight extraction in soccer videos (2003)

  19. Suzuki, N., Hirasawa, K., Tanaka, K., Kobayashi, Y., Sato, Y., Fujino, Y.: Learning motion patterns and anomaly detection by Human trajectory analysis. In: IEEE International Conference on Systems, Man and, Cybernetics, pp. 498–503 (2007)

  20. Porikli, F., Haga, T.: Event detection by eigenvector decomposition using object and frame features. In: Conference on Computer Vision and Pattern Recognition, Workshop (2004)

  21. Huang, C.-L., Shih, H.-C., Chao, C.-Y.: Semantic analysis of soccer video using dynamic Bayesian network (2006)

  22. Piciarelli, C., Foresti, G.L., Snidaro, L.: Trajectory clustering and its applications for video surveillance. In: IEEE Conference on Advanced Video and Signal Based Surveillance (2005)

  23. Zhan, B., Monekosso, D.N., Remagnino, P., Velastin, S.A., Xu, L.-Q.: Crowd analysis: a survey. Mach. Vision Appl. 19(5–6), 345–357 (2008)

    Google Scholar 

  24. Andrade, E.L., Blunsden, S., Fisher, R.B.: Modelling crowd scenes for event detection. In: 18th International Conference on Pattern Recognition, vol. 1, pp. 175–178 (2006)

  25. Soori, U., Arshad, M.: Underwater crowd flow detection using Lagrangian dynamics. Indian J. Marine Sci. 38, 359–364 (2009)

    Google Scholar 

  26. Meila, M., Shi, J.: A random walks view of spectral segmentation. In: AISTATS, pp. 8–11. AISTATS (2001)

  27. Rissanen, J.: Stochastic Complexity in Statistical Inquiry, Series in Computer Science, vol. 15. World Scientific, Singapore (1989)

    Google Scholar 

  28. Wang, F., Jiang, Y.-G., Ngo, C.-W.: Video event detection using motion relativity and visual relatedness. In: Proceedings of ACM multimedia (2008)

  29. Branson, K., Robie, A.A., Bender, J., Perona, P., Dickinson, M.H.: High-throughput ethomics in large groups of Drosophila. Nat. Methods 6, 451–457 (2009)

    Article  Google Scholar 

  30. Palmer, T., Tamte, M., Halje, P., Enqvist, O., Petersson, P.: A system for automated tracking of motor components in neurophysiological research. J. Neurosci. Methods 205, 334–344 (2012)

    Article  Google Scholar 

  31. Poppe, R.: A survey on vision-based human action recognition. Image Vision Comput. 28, 976–990 (2010)

    Article  Google Scholar 

  32. Burgos-Artizzu, X., Dollár, P., Lin, D., Anderson, D., Perona, P.: Social behavior recognition in continuous videos. In: CVPR (2012)

  33. Haritaoglu, I., Harwood, D., Davis, L.S.: W4: Who? when? where? what? a real time system for detecting and tracking people. In: Proceedings of the International Conference on Automatic Face and Gesture Recognition, vol. 1, (Nara, Japan), pp. 222–227 (2008)

  34. Faro, A., Giordano, D., Spampinato, C.: Adaptive background modeling integrated with luminosity sensors and occlusion processing for reliable vehicle detection. IEEE Trans. Intell. Transportation Syst. 12, 1398–1412 (2011)

    Article  Google Scholar 

  35. Porikli, F.: Achieving real-time object detection and tracking under extreme conditions. J. Real-Time Image Process. 1(1), 33–40 (2006)

    Article  Google Scholar 

  36. Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: Proceedings 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Cat No PR00149, 2(c), 246–252 (1999)

  37. Faro, A., Giordano, D., Spampinato, C.: Integrating location tracking, traffic monitoring and semantics in a layered its architecture. IET Intell. Transport Syst. 5(3), 197–206 (2011)

    Article  Google Scholar 

  38. Porikli, F., Wren, C.: Change detection by frequency decomposition: Wave-back. In: Proceedings of Workshop on Image Analysis for Multimedia Interactive Services (2005)

  39. Porikli, F.: Multiplicative background-foreground estimation under uncontrolled illumination using intrinsic images. In: Proceedings of IEEE Motion Multi-Workshop (2005)

  40. Barnich, O., Van Droogenbroeck, M.: ViBe: a universal background subtraction algorithm for video sequences. IEEE Trans. Image Process. 20, 1709–1724 (2011)

    Article  MathSciNet  Google Scholar 

  41. Porikli, F.: Change detection by frequency decomposition: Wave-back. In: Proceedings of Workshop on Image Analysis for Multimedia Interactive Services (2005)

  42. Schapire, R.E., Singer, Y.: Improved boosting algorithms using confidence-rated predictions. Mach. Learn. 37, 297–336 (1999)

    Article  MATH  Google Scholar 

  43. Kim, K., Chalidabhongse, T., Harwood, D., Davis, L.: Background modeling and subtraction by codebook construction. In: International Conference on Image Processing, 2004. ICIP ’04. 2004, vol. 5, pp. 3061–3064 (2004)

  44. Spampinato, C., Palazzo, S.: Enhancing object detection performance by integrating motion objectness and perceptual organization. In: Proceedings of IEEE International Conference on, Pattern Recognition, pp. 3640–3643 (2012)

  45. Alexe, B., Deselaers, T., Ferrari, V.: Measuring the objectness of image windows. In: IEEE Transactions on PAMI, vol. 99, PrePrints (2012)

  46. Cheng, C., Koschan, A., Chen, C.-H., Page, D.L., Abidi, M.A.: Outdoor scene image segmentation based on background recognition and perceptual organization. IEEE Trans. Image Process. 21(3), 1007–1019 (2012)

    Article  MathSciNet  Google Scholar 

  47. Rother, C., Kolmogorov, V., Blake, A.: GrabCut: interactive foreground extraction using iterated graph cuts. ACM Trans. Graphics (TOG), pp. 309–314 (2004)

  48. He, X.C., Yung, N.H.C.: Curvature scale space corner detector with adaptive threshold and dynamic region of support. In: International Conference on Pattern Recognition, vol. 2, pp. 791–794. IEEE Computer Society, Los Alamitos, CA, USA (2004)

  49. Mokhtarian, F., Suomela, R.: Robust image corner detection through curvature scale space. IEEE Trans. Pattern Anal. Mach. Intell. 20(12), 1376–1381 (1998)

    Article  Google Scholar 

  50. Spampinato, C., Giordano, D., Salvo, R.D., Chen-Burger, Y.H., Fisher, R.B., Nadarajan, G.: Automatic fish classification for underwater species behavior understanding. In: Proceedings of the first ACM international workshop on analysis and retrieval of tracked events and motion in imagery streams, New York, NY, USA, pp. 45–50 (2010)

  51. Chih-Chung, C., Chih-Jen, L.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 1–27 (2011)

    Article  Google Scholar 

  52. Porikli, F., Tuzel, O., Meer, P.: Covariance tracking using model update based on lie algebra. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (2005)

  53. Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77(2), 257–286 (1989)

    Article  Google Scholar 

  54. Kavasidis, I., Palazzo, S., Di Salvo, R., Giordano, D., Spampinato, C.: A semi-automatic tool for detection and tracking ground truth generation in videos. In: VIGTA ’12: Proceedings of the 1st International Workshop on Visual Interfaces for Ground Truth Collection in Computer Vision Applications, pp. 1–5, ACM (2012)

  55. Isard, M., Blake, A.: Condensation–conditional density propagation for visual tracking. Int. J. Comput. Vision 29(1), 5–28 (1998)

    Article  Google Scholar 

  56. Lazarevic-McManus, N., Renno, J., Jones, G.A.: Performance evaluation in visual surveillance using the f-measure. In: Proceedings of the 4th ACM international workshop on Video surveillance and sensor networks, VSSN ’06, pp. 45–52, ACM, New York, NY, USA (2006)

Download references

Author information

Authors and Affiliations

  1. Department of Electrical, Electronics and Computer Engineering, University of Catania, Catania, Italy

    Concetto Spampinato & Simone Palazzo

  2. Centrum voor Wiskunde en Informatica (CWI), Amsterdam, Netherlands

    Emmanuelle Beauxis-Aussalet, Jacco van Ossenbruggen & Jiyin He

  3. School of Informatics, University of Edinburgh, Edinburgh, UK

    Cigdem Beyan, Bas Boom & Xuan Huang

Authors
  1. Concetto Spampinato
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Emmanuelle Beauxis-Aussalet
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Simone Palazzo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Cigdem Beyan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jacco van Ossenbruggen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jiyin He
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Bas Boom
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Xuan Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Concetto Spampinato.

Additional information

This research was funded by European Commission FP7 grant 257024, in the Fish4Knowledge project (www.fish4knowledge.eu).

Rights and permissions

Reprints and permissions

About this article

Cite this article

Spampinato, C., Beauxis-Aussalet, E., Palazzo, S. et al. A rule-based event detection system for real-life underwater domain. Machine Vision and Applications 25, 99–117 (2014). https://doi.org/10.1007/s00138-013-0509-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-013-0509-x

Keywords

Search

Navigation