Proceedings of the Institute of Acoustics

 

Automatic identification of biophonics and sea ice processes in large datasets from the High Arctic Ocean

 

J Cleverly, Department of Physics, University of Bath, Bath, Somerset, UK
Ph Blondel, Department of Physics, University of Bath, Bath, Somerset, UK
H Sagen, Acoustic and Oceanography Group, Nansen Environmental and Remote Sensing Center, Bergen, Vestland, Norway
M Dzieciuch, Scripps Institution of Oceanography, University of California San Diego, San Diego, California, USA
E Storheim, Acoustic and Oceanography Group, Nansen Environmental and Remote Sensing Center, Bergen, Vestland, Norway

 

 

1 INTRODUCTION

 

Accelerated climate change in the Arctic region is leading to major modifications to its underwater soundscapes 1. With the reduction of sea ice coverage across seasons, marine mammals are having to adapt their migratory behaviours^2,3 and waters are opening up for maritime industries^4,5, phenomena which can be monitored acoustically. Advancements in offshore technology for the challenging Arctic conditions have allowed for multi-sensor moorings to run continuously across seasons, in particular with hydrophones for acoustic thermometry and to better understand local soundscape variations. However, the accumulation of data requires automation in processing to identify times featuring processes of interest.

 

The terrestrial ecoacoustics community is facing the same issue of ”big data” 6, but they have developed acoustic indices to concisely summarise temporal and spectral variations within audio recordings, which can then be used to determine levels of biodiversity. Acoustic indices have been successfully implemented in ecological studies across a range of terrestrial environments, although their application to marine environments has not yet been widely accepted, with most studies considering coastal and coral reef ecosystems^7,8, while only a few deep-water ecosystems have been considered⁹. The use of acoustic indices to study Arctic soundscapes has yet to be explored¹⁰, with few observations into how they respond to the highly variable sounds from sea ice processes. Recent ecoacoustic studies in the Southern Ocean were successful in obtaining insightful biodiversity findings despite the presence of sea ice sounds, but this is still a topic that needs further assessment¹¹.

 

This study investigates how acoustic indices can be used as a preliminary marker for seasonal trends of biophony and sounds from sea ice processes (hereinafter ”cryophony” and ”cryophonics”) within large acoustic datasets. Data was taken from the long-term research projects “Acoustic Ocean Under Melting Ice” (UNDER-ICE, collecting data across the Fram Strait from 02/09/2014 to 06/03/2016)¹² and the “Coordinated Arctic Acoustic Thermometry Experiment” (CAATEX, collecting data along the Eurasian Basin from 31/08/2019 to 02/08/2020)¹³. Mooring positions for both experiments are presented in Figure 1, along with satellite data of sea ice thickness across the region in April 2020. As whale presence and migratory patterns in the Arctic are influenced by climate change, there was interest in detecting their vocalisations, especially those of bowhead whales (Balaena mysticetus). To count individual vocalisations in the UNDER-ICE dataset, we use energy analyses and image processing of the individual spectrograms.

 

 

Figure 1: Mooring positions for UNDER-ICE (UI1-5)¹² and CAATEX (NERSC1-3)¹³ experiments across the Fram Strait and Eurasian Basin. Sea ice thickness across the region in April 2020 is measured by the Copernicus programme^14,15,16. Map created using Cartopy¹⁷).

 

 

2 METHODOLOGY

 

2.1 Data Processing

 

UNDER-ICE acoustic measurements were made at a sampling rate of 1,953 Hz, with two recordings of 130 s separated by a few minutes, every 3 hours. The moorings were 230 to 570 m deep along the Fram Strait, the Arctic Ocean’s only deep water connection to the rest of the world ocean, therefore experienced pulldowns from strong ocean currents, causing a significant amount of self-noise from cable strumming across the duration of the experiment¹⁸. The UI4 mooring was least affected by self-noise and selected for analysis here. Recordings from CAATEX were captured every 12 hours at 976 Hz sampling frequency, each with a duration of 45 minutes and 12 seconds, each mooring featuring two time synchronised hydrophones. Every 3 days, an acoustic thermometry signal was transmitted from NERSC1, dominating the first 17 minutes of recording. These segments were removed to evaluate the seasonal trends of acoustic indices.

 

Spectrograms were computed on segments 2 s long for the acoustic indices study and 0.1 s long for the whale vocalisation detection study. In both studies, the spectrograms used 50% overlap and Hann windows, in line with PAMGuide practice 10,19. Hydrophone calibration was provided by NERSC: receiving sensitivity of -168 dB re 1 µPa, 12 dB gain and analogue-to-digital conversion voltage of 2.5 V. Spectro-grams of typical cryophony and biophony are given in Figure 2.

 

 

Figure 2: Example spectrograms of cryophony (a) and bowhead whale calls (b), made with be- spoke code inspired by PAMGuide¹⁹ and PAM2Py²⁰.

 

2.2 Acoustic Indices

 

Acoustic indices were computed using scikit-maad, an open source soundscape analysis package written in Python^21,22. A variety of popular acoustic indices were considered for the study, including the Acoustic Complexity Index (ACI)²³, the Acoustic Diversity Index (ADI)²⁴, the Acoustic Evenness Index (AEI)²⁴, the Acoustic Richness Index (ARI)²⁵, the Biodiversity Index (BI)²⁶ and Acoustic Entropy (H)²⁷. While not an established acoustic index, the zero-crossing rate of the signal (ZCR), i.e. the rate in which the signal amplitude changes sign, was also computed as it can be used to emphasise impulsive or ”percussive” sounds in the signal²⁸, which may have potential for highlighting transient-like cryophonics. Constituents of H (temporal and spectral entropies, H_t and H_f, respectively) and ARI (H_t and temporal median, H_f) were also considered as standalone indices. For indices computed across the spectrogram of a signal, the entire actual frequency range was used (1-976 Hz for UNDER-ICE, 1-488 Hz for CAATEX).

 

Acoustic indices typically involve summations of amplitudes across the time series and along the spectrogram, meaning that comparisons between recordings can only be made if they have the same duration. Therefore, CAATEX recordings were truncated into subsections commensurate with the UNDER-ICE time lengths. Subsections with acoustic thermometry signals were not used for the analyses, although some of them also showed concurrent sounds like whale vocalisations. After computing acoustic indices, the corresponding spectrograms were visually assessed to identify sound sources.

 

2.3 Detection of Whale Vocalisations

 

Whale vocalisations were detected across the UI4 dataset using the region of interest (ROI) module in scikit-maad^21,22. Here, spectrograms were treated like images and first prepared by being passed through a median equaliser to remove background noise, then smoothed using a Gaussian filter (with a standard deviation of 1.25 dB). Edges of vocalisations within the spectrogram were found by masking with a double threshold (the first as a standard deviation of 5.5 dB from the 75^th percentile of spectrogram amplitudes and the second being 2% of this value)²⁹. ROI computations were separated into two frequency bands, 100 to 500 Hz and 500 to 976 Hz, to detect vocalisations similar to those in Figure 2b. Additionally, ROIs were only counted if they lasted 0.25 to 5 s and spanned a frequency range of at least 50 Hz, in line with known and expected whale calls in this environment and context. An attempt to utilise overlapping frequency bands was made to account for whale vocalisations crossing 500 Hz, but this will not be discussed here as the approach needed further criteria to prevent individual ROIs from being counted twice.

 

 

3 DISCUSSION

 

3.1 Acoustic Indices

 

Ecoacoustic studies in general need anchoring values (and variations) for calibration settings and recording lengths^30,31 and a few examples are provided in Table 1. Monthly variation of ACI and Spearman’s rho correlation between indices over deployment durations are shown in Figure 3. UI4 values of ACI (double those of NERSC2, for twice more bandwidth) provided the most immediate insights into changes in the soundscape, with increased averages between October 2014 and April 2015, peaking in January 2015, and a similar trend starting from October 2015. This increase in acoustic complexity is attributed to bowhead whale vocalisations, due to the call variability characteristic to the species 32 and previous studies noting their presence at this time of year, potentially for mating^33,34.

 

 

Figure 3: ACI Boxplots and Spearman’s rho correlation matrices for deployments at UI4 (a,b) and NERSC2 (c,d). Median ACI values are in green and boxes encompass 25^th to 75^th percentiles.

 

In general, ADI, AEI and ZCR were quite effective at highlighting biophony but were also sensitive to background noise. In instances where biophony was faint and there was increased background noise (from stronger winds) these indices would increase (decrease for ARI) implying higher acoustic activity despite the lower biophonic component. As BI is proportional to the area under the mean spectrum of a recording, it was too dependent on the intensity of the incoming sounds within the recordings rather than their variability. H and H_f correlated slightly with ACI although their values only changed subtly with increased biophony. Time-based indices H_t, M_t and ARI did not match biophony, likely due to the high levels of low-frequency noise. Cryophony was present throughout the deployment of UI4 and was most frequent from October 2014 to March 2015, but usually fainter than the biophonic sounds observed during this time. For occasions where there was significant cryophony but no biophony, the values of acoustic indices would increase moderately (decrease for AEI).

 

 

 

Anthropophony was rare throughout the deployment, with the only notable sources being some shipping with modulated engine tones in September 2014 and distant seismic airgun sounds (below 100 Hz) in September 2014 and then irregularly between April and November 2015³³. The modulated engine tones featured multiple harmonics, dominating the spectrum enough to result in anomalous values, most notably a drop in AEI. Recordings with only airgun sounds increased BI (as the airguns add a prominent peak to the mean spectrum), along with notable decreases in ADI and H (increase in AEI) as the spectrogram becomes more uniform. Strumming was present throughout the deployment and had varying effects on acoustic indices calculations depending on severity. Low levels of strumming resulted in an increase in self-noise at low frequencies (less than 50 Hz) which generally had little impact on calculations, but high levels of strumming resulted in bands of intermittent noise across the spectrum, causing a perceived increase in acoustic activity. Data from June to September 2015 was most affected by strumming, resulting in unusually high outliers during these months (Figures 3a).

 

Monthly ACI trends throughout the NERSC2 deployment indicated a rise in acoustic activity from May to July 2020, with short down-sweep whale vocalisations, although this was not picked up particularly well by other indices. All spectral-based acoustic indices corresponded to faint, but varied whale vocalisations throughout October 2019 to March 2020, although cryophony was also very prevalent during this time. Similar to what was observed for UI4, ADI, AEI and ZCR would indicate increased acoustic activity if biophony and cryophony were faint against the noise levels across the recordings. During the same time period, no confirmed biophony was observed at NERSC1 and NERSC3, resulting in low ACI values, yet values for ADI, AEI and BI remained on average highest around this time, further suggesting that these indices were sensitive to background noise levels and cryophony. Once again, anthropophony was rare during CAATEX deployment, with distant airgun sounds between September and October 2019, which did not have a significant impact on acoustic indices calculations. Shipping or machinery at the very start and end of the deployment resulted in some outlying values for indices, but otherwise there was no other noticeable anthropogenic activity. Self-noise from strumming, though present on occasion, did not influence computations of acoustic indices.

 

3.2 Whale Vocalisation Detection

 

Figure 4 presents the number of ROIs identified per day over UI4 deployment. The majority occurred between November 2014 and March 2015 (and November 2015 and January 2016), corresponding well with the seasonal trends from the acoustic indices study (Section 3.1), but there was a significant number of ROIs between April and July 2015, a period of time in which the acoustic indices indicated, on average, little acoustic activity. Visual spectrogram inspections showed there were many call types quite different to what was observed earlier in the data, including some short narrow-band calls with multiple undulations to long ( > 5 s) sweeping down-sweeps spanning both frequency bands considered. Non-biophonic sounds were also detected as ROIs if they met the criteria of Section 2.3, which included prominent cracking, icequakes and tremor sounds, along with some modulated engine tones in August. Computations were quite resilient to the effects of strumming, although some intermittent artefacts were counted as ROIs.

 

The ROI method worked best for recordings where there were low noise levels and different call types were well separated in frequency. There were several instances where if a vocalisation was high in intensity, the reverberated echo trailing it would be also picked up as an individual ROI, leading to overcounting. Another issue encountered were multiple ROIs being computed across individual calls, which occurred more often in the 500-976 Hz band where individual down-sweep calls covered a wide frequency range but were short in time and usually faint against noise levels. For some recordings with prominent whale songs, as seen in Figure 2b, individual tones would be detected as an ROI whilst others would be ignored despite being of comparable intensity.

 

 

Figure 4: Number of ROIs detected per day (about 35 minutes of audio recording each day) during the UI4 deployment in the 100-500 Hz and 500-976 Hz frequency bands (in blue and orange respectively). Note that this plot includes all ROIs counted, not just whale calls.

 

 

4 CONCLUSIONS

 

Large datasets from the UNDER-ICE¹² and CAATEX¹³ Arctic deployments were analysed with time-frequency descriptors (acoustic indices) and energy detectors (Regions of Interest image processing).

 

ACI, ADI, AEI and ZCR provided the most useful insights for seasonal trends of biophonics and cryophonics. ACI was perhaps best at signifying prominent biophony whereas ADI, AEI and ZCR were better at indicating presence of fainter biophony against increased background noise. While acoustic indices were able to indicate when biophony was most common, more detailed ecological conclusions may not be achievable due to the influence of cryophony. Acoustic indices can however be used as quick, preliminary tools to assess soundscape evolution.

 

There is potential for using image processing of spectrograms (like ROIs) to detect whale vocalisations. The current approach needs further refinement to avoid detection of non-biophonic events and improve detection of faint calls, while avoiding detection of the reverberant trails of the louder calls. ROIs will then have applications in training Deep Learning algorithms to detect biophony and cryophony separately.

 

 

ACKNOWLEDGEMENTS

 

The UNDER-ICE Data used in this paper are used with permission from Principal Investigator of the UNDER-ICE project: Dr. Hanne Sagen, NERSC, (project no. 226373) funded by the Research Council of Norway. The CAATEX Data used in this paper are used with permission from Principal Investigators of the CAATEX projects: Dr. Hanne Sagen, NERSC, (project no. 280531) funded by the Research Council of Norway, and Dr. Matthew Dzieciuch, Scripps Institution of Oceanography, University of California (project no. N00014-18-1-2698) funded by the Office of Naval Research. This work has been completed as part of JC’s EPSRC-funded PhD studentship (reference no. EP/W524712/1) at the University of Bath.

 

 

REFERENCES

 

1. M. Ladegaard, J. Macaulay, M. Simon, K. L. Laidre, A. Mitseva, S. Videsen, et al., Soundscape and ambient noise levels of the Arctic waters around Greenland, Sci. Rep., 11 (1), 23360 (2021)

2. H. P. Huntington, L. T. Quakenbush and M. Nelson, Effects of changing sea ice on marine mammals and subsistence hunters in northern Alaska from traditional knowledge interviews, Biol. Lett., 12 (8), 20160198 (2016)

3. A. R. Szesciorka and K. M. Stafford, Sea ice directs changes in bowhead whale phenology through the Bering Strait, Mov. Ecol., 11 (1), 8 (2023)

4. L. C. Smith and S. R. Stephenson, New Trans-Arctic shipping routes navigable by midcentury, Proc. Nat. Acad. of Sci., 110 (13), E1191–E1195 (2013)

5. D. L. Gautier, K. J. Bird, R. R. Charpentier, A. Grantz, D. W. Houseknecht, T. R. Klett, et al., Assessment of Undiscovered Oil and Gas in the Arctic, Sci., 324 (591), 1175–1179 (2009)

6. T. Bradfer-Lawrence, C. Desjonqueres, A. Eldridge, A. Johnston and O. Metcalf, Using acoustic indices in ecology: Guidance on study design, analyses and interpretation, Mtds. in Ecol. and Evol., 14 (9), 2192–2204 (2023)

7. N. Pieretti and R. Danovaro, Acoustic indexes for marine biodiversity trends and ecosystem health, Phil. Transit. of the Royal Soc. B: Bio. Sci., 375 (1814), 20190447 (2020)

8. M. Minello, L. Calado and F. C. Xavier, Ecoacoustic indices in marine ecosystems: a review on recent developments, challenges, and future directions, ICES J. Mar. Sci., 78 (9), 3066–3074 (2021)

9. S. E. Parks, J. L. Miksis-Olds and S. L. Denes, Assessing marine ecosystem acoustic diversity across ocean basins, Ecol. Inf., 21, 81–88 (2014)

10. Ph. Blondel, B. Dell and C. Suriyaprakasam, Acoustic Signatures of Shipping, Weather and Marine Life: Comparison of NE Pacific and Arctic Soundscapes, Proc. Mtgs. Acoust., 40 (2020)

11. I. T. Roca and I. Van Opzeeland, Using acoustic metrics to characterize underwater acoustic biodiversity in the Southern Ocean, Rem. Sens. Ecol. and Cons., 6 (12), 262–273 (2020)

12. Nansen Environmental and Remote Sensing Center (NERSC), UNDER-ICE – Acoustic Ocean Under Melting Ice, https://under-ice.nersc.no/ (Last accessed: April 2024)

13. Nansen Environmental and Remote Sensing Center (NERSC), CAATEX – Coordinated Arctic Acoustic Thermometry Experiment, https://caatex.nersc.no/ (Last accessed: April 2024)

14. Copernicus Marine Service, Arctic Ocean – Sea Ice Thickness REPROCESSED, https://doi.org/10.48670/moi-00127 (Last accessed: April 2024)

15. Copernicus Marine Service, Arctic Ocean Sea Ice Analysis and Forecast, https://doi.org/10.48670/moi-00004 (Last accessed: April 2024)

16. T. Williams, A. Korosov, P. Rampal and E. Ólason, Presentation and evaluation of the Arctic sea ice forecasting system neXtSIM-F, Cryosph., 15, 3207–3227 (2021)

17. Met Office, Cartopy: a cartographic Python library with a Matplotlib interface, https://scitools.org.uk/cartopy/docs/latest/ (Last accessed: April 2024)

18. F. Geyer, H. Sagen, E. Storheim, A dataset of underwater passive acoustic recordings from a 2-year mooring deployment in Fram Strait (UNDER-ICE), Proc. Mtgs. Acoust., 44, 070034 (2021)

19. N. D. Merchant, K. M. Fristrup, M. P. Johnson, P. L. Tyack, M. J. Witt, Ph. Blondel and S. E. Parks, Measuring acoustic habitats, Meth. Ecol. and Evol., 6 (3), 257–265 (2015)

20. O. Rodríguez, R. J. Duarte and S. M. Jesus, PAM2PY: a Python tool for sharing ocean soundscape data, Proc. Mtgs. Acoust., 44 (1), 070011 (2021)

21. J. S. Ulloa, S. Haupert, J. F. Latorre, T. Aubin and J. Sueur, scikit-maad: An open-source and modular toolbox for quantitative soundscape analysis in Python, Meth. Ecol. and Evol., 12 (12), 2334–2340 (2021)

22. Scikit-MAAD, Soundscape analysis in Python, https://scikit-maad.github.io/ (Last accessed: April 2024)

23. N. Pieretti, A. Farina and D. Morri, A new methodology to infer the singing activity of an avian community: The Acoustic Complexity Index (ACI), Ecol. Ind., 11 (3), 868–873 (2011)

24. L. J. Villanueva-Rivera, B. C. Pijanowski, J. Doucette and B. Pekin, A primer of acoustic analysis for landscape ecologists, Lands. Ecol., 26 (9), 1233–1246 (2011)

25. M. Depraetere, S. Pavoine, F. Jiguet, A. Gasc, S. Duvail and J. Sueur, Monitoring animal diversity using acoustic indices: Implementation in a temperate woodland, Ecol. Ind., 13 (1), 46–54 (2012)

26. N. T. Boelman, G. P. Asner, P. J. Hart and R. E. Martin, Multi-trophic invasion resistance in Hawaii: bioacoustics, field surveys, and airborne remote sensing, Ecol. App., 17 (8), 2137–2144 (2007)

27. J. Sueur, S. Pavoine, O. Hamerlynck and S. Duvail, Rapid Acoustic Survey for Biodiversity Appraisal, PLOS ONE, 3 (12), e4065 (2009)

28. F. Gouyon, F. Pachet and O. Delerue, On the Use of Zero-Crossing Rate for an Application of Classification of Percussive Sounds, Proc. COST G-6 Conf. Dig. Aud. Eff. (DAFX-00), Verona, Italy (2000)

29. J. Canny, A Computational Approach to Edge Detection, IEEE Trans. Patt. Analysis and Mach. Intell. PAMI, 8 (6), 679–698 (1986)

30. E. Fakiris et al., The soundscape of the Inner Ionian Archipelago as evinced through the West Patraikos Gulf Ambient and Seismic Noise Monitoring Project, Proc. UACE-2019, 351–358 (2019)

31. Ph. Blondel, A. A. Zailani Hatta, Acoustic soundscapes and biodiversity – comparing metrics, seasons and depths with data from the NEPTUNE ocean observatory offshore British Columbia, Proc. UACE2019, 763–768 (2017)

32. K. M. Stafford, C. Lydersen, Ø. Wiig and K. M. Kovacs, Extreme diversity in the songs of Spitsbergen’s bowhead whales, Biol. Lett., 14 (4), 20180056 (2018)

33. F. Geyer, H. Sagen, G. Hope, M. Babiker and P. F. Worcester, Identification and quantification of soundscape components in the Marginal Ice Zone, J. Acoust. Soc. Am., 139 (4), 1873–1885 (2016)

34. K. Thomisch, K. Hiemer, O. Boebel, E. Burkhardt, S. Spiesecke and I. van Opzeeland, Acoustic presence and vocal repertoire of bowhead whales (Balaena mysticetus) in Eastern and Central Fram Strait, Front. Rem. Sens., 3 (2022)