Proceedings of the Institute of Acoustics

 

Content generation for SAS change detection simulations

 

S. F. Johnson, The Pennsylvania State University, USA

 

 

1 INTRODUCTION

 

Synthetic Aperture Sonar (SAS) based Change Detection requires estimating and accounting for differences in the navigation track between base and repeat pass collections. In dynamic environments, there are also changes of the seabed because of hydrodynamic and/or biologic activity, which can mask detection of the intended anthropogenic activity. Combined, these uncertainties make the use of collected data challenging for purposes such as validation of signal processing techniques and study of the spatiotemporal coherence of the scattered acoustic field. In this paper we describe the development of a dataset specifically generated for such assessments leveraging the prescriptive nature of simulation.

 

Content generation is a topic which has received substantive investigation, with significant advances made in the fields of computer-generated imagery (CGI), and both recreational and “serious” gaming. Techniques used to generate and combine seabed textures, such as hydrodynamically-generated sand ripples, biotubatively-generated fish pits, and power-law roughness, will be described. To represent the evolving nature of the seabed in dynamic environments, a model for horizontal diffusion was employed.

 

Two evolving seabed scenes were generated, with several collection aspects. For each of these combinations four sets of acoustic time-series data and reconstructed imagery were generated: an initial pass for a base image, an easy case with just a handful of localized disturbances, a medium case also including diffusive sand ripples, and a hard case also including a collection geometry mismatch. These sets of data are intended to test SAS reconstruction techniques, change detection techniques, and data/registration techniques. The process of developing these SAS datasets can be thought of in several discrete steps:

 

•  Relief Generation (i.e. generating the ”world” that the sonar is operated in),

•  Acoustic Rendering (i.e. calculating the sonar time series response of the ”world”),

•  Image Reconstruction (i.e. application of synthetic aperture techniques), and

•  Image Processing (i.e. image registration, and change detection).

 

The focus here is the relief generation with Procedural and Declarative techniques and description of the dataset applicable to SAS based Change Detection. The intermediary steps of acoustic rendering and image reconstruction will be discussed only briefly as there are myriad other papers on these topics. The image registration and change detection steps are left to the consumer to develop and demonstrate.

 

 

Figure 1: Semantic map (left), individual texture layers (middle), and composite seafloor relief (right).

 

 

2 CONTENT GENERATION

 

2.1 Overview

 

For several decades, computer-generated imagery (CGI) has been utilized for generating or augmenting images and movies, relying on a combination of algorithmically or procedurally generated, along with manually or declaratively generated techniques. Perhaps the most notable improvement in the realism of procedurally generated content is often attributed to Perlin 1 following his involvement in the 1982 movie Tron. Interestingly, Bell² mentions “procedurally defined objects” in the 1997 paper on side-scan sonar simulation. Here we describe combining both procedural and declarative content generation techniques to develop realistic and representative environments as the basis for a simulated dataset intended to test several aspects of Synthetic Aperture Sonar based Change Detection.

 

2.2 Procedural Generation

 

Procedural approaches are those which are generated algorithmically, and are often driven by a stochastic component to promote variety and disrupt unnatural repetition. While certain algorithms are entirely heuristic, the more enticing and often realistic algorithms are based on models of physical processes. When procedural models are applied to textures, they can often be layered to create more heterogenous and interesting scenes. The upper limit of realism for procedural approaches is our ability to develop models for, and supporting parameterizations of, physical processes.

 

 

Figure 2: Composite seafloor relief (left) and heightmap difference (right) for Scene 1.

 

Seafloor relief has long been described as fractal, ranging from large scale bathymetry³ to small scale roughness where it is often described as power-law⁴. This physical basis, supported by known parameterizations and application to acoustics^5,6, combined with algorithmic implementations in both spatial and wavenumber domains, make random fractal or power-law roughness texture a common starting point for use in sonar simulations^2,7. 

 

To expand upon random roughness, several other models have been applied in a procedural approach. Tang⁸ proposed a rippled-sand texture model, which was applied to the generation of computationally efficient SAS-like imagery⁹. The effect of bioturbation was modeled with horizontal diffusion¹⁰, and utilized to simulate the decay of sand ripples^11,12 and subsequent decorrelation of simulated SAS imagery¹¹. A spatial domain model for a fish-feeding pit¹³ was combined with the horizontal diffusion model¹⁰ to simulate production of a random roughness seafloor texture with a spectrum that is power-law-like attributable to bioturbation¹⁴. These basis textures of random roughness, rippled-sand, bioturbation, and the process of horizontal diffusion serve as the procedural textures for this work. 

 

2.3 Declarative Generation

 

Declarative approaches are those which are generated manually, and often rely on the expertise of the user to ensure realism. In the entertainment industry this is often an artist composing content, while in the education industry it may be by the participation of an instructor to manipulate content of a serious game in a pedagogical manner. Declarative approaches can also be utilized when either a physical model has not yet been identified, or is too difficult to be implemented procedurally.

 

Smelik¹⁵ proposed an approach where a designer layers procedurally generated textures in a prescribed manner with manually defined boundaries. This concept has been employed here with simple raster software to design arbitrary regions creating a semantic map with labels such as roughness, ripples, and bioturbation. Although manually designed, inspiration for the boundaries between regions was drawn from analysis of collected SAS imagery in order to capture natural heterogeneity between seafloor textures. The semantic map is then used to mask layers of procedurally-generated realizations of corresponding texture models.

 

 

Figure 3: Composite seafloor relief (left) and heightmap difference (right) for Scene 2.

 

3 DATASET GENERATION

 

3.1 Seafloor Relief Scenes

 

Two distinct seafloor relief scenes were synthesized with a combination of the procedural and declarative approaches described above. Each scene measured 125m by 125m and consisted of regions of random roughness, sand ripples, and 35-40 large fish pits ranging from 0.50-1.15m in diameter. For application to the Change Detection problem, the random roughness regions were held static while the sand ripples were either held static or allowed to evolve with horizontal diffusion, and an additional 7-9 fish pits ranging from 0.62-1.18m in diameter were placed to serve as the intended change to detect (n.b. these may not be detectible in both or either image because the SAS image, described in the next section, is a subset of the seabed relief scene).

 

3.2 Acoustic Rendering & Image Reconstruction Techniques

 

Previous work employed a computationally efficient approach to render seafloor relief maps utilizing pseudo-image SAS techniques^9,11,16, which produces visually representative imagery capturing several hallmark traits of SAS and demonstrated expected repeat-pass decorrelation. However, several of the approximations preclude use of coherent image reconstruction, registration, and difference techniques important in the change detection process. In order to address this need, the seafloor relief scenes were rendered into acoustic time-series with The Point-based Sonar Signal Model (PoSSM)^17,18. The SAS sensor described by Bellettini¹⁹ and Pinto²⁰ was modeled with representative acoustic signal, transmit and receive beampatterns, and array spacing. The acoustic time series were coherently reconstructed into synthetic aperture imagery with standard techniques^21,22 to create 50m along-track by 40-150m slant range complex images.

 

 

Figure 4: Example SAS imagery of Scene 1(b).

 

 

Figure 5: Example SAS imagery of Scene 2(a).

 

 

Figure 6: Example Change Detection solutions for Scene 1(b).

 

 

Figure 7: Example Change Detection solutions for Scene 2(a).

 

3.3 Dataset Description

 

Two orthogonal passes of the sonar vehicle were made for each scene for a total of 4 reference or baseline images for “ReferencePass.” Then data were generated for each scene for each direction at 3 levels of difficulty for a total of 12 repeat-pass images. For “RepassEasy” the objective change detection fish pits are placed, all regions of the seafloor are held static, and sensor trajectory from “ReferencePass” is reused. This level is intended to test just the detection aspect of consumers’ change detection approach. For “RepassMedium” the objective change detection fish pits are placed, roughness regions of the seafloor are static while sandy regions are allowed to diffuse, and sensor trajectory from “ReferencePass” is reused. The diffusion of sandy regions is intended to resemble combined hydrodynamics and smaller scale bioturbation (n.b. both scenes have the same diffusion rate, but were allowed to diffuse for different amounts of time; fish pits present in the reference image in rippled sand regions will diffuse, however those in the more stable roughness regions will not). This level is intended to test detection as well as false-alarm reduction or background change mitigation. For “RepassDifficult” the objective change detection fish pits are placed, roughness regions of the seafloor are static while sandy regions are allowed to diffuse, and sensor trajectory different from “ReferencePass” is used. The trajectory difference is either a slight heading (up to 1 degree) and/or baseline offset (up to 0.5m vertical and 1.5m horizonal) however still with a straight path to permit simple reconstruction. This level is intended to test detection, false-alarm reduction or background change mitigation, as well as image alignment techniques.

 

3.4 Example Results

 

The intent of the dataset is to demonstrate and test the consumers’ image registration and change detection routines, and as such those data products are not provided. However the height maps can be differenced to identify all changes between the seafloor scenes. Additionally a few examples of rudimentary change detection²³ results are provided here.

 

4 CONCLUSION

 

A dataset was generated for the development and testing of SAS based Change Detection algorithms. The seafloor relief utilized content generation techniques to generate realistic textures with representative temporal evolution. These scenes were rendered with a high-fidelity acoustic model to generate acoustic time-series data, analogous to a fielded system, which are suitable for coherent signal processing including synthetic aperture image reconstruction techniques. Several levels of difficulty were modeled to test various aspects of SAS based Chance Detection algorithms.

 

ACKNOWLEDGMENTS

 

This work was supported by the U.S. Office of Naval Research under Grants N00014-16-1-3022 and N00014-20-1-2581.

 

REFERENCES

 

1. Perlin, K., “An image synthesizer,” SIGGRAPH ’85: Proceedings of the 12th annual conference on Computer graphics and interactive techniques, 19(3): 287-296, 1985.

2. Bell, J. M. and L. M. Linnett, “Simulation and analysis of synthetic sidescan sonar images,” IEE Proc. - Radar, Sonar Navig., 144(4): 219-229, 1997.

3. Goff, J. A. and T. H. Jordan, “Stochastic modeling of seafloor morphology: inversion of Sea Beam data for second-order statistics,” Journal of Geophysical Research: Solid Earth (1978–2012), 93(B11): 13589-13608, 1988.

4. Jackson, D. R., D. P. Winebrenner, and A. Ishimaru, “Application of the composite roughness model to high-frequency bottom backscattering,” The Journal of the Acoustical Society of America, 79(5): 1410-1422, 1986.

5. Newhall, B. K., “A physical model for the distribution of sonar clutter from a rough interface,” Proceedings of OCEANS 2006, IEEE: Boston, Massachusetts, 2006.

6. Jackson, D. R. and M. D. Richardson, High-Frequency Seafloor Acoustics, New York, Springer, 2007.

7. Hunter, A. J., Underwater acoustic modelling for synthetic aperture sonar, University of Canterbury, Christchurch, New Zealand, 2006.

8. Tang, D., F. S. Henyey, B. T. Hefner, and P. A. Traykovski, “Simulating realistic-looking sediment ripple fields,” IEEE J. Ocean. Eng., 34(4): 444-450, 2009.

9. Johnson, S. F. and A. P. Lyons, “Simulation of rippled-sand synthetic aperture sonar imagery,” Institute of Acoustics Proceedings, 32.pt 4 (2010).

10. Jackson, D. R., M. D. Richardson, K. L. Williams, A. P. Lyons, C. D. Jones, K. B. Briggs, and D. Tang, “Acoustic observation of the time dependence of the roughness of sandy seafloors,” IEEE J. Ocean. Eng., 34(4): 407-422, 2009.

11. Johnson, S. F. and A. P. Lyons, “Simulation of rippled-sand seafloor evolution for synthetic SAS imagery,” Proceedings of the 4th Underwater Acoustic Measurements Conference: Technologies and Results, 2011.

12. Penko, A. S. Johnson, and J. Calantoni, “Simulation of measured seafloor roughness spectrum time series using a coupled ripple-bioturbation model,” Institute of Acoustics Proceedings, 37.pt 1, 2015.

13. Tang, D., “Fine-scale measurements of sediment roughness and subbottom variability,” IEEE J. Ocean. Eng., 29(4): 929-939, 2004.

14. Johnson, S. F. and D. R. Jackson, “Modelling seafloor bioturbation. Seabed and Sediment Acoustics: Measurements and Modelling,” Institute of Acoustics Proceedings, 37.pt 1, 2015.

15. Smelik, R. M., R. Tutenel, K. J. de Kraker, and R. Bidarra, “A proposal for a procedural terrain modelling framework,” Eurographics, 39-46, 2008.

16. Johnson, S. F., Synthetic aperture sonar image statistics, The Pennsylvania State University, State College, Pennsylvania, 2009.

17. Johnson, S. F. and D. C. Brown, “SAS Simulations with Procedural Texture and the Point-based Sonar Scattering Model,” Proceedings of OCEANS 2018 Marine Technology Society / IEEE Oceanic Engineering Society, Charleston, South Carolina, 2018.

18. Brown, D. C., S. F. Johnson, and D. R. Olson, “A point-based scattering model for the incoherent component of the scattered field,” J. Acoust. Soc. Am., vol. 141, no. 3, pp. EL210–EL215, 2017.

19. Bellettini, A. and M. Pinto, “Design and experimental results of a 300-kHz Synthetic Aperture Sonar optimized for shallow-water,” 2009.

20. Pinto, M., “Design of Synthetic Aperture Sonar systems for high-frequency seabed imaging (tutorial slides),” 2006.

21. Hawkins, D., Synthetic Aperture imaging algorithms: with application to wide bandwidth sonar, University of Canterbury, Christchurch, New Zealand, 1996.

22. Callow, H., Signal processing for Synthetic Aperture Sonar image enhancement, University of Canterbury, Christchurch, New Zealand, 2003.

23. Lyons, A. P. and D. C. Brown, “Temporal variability of seafloor roughness and its impact on coherent change detection,” Institute of Acoustics Proceedings, 32.pt 4 (2010).