Attaching procedural audio to an ARKit SCNNode

Question

I am trying to create an AR app with procedural (generated) audio attached to detected planes in the space. I am using ARKit, and was previously using AudioKit to generate audio, although I’ve stripped it back now.

What successfully works:

• Creating a SCNAudioSource with a static wav file
• Then creating a SCNAudioPlayer with the source
• Then adding the audio player to the attached plane with node.addAudioPlayer()

Like so:

let audioSource = SCNAudioSource(fileNamed: "chime.wav")
let audioPlayer = SCNAudioPlayer(source: audioSource)
node.addAudioPlayer(audioPlayer)

This works successfully. The wav file plays back and generates sound.

What doesn’t work is creating some procedural audio with an AVAudioNode, as documented here https://developer.apple.com/documentation/scenekit/scnaudioplayer/init(avaudionode:), like this:

let audioNode = AVAudioSourceNode(format: format, renderBlock: { _, _, frameCount, audioBufferList -> OSStatus in 
 // some code for generating audio
 })
let audioPlayer = SCNAudioPlayer(avAudioNode: audioNode)
node.addAudioPlayer(audioPlayer)

No audio is generated, no error message is generated. It just silently fails. The audio for the AVAudioSourceNode is here

To make doubly sure, I tested that it works by attaching the AVAudioSourceNode directly to an AVAudioEngine instance. It plays/generates sound correctly.

I then searched for instances of using the SCNAudioPlayer(avAudioNode:) on GitHub, and there is a whole AR game project from WWDC18. I fixed up the build errors (for the latest XCode) and built it. All the graphics work, but guess what doesn’t work? The audio. The project is forked below:

https://github.com/joshkopecek/SwiftShot

Interestingly, as a side note, the Apple Documentation says

Using this initializer is typically not necessary. Instead, call the audioPlayerWithAVAudioNode: method, which returns a cached audio player object if one for the specified AVAudioNode object has already been created and is available for use.

...but this method is not available in Swift.

• Gordon Childs: Does your AVAudioSourceNodeRenderBlock get called?
• Josh Kopecek: @GordonChilds no it doesn't.
• Josh Kopecek: Just to be clear @GordonChilds it does get called when I attach it to an AVAudioEngine instance directly, but doesn't when I attach it to the SCNAudioPlayer