VirtualBox 5.2.0: Interrupt storm on emulated HDA controller (FreeBSD) -> fixed in 5.2.10

Scenario:

• VirtualBox 5.2.0 compiled from ports on FreeBSD ​https://svnweb.freebsd.org/ports/head/emulators/virtualbox-ose/
• patch svn diff -r69153:69154 http://www.virtualbox.org/svn/vbox (iSCSI fix) applied manually
• Both host and client system are FreeBSD 11.1

Result:

• The client boots o.k.
• The client kernel reports an excessive interrupt rate for the HDA audio controller (1k..50k/sec depending on client load) even with no audio playing
• Console output:

  interrupt storm detected on "irq21:"; throttling interrupt source
  interrupt storm detected on "irq21:"; throttling interrupt source
  interrupt storm detected on "irq21:"; throttling interrupt source
  

• Running systat -vm 1 :

   users Load 0.24 0.20 0.09 Oct 27 16:12
   Mem usage: 7%Phy 4%Kmem
  Mem: KB REAL VIRTUAL VN PAGER SWAP PAGER
   Tot Share Tot Share Free in out in out
  Act 162036 14172 1293824 16504 2855288 count
  All 162644 14664 1303256 25644 pages
  Proc: Interrupts
   r p d s w Csw Trp Sys Int Sof Flt ioflt 3318 total
   96 404 13 544 2973 30 cow atkbd0 1
   zfod ata0 14
   0.2%Sys 2.6%Intr 0.0%User 0.0%Nice 97.2%Idle ozfod 2 ata1 15
  | | | | | | | | | | %ozfod 17 le0 19
  + daefr vboxguest0
   1 dtbuf prcfr 2953 hdac0 21
  Namei Name-cache Dir-cache 42858 desvn totfr 228 cpu0:timer
   Calls hits % hits % 3572 numvn react 114 cpu3:timer
   3 3 100 1740 frevn pdwak 1 cpu2:timer
   40 pdpgs 3 cpu1:timer
  Disks md0 ada0 cd0 pass0 pass1 intrn
  KB/t 0.00 0.00 0.00 0.00 0.01 64464 wire
  tps 0 0 0 0 1 96496 act
  MB/s 0.00 0.00 0.00 0.00 0.00 64636 inact
  %busy 0 0 0 0 0 laund
   2855288 free
  

Expected result:

• No HDA interrupts as long as no audio is playing

VBox.Log when using VirtualBox 5.2.0

VBox.Log when running the same machine using VirtualBox 5.1.30

I've just uploaded a new 5.2 test build which contains more audio fixes. Could you please give these builds a try and report back if these fix the issue for you? You can get the latest 5.2 test builds here: https://www.virtualbox.org/wiki/Testbuilds

I just re-tried this using the latest FreeBSD port, virtualbox-ose-5.2.6_2. The behavior is still the same, there are thousands of interrupts per second on hdac0.

Sorry, I might have missed your previous response. Does this test with virtualbox-ose-5.2.6_2 already include the changes in the test build you wanted me to try 8 weeks ago?

Diffing a VBox.log from 5.1.30 and from that 5.2.6, there are three additional lines involving HDA:

 HDA: Asynchronous I/O enabled

 HDA: Codec reset

 HDA: Codec reset

So maybe the issue is that on FreeBSD, async I/O with HDA does not work?

I have the same problem, and it still occurs with 5.2.8. 5.1.34 is working fine.

I have now run openSUSE tumbleweed as client under VBox 5.2.8 on a FreeBSD host in order to get some debug audio using skype in the client.

The audio is very choppy.

I have enabled audio debug acc. to https://www.virtualbox.org/wiki/AudioDebug

The resulting .wav files plus the VBox.log file are attached.

There are many messages like "OSS: Warning: Too big output size (13120 > 256), limiting to 256" in the log file.

VBox.log running openSUSE tumbleweed on 5.2.8, with audio debug enabled

I just noticed that I cannot upload the corresponding .wav files because of a file size limit of 512k. There are the following files:

% ll *.wav
-rw-r--r-- 1 test wheel 2892280 Mar 18 11:02 CaptureNonInterleaved-0.wav
-rw-r--r-- 1 test wheel 3261088 Mar 18 11:02 DebugAudioOut-0.wav
-rw-r--r-- 1 test wheel 4541392 Mar 18 11:02 hdaDMAReadSD4-0.wav
-rw-r--r-- 1 test wheel 2684576 Mar 18 11:02 hdaDMAWriteSD0-0.wav
-rw-r--r-- 1 test wheel 4541392 Mar 18 11:02 hdaStreamReadSD4-0.wav
-rw-r--r-- 1 test wheel 2754852 Mar 18 11:02 hdaStreamWriteSD0-0.wav
-rw-r--r-- 1 test wheel 4544472 Mar 18 11:02 PlayNonInterleaved-0.wav
-rw-r--r-- 1 test wheel 4544472 Mar 18 11:02 PlayNonInterleaved-2.wav
-rw-r--r-- 1 test wheel 2755028 Mar 18 11:02 StreamRead-0.wav
-rw-r--r-- 1 test wheel 4544472 Mar 18 11:02 StreamWrite-0.wav
-rw-r--r-- 1 test wheel 4544472 Mar 18 11:02 StreamWrite-1.wav
-rw-r--r-- 1 test wheel 4544472 Mar 18 11:02 StreamWrite-2.wav
% 

They all contain the same very choppy sound heard in the emulated OS, so it seems it is a case of

Distorted 	Distorted 	Device emulation problem

according to https://www.virtualbox.org/wiki/AudioDebug (some of them with a very low volume, they seem to be the microphone input fed back from the speakers).

If there is a possibility to upload these files tell me, I'll keep them for a while.

How are comments 6 and 7 relevant to this ticket, which is about an interrupt storm in FreeBSD guests?

The FreeBSD problem should be fixed in the latest 5.2 test build, it was a bug in the HDA emulation (incorrectly reporting an interrupt that the guest OS didn't know how to clear). Probably has zero impact on other guest OSes.

For the sake of completeness: The FreeBSD HDA interrupt handler writes the INTSTS register, which is specified as read-only. It's not clear to us what that is meant to accomplish, although the only harm it causes is slowing things down.

Replying to michaln:

The FreeBSD problem should be fixed in the latest 5.2 test build, it was a bug in the HDA emulation (incorrectly reporting an interrupt that the guest OS didn't know how to clear). Probably has zero impact on other guest OSes.

For the sake of completeness: The FreeBSD HDA interrupt handler writes the INTSTS register, which is specified as read-only. It's not clear to us what that is meant to accomplish, although the only harm it causes is slowing things down.

The original HDA 1.0 Spec. said INTSTS was RW1C instead of RO, it seems. Actually, Google found this:

​http://crashrecovery.org/hdaudio_pdf/hda001-a.pdf

Replying to michaln:

How are comments 6 and 7 relevant to this ticket, which is about an interrupt storm in FreeBSD guests?

The FreeBSD problem should be fixed in the latest 5.2 test build, it was a bug in the HDA emulation (incorrectly reporting an interrupt that the guest OS didn't know how to clear). Probably has zero impact on other guest OSes.

For the sake of completeness: The FreeBSD HDA interrupt handler writes the INTSTS register, which is specified as read-only. It's not clear to us what that is meant to accomplish, although the only harm it causes is slowing things down.

Actually, it was me who reported the ticket and who added both these two comments. I simply thought the problems might be related.

But thank you for providing a fix to the original issue, I'm just compiling now and will report how well the latest version works. Also thanks to jkim for quickly updating the FreeBSD port.

Replying to jkim:

The original HDA 1.0 Spec. said INTSTS was RW1C instead of RO, it seems. Actually, Google found this:

​http://crashrecovery.org/hdaudio_pdf/hda001-a.pdf

Ah, nice -- you found an official document showing the INTSTS write done by FreeBSD is redundant and explaining why the code was written that way :) I unfortunately don't know if there might be any hardware implementing the originally specified behavior. Actual Intel chips definitely should not need it, they always documented INTST as R/O in datasheets.

The interrupt issue is indeed resolved, thanks for the fix.

And the skype audio quality (as described in comments 6 and 7) is still bad, so indeed this is another issue. This is with a Linux client. However, amarok audio output is o.k., so maybe this is a skype issue.

Interestingly, with a Windows 10 preview client the audio is good.

Interestingly, also with a FreeBSD client the audio is good (only output tested).