| To: | xen-users@xxxxxxxxxxxxxxxxxxx |
|---|---|
| Subject: | Re: [Xen-users] Re: HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails. |
| From: | Dietmar Hahn <dietmar.hahn@xxxxxxxxxxxxxx> |
| Date: | Wed, 8 Dec 2010 14:48:20 +0100 |
| Cc: | "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>, Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>, yunhong.jiang@xxxxxxxxx, "JBeulich@xxxxxxxxxx" <JBeulich@xxxxxxxxxx>, yuan.b.liu@xxxxxxxxx, anthony.perard@xxxxxxxxxx, Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>, Mark Adams <mark@xxxxxxxxxxxxxxxxxx> |
| Delivery-date: | 2010年12月08日 05:49:59 -0800 |
| Dkim-signature: | v=1; a=rsa-sha256; c=simple/simple; d=ts.fujitsu.com; i=dietmar.hahn@xxxxxxxxxxxxxx; q=dns/txt; s=s1536b; t=1291816104; x=1323352104; h=from:to:subject:date:cc:references:in-reply-to: mime-version:message-id; z=From:=20Dietmar=20Hahn=20<dietmar.hahn@xxxxxxxxxxxxxx> |To:=20xen-users@xxxxxxxxxxxxxxxxxxx|Subject:=20Re:=20[Xe n-users]=20Re:=20HVM=20DomU,=20msi_translate=3D0,=20MSI/M SI-X=20PCI=20passthrough=20fails.|Date:=20Wed,=208=20Dec =202010=.&checktime(2014,48,20,':')=20+0100|Cc:=20Mark=20Adams=20<mark@cam pbell-lange.net>,=0D=0A=20Konrad=20Rzeszutek=20Wilk=20<ko nrad.wilk@xxxxxxxxxx>,=0D=0A=20"xen-devel@xxxxxxxxxxxxxxx .com"=20<xen-devel@xxxxxxxxxxxxxxxxxxx>,=0D=0A=20Stefano =20Stabellini=20<stefano.stabellini@xxxxxxxxxxxxx>,=0D=0A =20yunhong.jiang@xxxxxxxxx,=0D=0A=20"JBeulich@xxxxxxxxxx" =20<JBeulich@xxxxxxxxxx>,=0D=0A=20yuan.b.liu@xxxxxxxxx, =0D=0A=20anthony.perard@xxxxxxxxxx|References:=20<2010111 2222215.GC26189@xxxxxxxxxxxx>=20<20101129163635.GA20417@d umpdata.com>=20<20101208125855.GA26222@xxxxxxxxxxxxxxxxxx >|In-Reply-To:=20<20101208125855.GA26222@xxxxxxxxxxxxxxxx et>|MIME-Version:=201.0|Message-Id:=20<201012081448.20935 .dietmar.hahn@xxxxxxxxxxxxxx>; bh=rBiHJbK3A1JVbg6hy6grfHQFSOWV9BnciBQj310UcOU=; b=ATd8rGskjQKAVSs01SBhzB9XvXPOyAw/+O/AkRVZr/SU4TzgDV/R8+Hl vfdDmaFiOZtfSS2pNCyPWIflmP9S9AHvh7a40K49kjZcOje5JAEgQOpBH pMmT8CwycvmByXMfQKDZDXOSzI2uIppoaCaBoR5fkfDo16Mf8zTqF2apg 75AtbQx7gpQ1pZlIokzfix2wFwFfDHCkDR4BuTuNG4Wreppk6d6Zwiy47 G8jplgezKC1B7/6SgwytKJz/iLCjU; |
| Domainkey-signature: | s=s1536a; d=ts.fujitsu.com; c=nofws; q=dns; h=X-SBRSScore:X-IronPort-AV:Received:X-IronPort-AV: Received:Received:From:To:Subject:Date:User-Agent:Cc: References:In-Reply-To:X-KMail-Markup:MIME-Version: Content-Type:Message-Id; b=b62iH2c3ritpWJR5A6DqNWLyGxGvL129upxBg9x3C0go69dWpX53E8Xs v3WkYBNqaHY060KsLNBQ75tubh8gZkv88Ct6murh+OSdYN6jxwNzkZaCA bIk9MQu9rBxC/E+yqbWL4ME5UdlAH4KrQJC1WUC3TpWBnDWRECqd7x2iJ gB1gqmOhJAlm2NVCj3V8kBhW4wgX/Awb8gey/t3KUPxJ3IhAk7Qynby0Z hIX2zRGugqsLULkAgtP1iRIYbXEnS; |
| Envelope-to: | www-data@xxxxxxxxxxxxxxxxxxx |
| In-reply-to: | <20101208125855.GA26222@xxxxxxxxxxxxxxxxxx> |
| List-help: | <mailto:xen-users-request@lists.xensource.com?subject=help> |
| List-id: | Xen user discussion <xen-users.lists.xensource.com> |
| List-post: | <mailto:xen-users@lists.xensource.com> |
| List-subscribe: | <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe> |
| List-unsubscribe: | <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe> |
| References: | <20101112222215.GC26189@xxxxxxxxxxxx> <20101129163635.GA20417@xxxxxxxxxxxx> <20101208125855.GA26222@xxxxxxxxxxxxxxxxxx> |
| Sender: | xen-users-bounces@xxxxxxxxxxxxxxxxxxx |
| User-agent: | KMail/1.13.5 (Linux/2.6.34.7-0.5-xen; KDE/4.5.4; x86_64; ; ) |
Hi,
Am 08.12.2010 schrieb "Mark Adams <mark@xxxxxxxxxxxxxxxxxx>":
> Hi - Apologies to top post this, but after alot of testing, I believe
> there must be an issue with IRQ's going missing between domU and dom0.
> Unfortunately I have no data to prove this!
>
> With msitranslate=0 as detailed below, and pci=nomsi in the guest kernel
> grub config, all 3 NIC's appear OK in the domU however I still had
> issues with the red-fone ISDN box. The interrupts were showing correctly
> (2000/s) in the domU but communication to the device via the NIC was
> still being interrupted (as shown in the asterisk console)Note that to
> get the igb driver to allow this many interrupts, the
> InterruptThrottleRate was set to 0. The same config (red-fone box,
> asterisk etc) works fine with a physical server.
>
> There is also the additional issue that I could not get the passthrough
> NIC's to show correctly when I also had a bridge setup.
>
> Throughout my testing however, I could not get the machine to crash.
>
> Not sure where to go with this one. For now we are keeping our VoIP
> servers physical when ISDN connections are required.
Today I did some tests with xen-unstable and found these problems too.
I tried to passthrough 2 pci cards and got some error messages on the xen
xonsole and in the qemu logs.
With msitranslate=0 and pci=nomsi I got the soundcard working in a domU linux
but it doesn't help on windows.
I attached the logs from the xen serial console and the qemu logs.
Thanks!
Dietmar.
>
> Regards,
> Mark
>
> On Mon, Nov 29, 2010 at 11:36:35AM -0500, Konrad Rzeszutek Wilk wrote:
> > >
> > > In my new test setup, I have seen some strange behaviour. 1 of the HVM's
> > > (with identical config in dom0 and domU) suddenly would not allow the
> > > igb driver to be loaded in domU, even though the device was visible in
> >
> > Let's create a new thread for this other issue.
> >
> > > lspci. Shutting the machine down, removing the power cord, waiting 5
> > > seconds then plugging it in again corrected that issue - Is this
> > > possibly a motherboard bug? I have also disabled the SR-IOV
> > > functionality in the BIOS incase this is causing any issues.
> > >
> > > In addition, to try to correct the MSI issue noted above, I have changed
> > > my pci= line to the following:
> > >
> > > pci=[ '08:00.0,msitranslate=0', '08:00.1,msitranslate=0' ]
> >
> > With the msi_translate=1 turned on the DomU HVM guests did work, right?
> >
> > >
> > > This has stopped the "already in use on device" log, and the devices
> > > appear to show correctly in the domU. Is it safe to disable
> > > msitranslate? as I understand it, its for allowing multifunction devices
> > > to be seen as such in domU. Is that correct?
> > >
> > > I haven't been able to reproduce the dropped raid issue yet, but I am
> > > awaiting delivery of the Red-Fone boxes (ISDN VoIP) which seem to cause
> > > this due to their very high interrupt usage (2000 per second).
> >
> > OK.
> > >
> > > In the mean time, I can see the following in the qemu-dm logs now with
> > > the msitranslate=0 enabled. Is it anything to worry about?
> >
> > Well, the "Error" ones are pretty bad, thought I am having a hard time
> > understanding what it means. Lets copy some of the QEMU folks on this.
> >
> > > pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:05.0][Offset:14h][Length:4]
> > > pt_ioport_map: e_phys=ffff pio_base=e880 len=32 index=2 first_map=0
> > > pt_ioport_map: e_phys=c220 pio_base=e880 len=32 index=2 first_map=0
> > > pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:06.0][Offset:14h][Length:4]
> > > pt_ioport_map: e_phys=ffff pio_base=ec00 len=32 index=2 first_map=0
> > > pt_ioport_map: e_phys=c240 pio_base=ec00 len=32 index=2 first_map=0
> > > pt_msix_update_one: Update msix entry 0 with pirq 4f gvec 59
> > > pt_msix_update_one: Update msix entry 1 with pirq 4e gvec 61
> > > pt_msix_update_one: Update msix entry 2 with pirq 4d gvec 69
> > > pt_msix_update_one: Update msix entry 3 with pirq 4c gvec 71
> > > pt_msix_update_one: Update msix entry 4 with pirq 4b gvec 79
> > > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function.
> > > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function.
> > > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function.
> > > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function.
> > > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function.
> > > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function.
> > > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function.
> > > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function.
> > > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function.
> > > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function.
> > > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function.
> > > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function.
> > > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function.
> > > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function.
> > > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function.
> > >
> > > >
> > > > Not yet. Need to serial log of the Linux kernel and the Xen hypervisor when your
> > > > machine is toast. I mentioned in the previous email the key sequences - look on Google
> > > > on how to pass in SysRQ if you are using a serial concentrator.
> > >
> > > I will do this when I can get the machine to crash.
> > >
> > > Best Regards,
> > > Mark
> > >
> > > _______________________________________________
> > > Xen-devel mailing list
> > > Xen-devel@xxxxxxxxxxxxxxxxxxx
> > > http://lists.xensource.com/xen-devel
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-users
>
>
--
Company details: http://ts.fujitsu.com/imprint.html
Attachment:
xen-serial-log.txt
Description: Text document
Attachment:
qemu-dm-OpenSuseHVM.log
Description: Text Data
_______________________________________________ Xen-users mailing list Xen-users@xxxxxxxxxxxxxxxxxxx http://lists.xensource.com/xen-users
| Previous by Date: | Re: [Xen-devel] Re: HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails. , Sander Eikelenboom |
|---|---|
| Next by Date: | Re: [Xen-devel] Re: HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails. , Mark Adams |
| Previous by Thread: | [Xen-users] Re: [Xen-devel] Re: HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails. , Mark Adams |
| Next by Thread: | [Xen-users] Kernel DomU 2.6.36-rc6-pvhvm dont boot PVonHVM , Roberto Scudeller |
| Indexes: | [Date] [Thread] [Top] [All Lists] |