WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

Re: [Xen-users] Problem with VT-d on Mainboard Intel DQ45CB

To: xen-users@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Xen-users] Problem with VT-d on Mainboard Intel DQ45CB
From: Franz Regensburger <Franz.Regensburger@xxxxxxxx>
Date: Mon, 27 Oct 2008 12:04:40 +0100
Delivery-date: Mon, 27 Oct 2008 04:05:25 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <1e16a9ed0810251710h29d3d45g54e2772f1d32a855@xxxxxxxxxxxxxx>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
References: <200810230925.04813.Franz.Regensburger@xxxxxxxx> <1e16a9ed0810251710h29d3d45g54e2772f1d32a855@xxxxxxxxxxxxxx>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: KMail/1.9.1
Hello Todd,

thank you for your comments. 
I did not have time yet to follow your suggestions (I certainly will).
> To get to the point where you can be helpful, you will need to get the
> latest xen-unstable.
>
> hg clone http://xenbits.xensource.com/xen-unstable.hg
> hg clone http://xenbits.xensource.com/linux-2.6.18-xen.hg
> cd xen-unstable
> make world

However, yesterday I was able to make some progress with respect to the Intel 
board DQ45CB that 
solves most of my problems.

It seems that most problems were caused by the board and not Xen!
I think it is important to share my new experience as early as possible in 
order to prevent others for
suffering in vain or doing unneccessary error analysis.

Some days ago Intel put a new firmware for the board onto their WEB-site
        CBQ4510H.86A.0061.BI.ZIP

I upgraded my board from CBQ4510H.86A.0059 with that new BIOS and the following 
problems seem to have vanished:

Summary:
========
Update to BIOS CBQ4510H.86A.0061.BI.ZIP solved the following issues:

- Whenever I boot my Xen 2.6.18.8 (Xen 3.3) with kernel parameter iommu=1
the boot process freezes.
SOLVED (details below)

and related to that

- VT-d broken. No PCI backend hiding with HVM guest possible 
SOLVED (details below)

- When I insert a PCIe x16 Graphics card (ATI Radeon HD 2400 Pro) the BIOS menu 
display
somehow becomes slow. The experience is like if you use a 300 Baut serial line 
connection. 
SOLVED (details below)

- PCIe x16 graphics with Xen kernel broken
SOLVED (details below)


I will also post the above short note on xen-devel

Regards,

Franz

##############################

Details:

My current setup:
Xen Source: http://bits.xensource.com/oss-xen/release/3.3.0/xen-3.3.0.tar.gz
Xen Kernel: hg clone http://xenbits.xensource.com/linux-2.6.18-xen.hg

Details for the solved issues:
======================

- Whenever I boot my Xen 2.6.18.8 (Xen 3.3) with kernel parameter iommu=1
the boot process freezes. (SOLVED)

After the BIOS update the Xen kernel boots without problems even if I use the 
iommu=1

#----------------------------

- VT-d broken. No PCI backend hiding with HVM guest possible (SOLVED)

PCI backend hideing and VT-d seem to work now!

The functionality 'pciback' is compiled fixed into my kernel (not a module). 

I use the following grub config

title           Xen 3.3.0 / kernel 2.6.18.8-xen with IOMMU=1 and backhide 1G NIC
root            (hd0,0)
kernel          /xen-3.3.0.gz console=vga noreboot max_loop=128 dom0_mem=1024M 
iommu=1
module          /vmlinuz-2.6.18.8-xen root=/dev/mapper/RootVG-root ro 
pciback.hide=(03:00.0)
module          /initrd-2.6.18.8-xen.img

The messages/dmesg show

kernel: Bootdata ok (command line is root=/dev/mapper/RootVG-root ro 
pciback.hide=(03:00.0))
...
kernel: pciback 0000:03:00.0: seizing device

The xm dmesg shows:
(XEN) Xen version 3.3.0 (root@xxxxxxxxx) (gcc version 4.1.2 20061115 
(prerelease) (Debian 4.1.1-21)) Tue Oct 21 16:22:47 CEST 2008
(XEN) Latest ChangeSet: unavailable
(XEN) Command line: console=vga noreboot max_loop=128 dom0_mem=1024M iommu=1
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN)  EDID info not retrieved because of reasons unknown
(XEN) Disc information:
(XEN)  Found 2 MBR signatures
(XEN)  Found 2 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 0000000000096400 (usable)
(XEN)  0000000000096400 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 00000000bfade000 (usable)
(XEN)  00000000bfade000 - 00000000bfb21000 (ACPI NVS)
(XEN)  00000000bfb21000 - 00000000bfc3f000 (reserved)
(XEN)  00000000bfc3f000 - 00000000bfc42000 (ACPI NVS)
(XEN)  00000000bfc42000 - 00000000bfd42000 (reserved)
(XEN)  00000000bfd42000 - 00000000bfd43000 (ACPI NVS)
(XEN)  00000000bfd43000 - 00000000bfd48000 (reserved)
(XEN)  00000000bfd48000 - 00000000bfd50000 (ACPI data)
(XEN)  00000000bfd50000 - 00000000bfd6b000 (ACPI NVS)
(XEN)  00000000bfd6b000 - 00000000bfd8a000 (reserved)
(XEN)  00000000bfd8a000 - 00000000bfd90000 (ACPI NVS)
(XEN)  00000000bfd90000 - 00000000bff00000 (usable)
(XEN)  00000000fed1c000 - 00000000fed20000 (reserved)
(XEN)  00000000ff000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 000000023c000000 (usable)
(XEN) System RAM: 8123MB (8318864kB)
(XEN) ACPI: RSDP 000F03C0, 0024 (r2  INTEL)
(XEN) ACPI: XSDT BFD4EE18, 005C (r1 INTEL  DQ45CB         3D MSFT    10013)
(XEN) ACPI: FACP BFD4DD98, 00F4 (r4  INTEL    A M I  6222004 MSFT    10013)
(XEN) ACPI: DSDT BFD48018, 4E9A (r1 INTEL  DQ45CB         3D INTL 20051117)
(XEN) ACPI: FACS BFD56F40, 0040
(XEN) ACPI: APIC BFD4DF18, 006C (r2 INTEL  DQ45CB         3D MSFT    10013)
(XEN) ACPI: MCFG BFD58E18, 003C (r1 INTEL  DQ45CB         3D MSFT       97)
(XEN) ACPI: ASF! BFD57D18, 00A0 (r32 INTEL  DQ45CB         3D TFSM    F4240)
(XEN) ACPI: SPCR BFD58D18, 0050 (r1 INTEL  DQ45CB         3D AMI.        3)
(XEN) ACPI: TCPA BFD58C98, 0032 (r2 INTEL  DQ45CB         3D MSFT  1000013)
(XEN) ACPI: DMAR BFD42F18, 00D8 (r1 INTEL  DQ45CB         3D INTL        1)
(XEN) Xen heap: 14MB (14496kB)
(XEN) Domain heap initialised
(XEN) Processor #0 7:7 APIC version 20
(XEN) Processor #1 7:7 APIC version 20
(XEN) Processor #2 7:7 APIC version 20
(XEN) Processor #3 7:7 APIC version 20
(XEN) IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) Intel VT-d has been enabled
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2833.076 MHz processor.
(XEN) HVM: VMX enabled
(XEN) CPU0: Intel(R) Core(TM)2 Quad  CPU   Q9550  @ 2.83GHz stepping 07
(XEN) Booting processor 1/1 eip 8c000
(XEN) CPU1: Intel(R) Core(TM)2 Quad  CPU   Q9550  @ 2.83GHz stepping 07
(XEN) Booting processor 2/2 eip 8c000
(XEN) CPU2: Intel(R) Core(TM)2 Quad  CPU   Q9550  @ 2.83GHz stepping 07
(XEN) Booting processor 3/3 eip 8c000
(XEN) CPU3: Intel(R) Core(TM)2 Quad  CPU   Q9550  @ 2.83GHz stepping 07
(XEN) Total of 4 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using new ACK method
(XEN) checking TSC synchronization across 4 CPUs: passed.
(XEN) Platform timer is 3.579MHz ACPI PM Timer
(XEN) Brought up 4 CPUs
(XEN) I/O virtualisation enabled
(XEN) I/O virtualisation for PV guests disabled
(XEN) mtrr: your CPUs had inconsistent fixed MTRR settings
(XEN) *** LOADING DOMAIN 0 ***
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 0xffffffff805c050c
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000230000000->0000000232000000 (253952 pages to be 
allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff80200000->ffffffff805c050c
(XEN)  Init. ramdisk: ffffffff805c1000->ffffffff81506800
(XEN)  Phys-Mach map: ffffffff81507000->ffffffff81707000
(XEN)  Start info:    ffffffff81707000->ffffffff817074a4
(XEN)  Page tables:   ffffffff81708000->ffffffff81717000
(XEN)  Boot stack:    ffffffff81717000->ffffffff81718000
(XEN)  TOTAL:         ffffffff80000000->ffffffff81800000
(XEN)  ENTRY ADDRESS: ffffffff80200000
(XEN) Dom0 has maximum 4 VCPUs
(XEN) Scrubbing Free RAM: 
.....................................................................done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to 
Xen)
(XEN) Freed 108kB init memory.
(XEN) mtrr: type mismatch for c0000000,10000000 old: write-back new: 
write-combining

In the config of the HVM WinXP guest I use:

pci=[ '03:00.0' ]

In the running guest I see a new (native) 1Gbit NIC.
I did not do any performance test yet.
However the VT-d/PCI backend hide mechanism seems to work :-)

#----------------------------

- When I insert a PCIe x16 Graphics card (ATI Radeon HD 2400 Pro) the BIOS menu 
display
somehow becomes slow.  The experience is like if you use a 300 Baut serial line 
connection. 
(SOLVED)
Details see next paragraph

- PCIe x16 graphics with Xen kernel broken
SOLVED

After the BIOS update I tested two PCIe x16 graphics adapters:
  - ATI Radeon HD 2400 Pro
  - MSI NX 8400GS (Nvidia 8400GS)
Both cards run now without problems on a non-Xen standard Debian etch 2.6.18.6 
kernel.
Especially, the BIOS menus are no longer slowed down.
3D runs perfectly with the propriatary drivers
        ATI: ati-driver-installer-8-5-x86.x86_64.run
        NVIDIA: NVIDIA-Linux-x86_64-177.80-pkg2.run

Caveat: after testing the Nvida Card/Driver you need to remove some links that 
point to
NVIDIA drivers before you test again the ATI card/driver.
Search /usr/lib for files like with '*177.80*' in them.

With the xen-kernel 2.6.18.8 booted (details about version above) I was able to 
configure the ATI card for X11
by using the ATI driver ati-driver-installer-8-5-x86.x86_64.run.
However, 3D s broken due to a problem with symbols

Oct 27 10:09:09 susi kernel: fglrx: Unknown symbol xen_invlpg_mask

With the xen-kernel 2.6.18.8 booted (details about version above) I was not 
able to compile the NVIDIA driver 
 NVIDIA-Linux-x86_64-177.80-pkg2.run
The build script from NVIDIA expclicitly states that it won't not run on a 
XEN-kernel

#-----------------------

Details about BIOS update

I download the archive
        CBQ4510H.86A.0061.BI.ZIP
from the Intel page

I had quite some hard time to install the update via the DOS-Utility IFLASH.EXE 
on a host without FD-drive
and only linux installed on the HDs. After frantically searching for some old 
Win boot media I came up with
a good old Win98SE2 boot CD. It has the nice feature to recognize USB sticks as 
FD disk drive B:

Then I proceeded as follows.
(Shame on me that I do not know how to do it better with purely Linux and 
freeDos basis
 Is anybody out there who can educate me?)

Unpack the zip archive.
This yields

CB0061.BIO
CB0061.ITK
IFLASH.EXE

Copy these files to a USB stick with FAT FS

Reboot the host and enter BIOS
Set Advanced/Drives Disk mode from RAID to IDE
This allows for a proper Win98SE2 boot from CD

Boot a Win98SE2 CD.
It will recognize the USB stick as drive b:

b:
iflash /PF CB0061.BIO

When prompted
- remove CD
- remove USB stick

type return an wait until BIOS flash has finished

It takes about 3-5 minutes



_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users