[Solved] Amd Machine Only Gets Iommu Enabled Whe
[SOLVED]
AMD machine only gets IOMMU enabled when setting intel_iommu=on… false positive?
-
*Thread starter
- Thread starter rhobinn
-
*Start date
- Start date Sep 29, 2022
- Forums
- Proxmox Virtual Environment
- Proxmox VE: Installation and configuration
https://forum.proxmox.com/threads/amd-machine-only-gets-iommu-enabled-when-setting-intel_iommu-on-false-positive.115920/
I have set up GPU passthrough with a few computers (all intel) but I have one that eludes me, and I don’t have a clue what’s happening:
I’m trying to get IOMMU enabled and keep failing
First, this is my hardware:
AMD ryzen 3900x
Gigabyte 570x gaming X
GeForce RTX 3080Ti GAMING X TRIO
I’m using Proxomox 7.2-3
Linux 5.15.30-2-pve #1 SMP PVE 5.15.30-3 (Fri, 22 Apr 2022 18:08:27 +0200)
My bios is updated to the last version (F37d), and have this settings (have tried them enabling and disabling each one of them one by one):
IOMMU: Enabled
CSM: Disabled
Above 4G decoding: Enabled
Resizable BAR: Disabled
ACS: Enabled
I have another GPU (cuadro p620) but that’s only for monitor output… and the bios is set to output on this one
the 3080 card seems to be on its own IOMMU Group (25)
lspci -nnv | grep VGA
04:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP107GL [Quadro P620] [10de:1cb6] (rev a1) (prog-if 00 [VGA controller]) 0a:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:2208] (rev a1) (prog-if 00 [VGA controller])
lspci -s 0a:00 -n
0a:00.0 0300: 10de:2208 (rev a1) 0a:00.1 0403: 10de:1aef (rev a1)
find /sys/kernel/iommu_groups/ -type l
/sys/kernel/iommu_groups/0/devices/0000:00:01.0 /sys/kernel/iommu_groups/1/devices/0000:00:01.2 /sys/kernel/iommu_groups/2/devices/0000:00:02.0 /sys/kernel/iommu_groups/3/devices/0000:00:03.0 /sys/kernel/iommu_groups/4/devices/0000:00:03.1 /sys/kernel/iommu_groups/5/devices/0000:00:04.0 /sys/kernel/iommu_groups/6/devices/0000:00:05.0 /sys/kernel/iommu_groups/7/devices/0000:00:07.0 /sys/kernel/iommu_groups/8/devices/0000:00:07.1 /sys/kernel/iommu_groups/9/devices/0000:00:08.0 /sys/kernel/iommu_groups/10/devices/0000:00:08.1 /sys/kernel/iommu_groups/11/devices/0000:00:14.3 /sys/kernel/iommu_groups/11/devices/0000:00:14.0 /sys/kernel/iommu_groups/12/devices/0000:00:18.3 /sys/kernel/iommu_groups/12/devices/0000:00:18.1 /sys/kernel/iommu_groups/12/devices/0000:00:18.6 /sys/kernel/iommu_groups/12/devices/0000:00:18.4 /sys/kernel/iommu_groups/12/devices/0000:00:18.2 /sys/kernel/iommu_groups/12/devices/0000:00:18.0 /sys/kernel/iommu_groups/12/devices/0000:00:18.7 /sys/kernel/iommu_groups/12/devices/0000:00:18.5 /sys/kernel/iommu_groups/13/devices/0000:01:00.0 /sys/kernel/iommu_groups/14/devices/0000:02:01.0 /sys/kernel/iommu_groups/15/devices/0000:02:02.0 /sys/kernel/iommu_groups/16/devices/0000:02:03.0 /sys/kernel/iommu_groups/17/devices/0000:02:04.0 /sys/kernel/iommu_groups/18/devices/0000:07:00.0 /sys/kernel/iommu_groups/18/devices/0000:02:08.0 /sys/kernel/iommu_groups/18/devices/0000:07:00.3 /sys/kernel/iommu_groups/18/devices/0000:07:00.1 /sys/kernel/iommu_groups/19/devices/0000:08:00.0 /sys/kernel/iommu_groups/19/devices/0000:02:09.0 /sys/kernel/iommu_groups/20/devices/0000:09:00.0 /sys/kernel/iommu_groups/20/devices/0000:02:0a.0 /sys/kernel/iommu_groups/21/devices/0000:03:00.0 /sys/kernel/iommu_groups/22/devices/0000:04:00.1 /sys/kernel/iommu_groups/22/devices/0000:04:00.0 /sys/kernel/iommu_groups/23/devices/0000:05:00.0 /sys/kernel/iommu_groups/24/devices/0000:06:00.0 /sys/kernel/iommu_groups/25/devices/0000:0a:00.0 /sys/kernel/iommu_groups/25/devices/0000:0a:00.1 /sys/kernel/iommu_groups/26/devices/0000:0b:00.0 /sys/kernel/iommu_groups/27/devices/0000:0c:00.0 /sys/kernel/iommu_groups/28/devices/0000:0c:00.1 /sys/kernel/iommu_groups/29/devices/0000:0c:00.3 /sys/kernel/iommu_groups/30/devices/0000:0c:00.4
My sistem supports remapping
dmesg | grep ‘remapping’
[ 0.370033] x2apic: IRQ remapping doesn't support X2APIC mode [ 0.635832] AMD-Vi: Interrupt remapping enabled
I added to /etc/modules the modules suggested
vfio vfio_iommu_type1 vfio_pci vfio_virqfd
I have tried different things in the fail /etc/default/grub
but if I put
GRUB_CMDLINE_LINUX_DEFAULT=”amd_iommu=on”
I get this from dmesg | grep -e DMAR -e IOMMU
[ 0.629125] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported [ 0.634719] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40 [ 0.635124] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
I have tried different options:
-quiet
-vfio-pci.ids=10de:1e81,10de:10f8,10de:1ad8,10de:1ad9
-modprobe.blacklist=radeon,nouveau,nvidia,nvidiafb,nvidia-gpu”
-
and every time I get exactly the same message
only when I add:
-pcie_acs_override=downstream,multifunction
it adds the line to the dmesg results:
[ 0.000000] Warning: PCIe ACS overrides enabled; This may allow non-IOMMU protected peer-to-peer DMA
but I cannot seem to be able to enable IOMMU
but…
by mistake I used
intel_iommu=on
and then I get
dmesg | grep -e DMAR -e IOMMU
[ 0.084082] DMAR: IOMMU enabled [ 0.627289] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported [ 0.632809] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40 [ 0.633146] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
and I don’t know if that is actually correct…
why does this happen? it is not an intel processor… why does IOMMU turns on when I set intel_iommu=on??
is it a false positive?
or that’s correct and I should use intel_iommu=on?
what should I do?
to test the GPU on the VM…
im gonna be running a simulation software GROMACS, and installing it and compiling it is long… so I haven’t gotten to it
I will try it later to see if with the intel ON setting works…
but meanwhile, what do you think is happening? why IOMMU only “works” when I set it to intel instead of AMD?? what should I do?
leesteken
Distinguished Member
May 31, 2020
6,726
1,901
228
amd_iommu is on by default.
Invalid parameters are ignored, therefore amd_iommu=on is ignored (because it is not defined) as well as intel_iommu=on (because it does not apply to your system). Resulting in amd_iommu being enabled just as it would when you don’t add any parameters.
I also think you are using the check for IOMMU being enabled for Intel systems, which does not apply.
You have several IOMMU groups (0 upto 30), so IOMMU is definitely enabled (which it is by default) and I don’t see any problems.
If you are having problems with a VM with passthrough, it’s not because IOMMU is not enabled but probably because the GPU is used during boot and by Proxmox. The various work-arounds are in this thread from several months ago.
Reactions: rhobinn
rhobinn
Member
Sep 29, 2022
5
0
6
solved here
https://forum.proxmox.com/threads/is-gpu-passthrough-working.115919/
You must log in or register to reply here.
Share:
About
The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!
Quick Navigation
Home Get Subscription Wiki Downloads Proxmox Customer Portal About
Get your subscription!
The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.