On the 9300-16i, I have been seeing the connected drives under a ‘physical drives’ menu that appears if drives are connected… on the 9305 I did not…

Would you expect the drives to spin up on boot?

I’ll try booting into proxmox and running lshw on both cards tomorrow. Thanks for the route to test!

Ok. Did a bunch more testing here tonight.

I have 2x 8tb SAS drives that have worked for several weeks now. Those show up in bios reliably regardless of which connector I use. So I think the HBA is working and the cables are good.

Of the 10x 10tb drives, 1 drive shows up reliably. It also seems to work on any connector I use. But it is the only drive the works.

Here is 8 of the 10 installed and only one showing in BIOS. I think the other 7 are not even spinning up.

An additional weird thing is that at least 1 other of the 10 drives did show up the first time I plugged it in from inside the xpenology VM. I did ‘hot swap’ that one in, but it then passed a SMART test. But since it passing the test and me pulling it, it hasn’t worked again. I’m also not positive which drive it is because originally I wasn’t expecting things to behave so weirdly, so I didn’t start taking notes…

As another test, I got a hold of a 9305-16i to see if the drives would read on that. And while that card would show up in BIOS, none of the drives – including the 8th that have always worked – showed up or spun up at all! I wonder if that card is not compatible with my mobo?

Is it possible other BIOS settings are interfering?

Ok. Did a bunch more testing here tonight.

I have 2x 8tb SAS drives that have worked for several weeks now. Those show up in bios reliably regardless of which connector I use. So I think the HBA is working and the cables are good.

Of the 10x 10tb drives, 1 drive shows up reliably. It also seems to work on any connector I use. But it is the only drive the works.

Here is 8 of the 10 installed and only one showing in BIOS. I think the other 7 are not even spinning up.

An additional weird thing is that at least 1 other of the 10 drives did show up the first time I plugged it in from inside the xpenology VM. I did ‘hot swap’ that one in, but it then passed a SMART test. But since it passing the test and me pulling it, it hasn’t worked again. I’m also not positive which drive it is because originally I wasn’t expecting things to behave so weirdly, so I didn’t start taking notes…

As another test, I got a hold of a 9305-16i to see if the drives would read on that. And while that card would show up in BIOS, none of the drives – including the 8th that have always worked – showed up or spun up at all! I wonder if that card is not compatible with my mobo?

Is it possible other BIOS settings are interfering?

Ah. Will trying to hit swap damage the drives? Or just won’t work?

Drives are hot-swappable with this HBA, right?

Will give this a go! thanks!

One of the drives for reference:

Ok. Fair enough.

I’m still working on the first 2, but on the 3rd, yes there are a couple drives that are up and working on the card. 2x 8tb drives (also HGST).

They are SAS drives, not sata. Mobo does have SATA, but not SAS.

I do!

Thanks for the link!

Yah. Trying to make an Uber server.

Bunch of drives for a nas vm

3060 for a bit of a gaming pc cam for things like moonlight.

Rx590 is for a hackintosh vm to make local backups of my iCloud stuff.

And a unix vm to manage docker containers.

Also plan to put Plex in a LXC (I think that’s the term) on Proxmox so it can hardware decode using the igpu.

Welp. Seems I’m an idiot. I’m very much a Unix noob so assumed it was something that Unix I didn’t understand rather than check the physical card again.

I was definitely shipped an nvidia card, not the amd I bid on! So case opened with eBay!

Sorry for the dumb question. Thanks for the great help.

Stay tuned for my next dumb question. :)

How do you do this? Idk what to even google, exactly.

So in the end, the intel 1g NIC just worked, so I gave up for now on trying to get the 2.5g Broadcom working instead.

I might try to link aggregate later and use the 2.5g Broadcom and circle back on this… but we’ll see.

I also got the second GPU installed and it shows up, too. But it’s an Rx590 and is showing as an RTX2070… so I’ll be making another post shortly!

Thanks for all the input!

Ohhh. In that last line. I wasn’t even looking at that, I assumed the block above that was setting up the primary NIC…

I’ll see if changing that interface name does it…

This may also help, my HBA is there:

root@prox:~# for d in /sys/kernel/iommu_groups//devices/; do n=${d#/iommu_groups/}; n=${n%%/}; printf 'IOMMU group %s ’ “$n”; lspci -nns "${d##/}"; done IOMMU group 0 00:02.0 VGA compatible controller [0300]: Intel Corporation Raptor Lake-S GT1 [UHD Graphics 770] [8086:a780] (rev 04)

IOMMU group 10 00:1f.0 ISA bridge [0601]: Intel Corporation Z690 Chipset LPC/eSPI Controller [8086:7a84] (rev 11)

IOMMU group 10 00:1f.3 Audio device [0403]: Intel Corporation Alder Lake-S HD Audio Controller [8086:7ad0] (rev 11)

IOMMU group 10 00:1f.4 SMBus [0c05]: Intel Corporation Alder Lake-S PCH SMBus Controller [8086:7aa3] (rev 11)

IOMMU group 10 00:1f.5 Serial bus controller [0c80]: Intel Corporation Alder Lake-S PCH SPI Controller [8086:7aa4] (rev 11)

IOMMU group 10 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (17) I219-V [8086:1a1d] (rev 11)

IOMMU group 11 01:00.0 Non-Volatile memory controller [0108]: Sandisk Corp Western Digital WD Black SN850X NVMe SSD [15b7:5030] (rev 01)

IOMMU group 12 02:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)

IOMMU group 13 03:00.0 Non-Volatile memory controller [0108]: Micron/Crucial Technology Device [c0a9:5415] (rev 01)

IOMMU group 14 04:00.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8724 24-Lane, 6-Port PCI Express Gen 3 (8 GT/s) Switch, 19 x 19mm FCBGA [10b5:8724] (rev ca)

IOMMU group 15 05:00.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8724 24-Lane, 6-Port PCI Express Gen 3 (8 GT/s) Switch, 19 x 19mm FCBGA [10b5:8724] (rev ca)

IOMMU group 16 05:08.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8724 24-Lane, 6-Port PCI Express Gen 3 (8 GT/s) Switch, 19 x 19mm FCBGA [10b5:8724] (rev ca)

IOMMU group 17 05:09.0 PCI bridge [0604]: PLX Technology, Inc. PEX 8724 24-Lane, 6-Port PCI Express Gen 3 (8 GT/s) Switch, 19 x 19mm FCBGA [10b5:8724] (rev ca)

**IOMMU group 18 06:00.0 Serial Attached SCSI controller [0107]: Broadcom / LSI SAS3008 PCI-Express Fusion-MPT SAS-3 [1000:0097] (rev 02)

IOMMU group 19 08:00.0 Serial Attached SCSI controller [0107]: Broadcom / LSI SAS3008 PCI-Express Fusion-MPT SAS-3 [1000:0097] (rev 02)**

IOMMU group 1 00:00.0 Host bridge [0600]: Intel Corporation Device [8086:a740] (rev 01)

IOMMU group 2 00:14.0 USB controller [0c03]: Intel Corporation Alder Lake-S PCH USB 3.2 Gen 2x2 XHCI Controller [8086:7ae0] (rev 11)

IOMMU group 2 00:14.2 RAM memory [0500]: Intel Corporation Alder Lake-S PCH Shared SRAM [8086:7aa7] (rev 11)

IOMMU group 3 00:15.0 Serial bus controller [0c80]: Intel Corporation Alder Lake-S PCH Serial IO I2C Controller #0 [8086:7acc] (rev 11)

IOMMU group 4 00:16.0 Communication controller [0780]: Intel Corporation Alder Lake-S PCH HECI Controller #1 [8086:7ae8] (rev 11)

IOMMU group 5 00:17.0 SATA controller [0106]: Intel Corporation Alder Lake-S PCH SATA Controller [AHCI Mode] [8086:7ae2] (rev 11)

IOMMU group 6 00:1a.0 PCI bridge [0604]: Intel Corporation Alder Lake-S PCH PCI Express Root Port #25 [8086:7ac8] (rev 11)

IOMMU group 7 00:1c.0 PCI bridge [0604]: Intel Corporation Alder Lake-S PCH PCI Express Root Port #2 [8086:7ab9] (rev 11)

IOMMU group 8 00:1c.4 PCI bridge [0604]: Intel Corporation Alder Lake-S PCH PCI Express Root Port #5 [8086:7abc] (rev 11)

IOMMU group 9 00:1d.0 PCI bridge [0604]: Intel Corporation Alder Lake-S PCH PCI Express Root Port #9 [8086:7ab0] (rev 11)

sorry this took so long… you know, life. trying that command altogether, I get this response: -bash: acpidump: command not found

trying just egrep “DMAR|IVRS” (in case they are two commands) seems to hang the terminal session.

I tried following a guide to enable PICe passthrough and get this. One important thing, there is no discrete GPU at the moment, I’m trying to pass through an HBA…

root@prox:~# dmesg | grep -e IOMMU [ 0.100411] DMAR: IOMMU enabled [ 0.254862] DMAR-IR: IOAPIC id 2 under DRHD base 0xfed91000 IOMMU 1 [ 0.629143] pci 0000:00:02.0: DMAR: Skip IOMMU disabling for graphics [ 0.713978] DMAR: IOMMU feature fl1gp_support inconsistent [ 0.713979] DMAR: IOMMU feature pgsel_inv inconsistent [ 0.713980] DMAR: IOMMU feature nwfs inconsistent [ 0.713981] DMAR: IOMMU feature dit inconsistent [ 0.713982] DMAR: IOMMU feature sc_support inconsistent [ 0.713983] DMAR: IOMMU feature dev_iotlb_support inconsistent

This is in proxmox?

How can I tell if my mobo even supports it?

Ok. So they are different?

How do I tell which motherboards support IOMMU?

I can’t find it as a filter or search option on any websites…?

Well, that’s at least part of my problem. I have no idea what it would be called so it’s hard to google. I guess the underlying technology is all ‘IOMMU’, but each motherboard manufacture and Intel and AMD all have other names for it…

I’m trying to pass an HBA through to a VM for a NAS.