Hi,
The PowerVault management DVD includes a hardware snapshot provider for Windows, so 3rd party software can make the SAN take snapshots.
This is crashy and unreliable for me, and appears to have been like that for literally years.
- Has anyone else seen this?
- Does anyone know anything about configuration which is necessary to make this work reliably?
- Does anyone at Dell know how often this has been reported and why it doesn't seem to have been fixed, or why the driver can't gracefully handle problems without crashing?
I've had to abandon the dream of backups over 12Gbps SAS connections, reading at 700 MB/s, and fall back to 1Gbps network backups. Veeam can't help. Dell support can't help so far.
:(
Details below
This hardware snapshot provider is useful for backups, to trigger snapshot creation and mounting in ways that the normal smcli.exe tool doesn't support.
When I try to use it with Veeam backup and replication, it crashes out on me with exceptions in the event log. When I look around the internet, other people have problems getting it to work. I have an open support call, but since it wasn't a break-fix situation, storage ProSupport couldn't do a lot - they raised a case and it's open, but the advice so far boils down to "restart the service" (which doesn't help) or failing that, restart the storage array (ouch - but it's the driver which is crashing, not the SAN).
The program crashes with heap corruption error codes. It smells incredibly strongly like a bug in the driver. I'm working with brand new Dell servers, MD34 series storage, Windows 2012 R2, and all the latest firmwares and patches on the servers, storage controller and OS - but you can look around the internet back to 2011 and find people mudslinging about this driver:
Apparently, they know that CSV backups are not supported (buggy) in their VSS providers, and because these SANs are not even produced by DELL, and they just resell from LSI, they don't care and won't do anything to fix it.
- from www.veritas.com/.../backup-exec-2010-and-dell-md3000i-vss-provider-issues
You can find people with Server 2008 R2 having issues - in this case, the same problem with "LocateLuns" that I'm getting -> http://forums.veeam.com/microsoft-hyper-v-f25/dell-md3200i-hardware-vss-t30269.html
I tried downgrading the snapshot driver, as they did, and it made no difference in my scenario.
The hardware snapshot provider version has not changed in the July 2015, or December 2014 releases of the PowerVault management DVD - ~ 18 months.
There seem to be two main results online:
1) People who don't want SAN snapshots and/or don't have a premium snapshot license; the resolution is to uninstall it, e.g. en.community.dell.com/.../19522253
2) People who want to use it, but can't, or couldn't make it work.
There are not many options to configure in SMRegTool.exe, and I think I cycled through all reasonable combinations. The most useful Dell guide I've found is http://www.dell.com/downloads/global/products/pvaul/en/hyperv_data_protection_guide_on_%20dell_powervault_md_series.pdf which explains SMRegTool settings - "Allow unregistered initiators when masking volumes" and the reasoning behind them.
Faulting application name: SMPROV~1.EXE, version: 11.20.206.4, time stamp: 0x547cb806
Faulting module name: ntdll.dll, version: 6.3.9600.18007, time stamp: 0x55c4c16b
Exception code: 0xc0000374
Fault offset: 0x00000000000f1280
Faulting process id: 0xf40
Faulting application start time: 0x01d13149004dace9
Faulting application path: C:\PROGRA~2\Dell\MDSTOR~1\VDS_VSS\provider\SMPROV~1.EXE
Faulting module path: C:\Windows\SYSTEM32\ntdll.dll
Report Id: f3acb8e3-9d3d-11e5-80c9-ecf4bbe68ac7
Faulting package full name:
Faulting package-relative application ID:
and this:
Volume Shadow Copy Service error: Error calling a routine on a Shadow Copy Provider {24e4e8a3-69cb-4370-8b7b-c5276cd49765}. Routine details LocateLuns returned 0x80004005 [hr = 0x80004005, Unspecified error
].
Operation:
Exposing Disks
Locating shadow-copy LUNs
Import Shadow Copies
Executing Asynchronous Operation
Context:
Execution Context: Provider
Provider Name: SmVssProvider
Provider Version: 11.20.0004
Provider ID: {24e4e8a3-69cb-4370-8b7b-c5276cd49765}
And a stopped service (on multiple Hyper-V hosts / backup host), and a failed backup, and abandoned SAN snapshots I need to clear up by hand. "Exception code: 0xc0000374" appears to mean heap corruption - memory corruption in code in SMProvider.exe
Premium snapshot license costs many hundreds of pounds - who buys it, if the driver to use it with Hyper-V backups is unreliable? What else do people use it with? Or what conditions are needed to make it work and be reliable?