Incomplete support for fixed format sense data

[maciej3]

File scsiata.cpp contains comments claiming that it supports fixed format response to SCSI request sense command (response code 0x70 and 0x71). Indeed, the implementation of sat_device::ata_pass_through() parses such responses correctly and returns true. However, it does not fill the descriptor buffer (adrp). Function smartcommandhandler() in atacmds.cpp calls sat_device::ata_pass_through() and fails if descriptor values are absent, specifically lba_high and lba_mid. It is not clear to me how this should be fixed:

1. Use a different logic when fixed format is returned (instead of checking lba_high and lba_mid),
2. Or, issue SCSI request sense command with the DESC bit set, which would force the descriptor format. As that may cause problems with older devices, perhaps a parameter should be introduced, settable in drivedb.h.

User-visible symptoms of this bug are:

1. smartd refuses to monitor the affected drive, producing an error such as: "Device: /dev/sdd [SAT], not capable of SMART Health Status check"
2. smartctl shows SMART info correctly, but with a complaint: "SMART STATUS RETURN: incomplete response, ATA output registers missing"

This problem was noticed with external USB drive: WD MyPassport 2TB (WDBY8L0020BBL), USB: 1058:0748

[maciej3]

smartctl -r ioctl,2 -a /dev/sdd

[Christian Franke]

The SAT (SCSI/ATA Translation) standard requires that ATA PASS-THROUGH commands with CK_COND bit set (bit 5 of CDB(2)) return sense data which includes a 13 byte ATA Return Descriptor (code 0x09).

The SAT layer of this USB bridge does not implement this properly, there is no descriptor with code 0x09 in sense data.

1. Use a different logic when fixed format is returned (instead of checking lba_high and lba_mid),

Please propose a different logic. The ATA SMART STATUS command returns its result in ATA output registers. The status IMO cannot be read if output register values are not available.

2. Or, issue SCSI request sense command with the DESC bit set, which would force the descriptor format.

There is no portable way to control the DESC bit because the actual REQUEST SENSE command is implicitly issued by the platform and controller specific SCSI pass-through I/O-control.

User-visible symptoms of this bug are:

1. smartd refuses to monitor the affected drive, producing an error such as: "Device: /dev/sdd [SAT], not capable of SMART Health Status check"

No, smartd then ignores the "-H" directive and continues to monitor the affected drive if other requested functions work.

2. smartctl shows SMART info correctly, but with a complaint: "SMART STATUS RETURN: incomplete response, ATA output registers missing"

The complaint is true and should be addressed to the authors of the USB bridge firmware :-)

[maciej3]

Replying to chrfranke:

Please propose a different logic. The ATA SMART STATUS command returns its result in ATA output registers. The status IMO cannot be read if output register values are not available.

If I knew what this logic should be, I would have included a patch. Unfortunately, my knowledge on this subject is limited to insights gained from debugging smartmontools code.

There is no portable way to control the DESC bit because the actual REQUEST SENSE command is implicitly issued by the platform and controller specific SCSI pass-through I/O-control.

Perhaps then the code would benefit from refactoring, or, if that's too much to ask, a special non-portable handling of the DESC bit could be added to the controller-specific code. I recall seeing some special cases in smartmontools code for handling quirks of specific devices, so may be such an approach would work here as well.

Regarding the DESC bit, I was so far unable to test my assumption that it changes the format of the sense data on the particular MyPassport drive. Maybe you could recommend a tool that would allow me to issue REQUEST SENSE. sg_requests claims to do that, complete with an option for the DESC bit, but I have found that it does not produce meaningful results for any of the drives on my system.

No, smartd then ignores the "-H" directive and continues to monitor the affected drive if other requested functions work.

Indeed. I have two versions of smartmontools on my system: 5.43 and 6.2. Version 5.43 refuses to monitor the affected drive, while 6.2 appears to monitor it fine. Sorry for the confusion.

The complaint is true and should be addressed to the authors of the USB bridge firmware :-)

Well, as a maintainer of smartmontools, do you have any competent contact at Western Digital that could be used to push this matter? Being a lowly user, I don't think that I would even manage to explain this issue to Western Digital tech support, and their most likely response would be simply that they don't support Linux. :(

[Christian Franke]

... a special non-portable handling of the DESC bit could be added to the controller-specific code.

The typical SCSI pass-through I/O-controls do not allow to modify the REQUEST SENSE command.

Maybe you could recommend a tool that would allow me to issue REQUEST SENSE.

To retrieve the ATA Return Descriptor, the REQUEST SENSE must be immediately issued after the ATA PASS-THROUGH command returns with a CHECK CONDITION status. This is done (and can only be done) within the I/O-control issuing the original command. It cannot be done later with user level tools.

... do you have any competent contact at Western Digital that could be used to push this matter?

Sorry, no.

[maciej3]

Replying to chrfranke:

The typical SCSI pass-through I/O-controls do not allow to modify the REQUEST SENSE command.

Please refactor your code, as it doesn't work with devices made by the leading hard drive manufacturer.

To retrieve the ATA Return Descriptor, the REQUEST SENSE must be immediately issued after the ATA PASS-THROUGH command returns with a CHECK CONDITION status. This is done (and can only be done) within the I/O-control issuing the original command. It cannot be done later with user level tools.

You may want to tell this to maintainers of sg3_utils. See "man sg_requests" for reference.

... do you have any competent contact at Western Digital that could be used to push this matter?

Sorry, no.

I sort of expected this answer. One of the reason I asked you this question was to see if you were serious about contacting the drive manufacturer, or were simply giving me a runaround. Given your answer and the general tone of your replies so far, I am inclined to think it was the latter. With this in mind, I see no reason to continue this discussion. After all, when it comes to reporting a bug, it's the maintainer attitude that counts the most, not the amount of work that I put into it.

[Christian Franke]

The typical SCSI pass-through I/O-controls do not allow to modify the REQUEST SENSE command.

Please refactor your code, as it doesn't work with devices made by the leading hard drive manufacturer.

Missing features in SCSI pass-through I/O-controls (which are part of the operating system) and/or bugs in USB bridge firmware cannot be fixed by refactoring of smartmontools code.

[Christian Franke]

This is a bug in the SAT layer of WD USB firmware: ATA Return Descriptor is not returned if CK_COND is set. This is properly detected by smartmontools. There is no easy workaround.

[Christian Franke]

Further research shows that the violation of the older SAT revisions by the WD SATL was probably intentional. Unfortunately the implementation of the newer SAT revisions is incomplete: KEY/ASC/ASCQ are correct, but the sense fields INFORMATION and COMMAND-SPECIFIC INFORMATION are all zero.

See ticket #296.