Support for Hardware raid using JMB39X

[Avi]

Hello
Would be nice to have the ability to monitor HDDs behind Jmicron Controllers JMB39X which are used in several Low budget external Raid cases.
I had found open source code which able to do that and would wish to have it as part of smartmontools.
Here is the link:
​https://github.com/Am1GO/JMraidcon

Avi

[Christian Franke]

Thanks for the info. The JMraidcon tool is apparently the result of reverse engineering and is not maintained since 2010.

The pass-through commands are issued via actual ATA READ/WRITE commands to a specific existing sector (0x21) of the logical RAID drive. This may damage data if not used carefully.

READ(0x21, old_sector);
WRITE(0x21, jm_wakeup_sector1);
WRITE(0x21, jm_wakeup_sector2);
WRITE(0x21, jm_wakeup_sector3);
WRITE(0x21, jm_wakeup_sector4);
WRITE(0x21, jm_pass_through_sector(cmd1));
READ(0x21, jm_result_sector);
WRITE(0x21, jm_pass_through_sector(cmd2));
READ(0x21, jm_result_sector);
...
WRITE(0x21, jm_pass_through_sector(cmdN));
READ(0x21, jm_result_sector);
WRITE(0x21, old_sector);

Is it safe if READ/WRITE commands from OS to other sectors are interleaved with the above sequence?

Please provide a sample output from JMraidcon and the output of smartctl -x -r ioctl,2 ... for the same drive as attachments to this ticket.

[Avi]

smartctl -x -r ioctl,2 output:

smartctl 6.5 2016-05-07 r4318 [x86_64-linux-4.4.9-300.fc23.x86_64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

[inquiry: 12 00 00 00 24 00 ]

scsi_status=0x0, host_status=0x0, driver_status=0x0
info=0x0 duration=8 milliseconds resid=0
Incoming data, len=36:

00 00 00 05 02 1f 00 00 00 4a 4d 69 63 72 6f 6e 20
10 48 2f 57 20 52 41 49 44 35 20 20 20 20 20 20 20
20 00 00 00 00
[inquiry: 12 01 00 00 fc 00 ]

scsi_status=0x0, host_status=0x0, driver_status=0x0
info=0x0 duration=1 milliseconds resid=212
Incoming data, len=40:

00 00 00 05 02 1f 00 00 00 4a 4d 69 63 72 6f 6e 20
10 48 2f 57 20 52 41 49 44 35 20 20 20 20 20 20 20
20 00 00 00 00 6a 35 35 31
[inquiry: 12 00 00 00 24 00 ]

scsi_status=0x0, host_status=0x0, driver_status=0x0
info=0x0 duration=0 milliseconds resid=0
Incoming data, len=36:

00 00 00 05 02 1f 00 00 00 4a 4d 69 63 72 6f 6e 20
10 48 2f 57 20 52 41 49 44 35 20 20 20 20 20 20 20
20 00 00 00 00

START OF INFORMATION SECTION

Vendor: JMicron
Product: H/W RAID5

[read capacity(10): 25 00 00 00 00 00 00 00 00 00 ]

scsi_status=0x0, host_status=0x0, driver_status=0x0
info=0x0 duration=0 milliseconds resid=0
Incoming data, len=8:

00 ff ff ff ff 00 00 02 00
[read capacity(16): 9e 10 00 00 00 00 00 00 00 00 00 00 00 20 00 00 ]

scsi_status=0x0, host_status=0x0, driver_status=0x0
info=0x0 duration=0 milliseconds resid=0
Incoming data, len=32:

00 00 00 00 04 17 ec ff ff 00 00 02 00 00 00 00 00
10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

User Capacity: 9,001,613,918,208 bytes [9.00 TB]
Logical block size: 512 bytes

[mode sense(6): 1a 00 84 00 40 00 ]

scsi_status=0x2, host_status=0x0, driver_status=0x8
info=0x1 duration=0 milliseconds resid=64
Incoming data trimmed to nothing by resid

Sense buffer, len=18:

00 70 00 05 00 00 00 00 0a 00 00 00 00 24 00 00 00
10 00 00

status=2: sense_key=5 asc=24 ascq=0

[mode sense(6): 1a 00 1c 00 40 00 ]

scsi_status=0x2, host_status=0x0, driver_status=0x8
info=0x1 duration=1 milliseconds resid=64
Incoming data trimmed to nothing by resid

Sense buffer, len=18:

00 70 00 05 00 00 00 00 0a 00 00 00 00 24 00 00 00
10 00 00

status=2: sense_key=5 asc=24 ascq=0

Vital Product Data (VPD) INQUIRY failed [3]
Vital Product Data (VPD) INQUIRY failed [3]
Device type: disk

[mode sense(6): 1a 00 19 00 40 00 ]

scsi_status=0x2, host_status=0x0, driver_status=0x8
info=0x1 duration=0 milliseconds resid=64
Incoming data trimmed to nothing by resid

Sense buffer, len=18:

00 70 00 05 00 00 00 00 0a 00 00 00 00 24 00 00 00
10 00 00

status=2: sense_key=5 asc=24 ascq=0

Local Time is: Thu Jun 9 10:12:37 2016 IDT

[test unit ready: 00 00 00 00 00 00 ]

scsi_status=0x0, host_status=0x0, driver_status=0x0
info=0x0 duration=0 milliseconds resid=0

SMART support is: Unavailable - device lacks SMART capability.

[unsupported field in scsi command]
[mode sense(6): 1a 00 08 00 40 00 ]

scsi_status=0x2, host_status=0x0, driver_status=0x8
info=0x1 duration=1 milliseconds resid=64
Incoming data trimmed to nothing by resid

Sense buffer, len=18:

00 70 00 05 00 00 00 00 0a 00 00 00 00 24 00 00 00
10 00 00

status=2: sense_key=5 asc=24 ascq=0

Read Cache is: Unavailable
Writeback Cache is: Unavailable

START OF READ SMART DATA SECTION

[log sense: 4d 00 40 00 00 00 00 00 04 00 ]

scsi_status=0x2, host_status=0x0, driver_status=0x8
info=0x1 duration=0 milliseconds resid=4
Incoming data trimmed to nothing by resid

Sense buffer, len=18:

00 70 00 05 00 00 00 00 0a 00 00 00 00 20 00 00 00
10 00 00

status=2: sense_key=5 asc=20 ascq=0

Log Sense for supported pages failed [unsupported scsi opcode]

[log sense: 4d 00 40 00 00 00 00 00 44 00 ]

scsi_status=0x2, host_status=0x0, driver_status=0x8
info=0x1 duration=1 milliseconds resid=68
Incoming data trimmed to nothing by resid

Sense buffer, len=18:

00 70 00 05 00 00 00 00 0a 00 00 00 00 20 00 00 00
10 00 00

status=2: sense_key=5 asc=20 ascq=0

Log Sense for supported pages failed (second attempt) [unsupported scsi opcode]

[request sense: 03 00 00 00 12 00 ]

scsi_status=0x0, host_status=0x0, driver_status=0x0
info=0x0 duration=0 milliseconds resid=0
Incoming data, len=18:

00 70 00 00 00 00 00 00 0a 00 00 00 00 00 00 00 00
10 00 00

Current Drive Temperature: 0 C
Drive Trip Temperature: 0 C

[read defect list(12): b7 0c 00 00 00 00 00 00 00 08 00 00 ]

scsi_status=0x2, host_status=0x0, driver_status=0x8
info=0x1 duration=0 milliseconds resid=8
Incoming data trimmed to nothing by resid

Sense buffer, len=18:

00 70 00 05 00 00 00 00 0a 00 00 00 00 20 00 00 00
10 00 00

status=2: sense_key=5 asc=20 ascq=0

[read defect list(10): 37 00 0c 00 00 00 00 00 04 00 ]

scsi_status=0x2, host_status=0x0, driver_status=0x8
info=0x1 duration=0 milliseconds resid=4
Incoming data trimmed to nothing by resid

Sense buffer, len=18:

00 70 00 05 00 00 00 00 0a 00 00 00 00 20 00 00 00
10 00 00

status=2: sense_key=5 asc=20 ascq=0

Read defect list (10) Failed: unsupported scsi opcode
Error Counter logging not supported

[mode sense(6): 1a 00 0a 00 40 00 ]

scsi_status=0x2, host_status=0x0, driver_status=0x8
info=0x1 duration=0 milliseconds resid=64
Incoming data trimmed to nothing by resid

Sense buffer, len=18:

00 70 00 05 00 00 00 00 0a 00 00 00 00 24 00 00 00
10 00 00

status=2: sense_key=5 asc=24 ascq=0

Device does not support Self Test logging
Device does not support Background scan results logging

[Avi]

smartctl -x -r ioctl,2 output:

[Avi]

JMraidcon out put

[Christian Franke]

The LBA 33 (0x21) is possibly selected carefully. It is never used for partitions and typically zero filled. LBA 33 is the last LBA of a GPT (entries 125-128). On disks with a MBR, LBA range 1-62 is unused or may be used for a boot manager (unlikely in this case).

[Christian Franke]

I possibly will try to add this feature when I have more time and access to a device for testing.

[Avi]

I can offer myself as a tester, just let me know what you need.

[Christian Franke]

I sent a request for a spec to fae at jmicron.com 3 month ago. JMicron sent no reply.
They were more cooperative back in 2009 when I requested the spec for the USB pass-through command.

[xrxca]

Replying to Christian Franke:

I possibly will try to add this feature when I have more time and access to a device for testing.

I know this is an older ticket, but I'd be willing to donate a 4 drive RAID that uses a JMB39X controller, and returns the same type of data from JMraidcon as in the existing attachment.

[Christian Franke]

Thanks. An alternative would be ssh access to a test machine with such a drive. Let's discuss details via private mail.

Note that the functionality will be limited. This pass-through interface truncates sector data.

It is also still unknown whether it is safe if interleaved regular I/O occurs during the pass-through READ/WRITE sequence.

[Christian Franke]

Thanks for remote access. Work is in progress.

It is also still unknown whether it is safe if interleaved regular I/O occurs during the pass-through READ/WRITE sequence.

A first test looks good.

[Avi]

Hello
I am chaining my email conversation with Janos from HD Sentinel.
He was able to use the open source code to read the JMB39X.
We had some issues they are discussed in the emails:

Hi Avi,

These are excellent questions.

As we just began testing this solution with JMRaidCon and as your RAID enclosure works
differently than my 3 devices, it is really hard to say in these situations.

In theory there should be no problem - but "in theory" accessing the 5th drive should cause no issue.

Janos

---

From: Avi Carmeli
To: 'Hard Disk Sentinel'
Sent: Friday, June 17, 2016 10:21 AM
Subject: RE: Re: Re: Re: Linux HDsentinel

Hi again

Thank you for the explanation.
If I have the tester hat I will ask few more question.
What would happen if one of the HDD completely fails?
And another one: what would happen while RAID is rebuilding?
Avi

From: Hard Disk Sentinel info@…
Sent: Friday, June 17, 2016 11:59 AM
To: Avi Carmeli
Subject: Re: Re: Re: Re: Linux HDsentinel

Hi Avi,

Thanks! I'm happy to hear things are better now :-)

So yes, it seems some JMicron chips do not like if we try to access the 5th drive, when not present.

To be honest, I have a such RAID enclosure, but all other users has only devices with 4 possible drives,
so I suspect it is completely perfect if we check only these 4 drives. Maybe if someone will contact me,
I can provide a different version which tries to enumerate the 5th drive too.

Excuse me for the troubles. As we had no information, no experience about using this unofficial detection method,
we did not know that something bad can happen. There were no warnings in the JMRaidCon source too,
I suspect the original developer also had enclosures like my ones, where he never experienced problems.


No, at the moment, it is not possible to calibrate the temperature sensor under Linux :-(
Generally if you use the calibration in the Windows version, the software saves the offset value you configured
and use in all further detections to correct the detected values.
Currently the Linux version does not have the functions to save/load and use such offset values, this is
why it is not possible now.
I'll look for possible solution and add such code to a future Linux version too.

Usually the temperature calibration is required for older hard disks, for example some early Samsung drives
(showing 7-8 Celsius lower temperature than actual) and some older WD drives (showing 20+ Celisus higher
temperature than actual)

Do you think the temperature sensor of one (or more) hard disk(s) is not accurate?
What temperatures do you see immediately after power up, after a long power OFF period (eg. 8 hours or more)?
Then all drives should show nearly same, the environment (room / office) temperature.

From the report, now I see different values (39 - 47 for the drives in the RAID box) but this may be caused
by the design of the enclosure as some drives (eg. the middle ones) may not able to cool down as the ones
on each ends of the enclosure.

Have a nice day,

Janos Mathe
developer
www.hdsentinel.com
info@…
Find us on: www.facebook.com/HDSentinel

---

From: Avi Carmeli
To: 'Hard Disk Sentinel'
Sent: Thursday, June 16, 2016 1:54 PM
Subject: RE: Re: Re: Re: Linux HDsentinel

Hi
It seems like you are right on the money.
I ran the new code without any problem.
I attached the out files.
Avi

P.S.
Is there any way to calibrate temperature sensors under Linux?


From: Hard Disk Sentinel info@…
Sent: Thursday, June 16, 2016 1:00 PM
To: Avi Carmeli
Subject: Re: Re: Re: Re: Linux HDsentinel

Hi Avi,

Yes, not surprising: when the first detection started and I suspect when the last drive position, drive #5 tried accessed,
then the JMicron chip confused - and problems started.

The updated version I sent does not try to access that drive, just detect drives on position 0-3 (where you have drives surely).
So hopefully there will be no problem with that version.

Janos

---

From: Avi Carmeli
To: 'Hard Disk Sentinel'
Sent: Wednesday, June 15, 2016 6:52 PM
Subject: RE: Re: Re: Re: Linux HDsentinel

Hi
I attach the system journal output of the relevant time.
What I found out that in both cases I sent you the files of the second try and the problem was present after the first try (./HDSentinel).
Avi

From: Hard Disk Sentinel info@…
Sent: Wednesday, June 15, 2016 6:51 PM
To: Avi Carmeli
Subject: Re: Re: Re: Re: Linux HDsentinel

Hi Avi,

Yes, of course, this is 100% known.

This is the reason why

• JMRaidCon saves and restores sector 21 in all cases, exactly to prevent damages.

• HDSentinel do the above, except that (as recommended by JMRaidCon) finds the first UNUSED

sector from sector 21 and used that, just to be 100% sure, to do not cause troubles.
So HDSentinel checked which was the first empty sector and used that - and also restored it (empty) after finish.

If you check jmbinfo.txt file, you can see these entries:

Sector 21 empty: 0 ,

Sector 22 empty: 1
Sector 22 used for communication

So there should be no issues at all: no data overwritten / cleared / modified in any ways by Hard Disk Sentinel
(and the detjm extension based on JMRaidcon)


So the problem is surely NOT this, but that the JMicron chip confused and after the detection, it does not work as should.
That resulted timeout, invalid status response on next access and maybe invalid data written / read - which
caused the corruption of the partition.


But as you never encountered problems with the original JMRaidCon (which always used sector 21 and always
tried to access only 2 drives, never tried to access non-available drives), the update I sent will work similarly:

• it will again use sector 21 (even if it's not empty - but saved and then restored)
• it will try to connect to 4 drives you have

as then hopefully we'll be safe and the JMicron RAID chip will not confused and there will be no delay, timeout, corruption, etc.

Janos

---

From: Avi Carmeli
To: 'Hard Disk Sentinel'
Sent: Wednesday, June 15, 2016 4:15 PM
Subject: RE: Re: Re: Re: Linux HDsentinel

Hi
Without any offence I came accros this :
Overwritten GPT main header or table—Various disk utilities have been written to store data in the unused (on true MBR disks) area between the MBR and the start of the first partition. Such utilities, if run on a GPT disk, might overwrite some or all of the GPT main header or partition table. You can use the 'b' and 'c' options on gdisk's experts' menu to have the program use the backup header and partition table, respectively, thus recovering from this problem
At :
​http://www.rodsbooks.com/gdisk/repairing.html

Avi

From: Hard Disk Sentinel info@…
Sent: Wednesday, June 15, 2016 4:56 PM
To: Avi Carmeli
Subject: Re: Re: Re: Re: Linux HDsentinel

Hi Avi,

I checked with other RAID box with 4 drives, formatted as ext4 etc..
see no problems, all 4 drives detected and displayed.
No corruption, no delays / timeouts even if I start multiple times.

Anyway, I made an updated version, hopefully it will work better with your device.

Please extract the file to the folder where Hard Disk Sentinel previously present, overwrite
the previous detjm file.
Maybe just to be sure, verify the file date with ls -l just to confirm that the new version is there.

Then start as you tried previously


./HDSentinel -test -r rep.txt >out.txt

and please send the files (plus the JMBINFO... files )

I keep fingers crossed, really hope things will be better now .....

Thanks for to be brave for testing ;-)

Janos

---

From: Avi Carmeli
To: 'Hard Disk Sentinel'
Sent: Wednesday, June 15, 2016 12:51 PM
Subject: RE: Re: Re: Re: Linux HDsentinel

Hi
See the files.
I am still having some issues with mounting the raid at startup. I think it is timing out.
You can send the modified code I will try it.
Avi

From: Hard Disk Sentinel info@…
Sent: Wednesday, June 15, 2016 2:25 PM
To: Avi Carmeli
Subject: Re: Re: Re: Re: Linux HDsentinel

Hi Avi,

This is so interesting.
If the original JMRaidCon worked without problems, I did not expect problems with the modified version
as it should be much safer and "light" in terms of communication with the RAID chispet.
I mean JMRaidCon performed many un-neccessary communication to illustrate possibilities but we
do not need most of them.

The modified code in Hard Disk Sentinel only try to detect only the relevant information.
Not sure, but I suspect the problem may be caused we try to detect all drives. This controller can manage
up to 5 hard disks in total, so the modified JMicron detection code (detjm used by Hard Disk Sentinel)
tries to ask all 5.

On my test devices (with 2 and 5 drives) there were no problems at all, but it is possible that the
JMicron chip in your device confused when we try to access the non-existing disk.
Then it may not work as should, for example can't manage the RAID itself (answers about corruption)
and can't properly access the drives again (returned incorrect drive identification information).

I suspect this is the answer of the delay too: timeout happened when the drive did not answer.
And this may confirm why first it works correctly (as it could detect 4 drives - and the controller
got confused when the 5th position is accessed).

Sorry for the troubles, of course if I'd know that (or there would be a warning in the JMRaidCon)
I'd not send this version for testing to you.

But as I did not encounter troubles with my test devices (even when I used 2 drives only, so there
were 3 "non-present" but accessed drives), the controllers worked correctly even after multiple retries
(even when I repeated detection once per minute).

Do you see jmbinfo.txt and jmbinfoX.txt files in the folder where Hard Disk Sentinel started?
(X = 0 .. 4, the drive positions accessed).
If so, please send them, as then I can examine the "raw" response of the JMicron chip, check
what the software got from the chip when tried to access the drives on the appropriate positions.

These would help, may contain information about the actual situation and then I can modify to code
to be more safe.

I'll also try on an other (3rd) enclosure with such chipset, just to verify if I can reproduce
and then improve in all possible ways.

And yes, if you prefer further testing, I can send an updated version which will try to access only 4 drives.
At least then we'd see if things are working differently (hopefully better) that way.

Have a nice day,

Janos Mathe
developer
www.hdsentinel.com
info@…
Find us on: www.facebook.com/HDSentinel

---

From: Avi Carmeli
To: 'Hard Disk Sentinel'
Sent: Tuesday, June 14, 2016 4:47 PM
Subject: RE: Re: Re: Re: Linux HDsentinel


Hi
Finley I was able to restore the RAID with sfdisk.
I was brave/stupid enough to retry your code.
It happened again.
The thing is that it doesn't fail immediately but after some time/ or maybe when you try to access the mount point.
I did as before :
./HDSentinel
And after:
./HDSentinel -test -r rep.txt >out.txt
At first I check after each line and seemed like ok but after few minuts when I tried to unmounted the RAID I found the old problem repeted.
Also I noticed that the ./HDSentinel needed about 30 seconds to complete which seems to me too much.
Remounting the RAID after correcting with sfdisk also took awhile.

I will do some backups and would be ready for new code from you with more trace if you will.

Avi
From: Hard Disk Sentinel info@…
Sent: Tuesday, June 14, 2016 2:00 PM
To: avi_ca
Subject: Re: Re: Re: Re: Linux HDsentinel

Hi Avi,

Did you have success with the RAID ?

I'm so sorry about any possible issues :-(
just do not really understand what could happen.

Since last evening I again did intensive testing: formatted the RAID system with ext4 (which previously formatted as NTFS under Windows)
copied / moved TB's of data and during that, constantly monitored with Hard Disk Sentinel (checking status every minute)
and had no troubles, no file (or file system) corruption or so.

Previously (before I sent the package to you) I did similar on the weekend - just then the array was formatted as NTFS.

Janos

---

From: avi_ca
To: Hard Disk Sentinel
Sent: Monday, June 13, 2016 8:43 PM
Subject: השב: Re: Re: Re: Linux HDsentinel

do you think it would be safe to use the beckup gpt table as it reads microsoft basic data which is wrong. it was ext4



Sent from Cellcom email services

---

מאת: Hard Disk Sentinel
תאריך:13/06/2016 21:19 (GMT+02:00)
אל: Avi Carmeli
נושא: Re: Re: Re: Linux HDsentinel
Hi Avi,

:-O

What happened?
I do not really understand what could be the issue.

I verified both the original JMRaidCon and Hard Disk Sentinel with this library and never encountered issues.
I checked with two different RAID enclosures (one with 2 x HDDs and one with 5 x HDDs), everything were fine,
the RAID array on it had no problems.

Moreover, the library I made based on JMRaidCon uses less detection functions (I removed most
unneccessary access of the RAID chip), plus added extra protection against troubles (improved
error handling and added check for empty sector instead of using hard-coded sector, exactly
as suggested by the original source).

So I do not really understand what could happen, why the GPT table could be corrupted/damaged.

I double verified the rep and out.txt files and I see everything completed without problems, did not see
any sign of problems, invalid operation or so.

I hope there will be no problems and the RAID can be re-mounted.

Janos

---

From: Avi Carmeli
To: 'Hard Disk Sentinel'
Sent: Monday, June 13, 2016 7:09 PM
Subject: RE: Re: Re: Linux HDsentinel

Hi
Well Now I am not able to remount the RAID.
When I run # fdisk –l /dev/sdb I get:
The primary GPT table is corrupt, but the backup appears OK, so that will be used.
Disk /dev/sdb: 8.2 TiB, 9001613918208 bytes, 17581277184 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 295BA275-A22B-4ED0-8209-066ADEFDA990
Device Start End Sectors Size Type
/dev/sdb1 2048 17581277150 17581275103 8.2T Microsoft basic data
Any advice?
Avi
From: Hard Disk Sentinel info@…
Sent: Monday, June 13, 2016 4:23 PM
To: Avi Carmeli
Subject: Re: Re: Re: Linux HDsentinel
Hi Avi,
Thanks for the file and the information !

I finished the first version, on my test system and enclosure, it can properly detect and show the status of drives in a such RAID enclosure with JMicron chipset.
If you have some time, please check this version:
www.hdsentinel.com/hdslin/hdsentinel-014-x64.zip
The zip file contain two files: HDSentinel itself plus a small library detjm .
This is designed to communicate with the JMicron device. I modified the original JMRaidCon for this purpose:
added check to detect free sector to be used for communication to be more safe.
To use it, please extract the files in a folder and set executable for both files
chmod 755 HDSentinel
chmod 755 detjm
then start ./HDSentinel
If I'm correct, the status of all drives should be detected and displayed.
For testing, I'd recommend to start this way:
./HDSentinel -test -r rep.txt >out.txt
and send the created rep.txt and out.txt
These would allow me to check the status information, verify the response of the chip and check if
further changes would be required or not. These files would especially helpful if something is not perfect.
Ps. this version is designed only to verify the situation and the results, so
please do not yet share / distribute in any ways.
Thanks!!
Have a nice day,
Janos Mathe
developer
www.hdsentinel.com
info@…
Find us on: www.facebook.com/HDSentinel

---

From: Avi Carmeli
To: 'Hard Disk Sentinel'
Sent: Tuesday, June 07, 2016 1:55 PM
Subject: RE: Re: Re: Linux HDsentinel
From: Hard Disk Sentinel info@…
Sent: Tuesday, June 7, 2016 12:40 PM
To: avi_ca
Subject: Re: Re: Re: Linux HDsentinel
Hi Avi,
To detect the status information, the RAID controller chip need to "know" how to provide the real status,
should be prepared for that.
From the viewpoint of the software we can only do reads and writes for hard disk sectors, but these information
are "outside" the addressable sectors. So the question is: how to "read" status information, if it's not on a readable sector? :-)
For this, the RAID controller has this method: if we make very special data package and write to a sector,
then the RAID controller will "know" that the next read command on this sector should provide not the sector
contents (what we wrote) but the status information we asked for. This can be the hard disk model / serial number
(to identify the drive) or its complete status (which contains the error count, temperature etc.)
For this purpose, the tool uses sector 21. This should be safe, as generally this sector is always empty:
sector 0 contains the master boot record (MBR) and then the first partition starts in a later sector (usually in sector 63
but can be different if the drive formatted under Linux).
So in theory sector 21 is always free. The test tool saves its contents and write it back after the detection finished
(just to be sure) but yes, the warning recommends that we should find an empty sector (just to be sure) and use it
instead of fixed sector 21
Just to be even more safe.
So this is what I'd need: to modify this test tool to check and find an empty sector and attempt to detect all drives.
If possible, can you please send the output of this tool when started with your device?
Just to see and compare with details I got, to confirm that both devices work similarly.
Thanks!

Janos

---

From: avi_ca
To: Hard Disk Sentinel
Sent: Monday, June 06, 2016 7:22 PM
Subject: השב: Re: Re: Linux HDsentinel
hi
thank you again for helping.
have you seen the warning in the code?
could you explain it to me?
Sent from Cellcom email services

---

מאת: Hard Disk Sentinel
תאריך:06/06/2016 18:07 (GMT+02:00)
אל: avi_ca
נושא: Re: Re: Linux HDsentinel
Hi Avi,
Thanks, this sounds excellent :-)

I'm in the middle of updating the Linux version.
I ask for some time, but hopefully this week I can build an updated version which may
be able to show the status information with this device.
Thanks for your kind help and increasing the attention to this !
Janos

---

From: avi_ca
To: Hard Disk Sentinel
Sent: Monday, June 06, 2016 4:43 PM
Subject: השב: Re: Linux HDsentinel
hi
already did. it works. I can see disks id and other data I dont understand.
Sent from Cellcom email services

---

מאת: Hard Disk Sentinel
תאריך:06/06/2016 15:25 (GMT+02:00)
אל: Avi Carmeli
נושא: Re: Linux HDsentinel
Hi Avi,
Thanks for the tip!

On the weekend, I downloaded this, compiled and it WORKS!
To be honest, I'm surprised, I did not expect this, but it seems this worked on my RAID devices
with such chipset, I could see the drives identified, their SMART data dumped, so this
tool seems working.
This is so interesting as if this works on your device too, of course we may use it,
so Hard Disk Sentinel Linux can access the disk status by a such add-on.
Do you have the opportunity to compile and try it?
I compiled under Fedora 15x64 so I have the binary, I can send it to you for testing.
BIG THANKS for your attention, to be honest, I did not expect such or similar solution
and I wonder how Werner Johansson could make it.....
Janos

---

From: Avi Carmeli
To: 'Hard Disk Sentinel'
Sent: Thursday, June 02, 2016 7:53 PM
Subject: RE: Linux HDsentinel
Hi again
While searching the net I had found that : ​http://git.xnk.nu/?p=JMraidcon.git;a=blob;f=src/JMraidcon.c;h=dd7bc689592a76cb8e1359f941e689e87575b312;hb=c07f774e7bd50c06429b644312e7fff847e00f33
Cant that help?
Avi
From: Hard Disk Sentinel info@…
Sent: Tuesday, May 31, 2016 6:31 PM
To: Avi Carmeli
Subject: Re: Linux HDsentinel
Hello Avi,
Thanks for the information and the report and thanks for your patience.
I tried some researching and performed some testing under Linux - but to be honest, with no luck.
Let me explain the situation:
Years ago, when I noticed that JMicron released the RAID controller chipsets and external
enclosures built on it become popular, I contacted JMicron about the possibilities, how
to determine the disk status information.
After some discussion, I received a closed source library for Windows which I use in Windows
version of Hard Disk Sentinel to access the status. Later I asked them for a Linux version too
as I planned to use under Linux - but to be honest, then I had no time for Linux, so I did not use it.
Now, after your message and the report (which confirmed that the Windows-specific code works
on this device) I tried the Linux version - but the code JMicron provided is not working, it is not
able to communicate with the controller (I used 100% same enclosure than yours and also
tried some other enclosures with 100% same chipsets).
The results were same: under Windows things work - but under Linux things do not work.
The problem is that JMicron did not receive any source code, just a compiled binary - which
does not work.
When I recently asked JMicron about similar situation with some other, newer RAID controller chipsets,
they answered that the original technical team (developed these code and programming interface)
already left the company and they can no longer provide support for their products
(which sounds more than weird for me).
So I'm stuck - as without an updated binary and/or a working source code (which they
would never relased) I'm afraid we can't have success, can't access the disk status information.
Personally I'd be happy to help and I asked JMicron again - but had no luck.

I'm writing this to you to please, if possible, contact JMicron or even Nexstar directly.
Please simply tell them that in your company you need to have the hard disk status automatically and periodically
logged and your developer (me :-) wants to make a such tool but this requires
programming interface / API documentation for the enclosure / chipset.
Hopefully for greater pressure they'll allow that and release some details - at least if their customers are important for them.
Or at least they may give some other details, for example a different tool which allows disk monitoring under Linux.
Sorry if I was too long / boring with the details, just wanted to describe the situation.
If you have any questions or I can help, please let me know.
Have a nice day,
Janos Mathe
developer
www.hdsentinel.com
info@…
Find us on: www.facebook.com/HDSentinel

---

From: Avi Carmeli
To: 'Hard Disk Sentinel'
Sent: Monday, May 23, 2016 7:42 PM
Subject: RE: Linux HDsentinel
Hello
Thank you for the fast replay.
I am running Fedora 64 bit.
I wasn’t able to send report from with in the HD Sentinel.
I am willing to help coding if needed , and certainly testing.
Avi
Here is the report:
Hard Disk Sentinel 4.60 PRO

[xeros]

I have the same problem with [0x152d:0x1567 (0x8034)] (ACASIS 8034) 2-bay case.
I've put there WD WD80EMAZ-00WJTA0 8TB & SAMSUNG HD250HJ 250GB (just for testing).
Small Samsung disk in bay 1 is visible correctly and all SMART data is gathered, but big WD in bay 2 isn't.
Tested with both smartctl 6.6 & 7.0, too - the same results.
Is there any work-in-progress version which may help here for testing?

$ sudo smartctl -a /dev/sda
smartctl 6.6 2017-11-05 r4594 [armv7l-linux-4.19.75-v7l+] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

/dev/sda: Unknown USB bridge [0x152d:0x1567 (0x8034)]
Please specify device type with the -d option.

Use smartctl -h to get a usage summary

$ sudo smartctl -a -d sat,auto /dev/sda
smartctl 6.6 2017-11-05 r4594 [armv7l-linux-4.19.75-v7l+] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               ACASIS
Product:
Revision:             8034
Compliance:           SPC-4
User Capacity:        8,001,563,222,016 bytes [8.00 TB]
Logical block size:   512 bytes
LU is fully provisioned
Form Factor:          3.5 inches
Logical Unit id:      0x5035333336383730
Serial number:        DB987654321116E
Device type:          disk
Local Time is:        Thu Nov 21 22:09:21 2019 GMT
SMART support is:     Unavailable - device lacks SMART capability.

=== START OF READ SMART DATA SECTION ===
Current Drive Temperature:     0 C
Drive Trip Temperature:        0 C

Error Counter logging not supported

Device does not support Self Test logging

$ sudo smartctl -a -d sat /dev/sda
smartctl 6.6 2017-11-05 r4594 [armv7l-linux-4.19.75-v7l+] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     WDC WD80EMAZ-00WJTA0
Serial Number:    ********
Firmware Version: 80003040
User Capacity:    8,001,563,222,016 bytes [8.00 TB]
Sector Size:      512 bytes logical/physical
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ATA/ATAPI-7 (minor revision not indicated)
Local Time is:    Thu Nov 21 22:06:21 2019 GMT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART Status not supported: Incomplete response, ATA output registers missing
SMART overall-health self-assessment test result: PASSED
Warning: This result is based on an Attribute check.

General SMART Values:
Offline data collection status:  (0x00)	Offline data collection activity
					was never started.
					Auto Offline Data Collection: Disabled.
Total time to complete Offline
data collection: 		(    0) seconds.
Offline data collection
capabilities: 			 (0x00) 	Offline data collection not supported.
SMART capabilities:            (0x0000)	Automatic saving of SMART data					is not implemented.
Error logging capability:        (0x00)	Error logging NOT supported.
					No General Purpose Logging support.

SMART Error Log not supported

SMART Self-test Log not supported

Selective Self-tests/Logging not supported

[Christian Franke]

Is there any work-in-progress version which may help here for testing?

First version of JMB39X support will possibly be committed next week.

[Christian Franke]

r4979.

[Christian Franke]

Does not work with JMB39X variant used in a QNAP TR-004 NAS, see ticket #1283.