Hard drive.... bandwidth?

Quick to answer questions about finding your way around Linux Mint as a new user.
Forum rules
There are no such things as "stupid" questions. However if you think your question is a bit stupid, then this is the right place for you to post it. Stick to easy to-the-point questions that you feel people can answer fast. For long and complicated questions use the other forums in the support section.
Before you post read how to get help. Topics in this forum are automatically closed 6 months after creation.
Locked
fumbles22

Hard drive.... bandwidth?

Post by fumbles22 »

Hello,

Over the past two days i've developed an odd problem on linux mint 17.

In my computer setup, I have a 320GB drive that I boot from and a 2TB drive that I store videos and music on. I have set my 2TB drive to load itself on startup using "NTFS Configuration tool".

Oddly though, I seem to have a really bad connection to this hard drive. I download torrents directly to this drive, and the download speed changes from 720KB per second to 0. It will then stay on 0 for a while and then increase back up to 720Kb again.

Similarly, when I play music, the music will suddenly cut out (in time with the download fluctuations) and then start again.

The same is true with video files.

In order to test if it was a problem with the hard drive, I copied and pasted a file from the 2TB hard drive to the 320GB boot drive. It played perfectly, without any problems.

Should I buy new data cables and switch the power cable to the hard drive, or is there a linux command I can run that will help me diagnose the problem?
Last edited by LockBot on Wed Dec 28, 2022 7:16 am, edited 1 time in total.
Reason: Topic automatically closed 6 months after creation. New replies are no longer allowed.
User avatar
xenopeek
Level 25
Level 25
Posts: 29510
Joined: Wed Jul 06, 2011 3:58 am

Re: Hard drive.... bandwidth?

Post by xenopeek »

NTFS support isn't perfect; is it an option to use a Linux filesystem instead? Might work better.
Image
nerdtron

Re: Hard drive.... bandwidth?

Post by nerdtron »

Can you post the output of "mount" command.
Also post the contents of your /etc/fstab just to see the mount options used to mount the partition.

Anyway, to properly test the hard drive, you can try connecting it on a windows computer (since it is NTFS) and then run hard drive diagnostic tools to see the health of the drive.
Mute Ant

Re: Hard drive.... bandwidth?

Post by Mute Ant »

Such a low rate suggests the data connection is 'stuck' at USB1 speeds. You can test the readability of each drive you have connected, ignoring the filesystem issue, like this...

Code: Select all

sudo clear ; sudo cat /dev/sda | pv > /dev/zero
The pv(1) command may not be in your system, just install it...

Code: Select all

sudo apt-get install pv
Let it run for a few seconds until the rate settles down, cached data can make it look faster or slower than it really is. CTRL-C will stop the command early.
Then try the command with /dev/sdX [replacing X with the handle of the remote drive].

EDIT Improved code...does the same with less typing...

Code: Select all

sudo pv /dev/sda > /dev/zero
Last edited by Mute Ant on Sun Nov 23, 2014 10:28 am, edited 1 time in total.
fumbles22

Re: Hard drive.... bandwidth?

Post by fumbles22 »

Hello again,

This is the outcome of the mount command (name and computer name changed):

Code: Select all

name@computer ~ $ mount
/dev/sda1 on / type ext4 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/cgroup type tmpfs (rw)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
none on /run/user type tmpfs (rw,noexec,nosuid,nodev,size=104857600,mode=0755)
none on /sys/fs/pstore type pstore (rw)
/dev/sdb1 on /media/Emmett_Brown type fuseblk (rw,nosuid,nodev,allow_other,blksize=4096)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noexec,nosuid,nodev)
systemd on /sys/fs/cgroup/systemd type cgroup (rw,noexec,nosuid,nodev,none,name=systemd)
gvfsd-fuse on /run/user/1000/gvfs type fuse.gvfsd-fuse (rw,nosuid,nodev,user=philip)
You can see that my 2TB drive is called Emmett Brown after Doc Brown on Back to the Future.

Here are the contents of my etc/fstab file:

Code: Select all

# /etc/fstab: static file system information.
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>

#Entry for /dev/sda1 :
UUID=c53c76a3-c6a0-4249-9806-7b56e72e365a	/	ext4	errors=remount-ro	0	1
#Entry for /dev/sdb1 :
UUID=325ED1F424F1D86E	/media/Emmett_Brown	ntfs-3g	defaults,locale=en_GB.UTF-8	0	0
#Entry for /dev/sda5 :
UUID=567cd368-c79d-4231-89ab-d7e3b10483c2	none	swap	sw	0	0
Here's the outcome of checking the 2Tb drive (Deluge is downloading onto this drive. This is the only program using it):

Code: Select all

name@computer ~ $ sudo clear ; sudo cat /dev/sdb1 | pv > /dev/zero
[sudo] password for name: 

cat: /dev/sdb1: Input/output error  <=>                                        ]
38.2GB 0:18:25 [35.4MB/s] [          <=>    
Aha! This appears to mean something!

I'll run it again (deluge is not on in this instance, so there aren't any programs using the 2tb drive):

Code: Select all

name@computer ~ $ sudo clear ; sudo cat /dev/sdb1 | pv > /dev/zero
[sudo] password for name: 

cat: /dev/sdb1: Input/output error          <=>                                ]
38.2GB 0:07:10 [90.7MB/s] [                  <=>   
Aha! We get the same result. It appears that there's a problem with the 2Tb drive. To my novice eye, it appears that the Input/output error happens at the same place on the drive. This means that it is not a problem with the cables connecting to the drive (someone confirm?).

How would I go about fixing this problem? Could I:

1). Copy and paste my files from the drive to an external hard drive
2). Reformat the drive to some linux extension (exFAT?)
3). Copy everything back again

?

I did run the same check on the 320Gb drive, but I think my computer hibernated and it stopped running. Either way, I think we've found the problem. I'll run the check later on my 320Gb drive when i'll be using it for longer.
fumbles22

Re: Hard drive.... bandwidth?

Post by fumbles22 »

Hello again,

I found this page that is all about fixing bad blocks on linux.

However, when I try to run it, I get this:

Code: Select all

name@computer ~ $ sudo e2fsck -cfpv /dev/sdb1
e2fsck: Bad magic number in super-block while trying to open /dev/sdb1
/dev/sdb1: 
The superblock could not be read or does not describe a valid ext2/ext3/ext4
filesystem.  If the device is valid and it really contains an ext2/ext3/ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>
 or
    e2fsck -b 32768 <device>
I am right? As far as I can tell, my 2Tb hard drive (Emmett Brown) is mounted on sdb1?

I've tried running the code with the hard drive mounted too. It asks me to unmount it, and I end up with the console output as above.

Can anyone help?
Mute Ant

Re: Hard drive.... bandwidth?

Post by Mute Ant »

e2fsck is for checking ext2/3/4 file systems, your /dev/sdb1 is NTFS...
  • #Entry for /dev/sdb1 :
    UUID=325ED1F424F1D86E /media/Emmett_Brown ntfs-3g
...so you got the 'correct' result there, it doesn't work. I have never used NTFS; what I read is that Windows tools are preferred for check and repair...they invented it after all.

That I/O error reading /dev/sdb1...I would ensure any impossible-to-replace data is duplicated to a physically different device, right now. If you are lucky, a simple chkdsk or scandisk that includes read-write tests will fix it. If you are unlucky, the disk really is growing defects and no amount of fiddling will heal it.
Last edited by Mute Ant on Mon Nov 24, 2014 1:43 pm, edited 1 time in total.
fumbles22

Re: Hard drive.... bandwidth?

Post by fumbles22 »

I'm in the process of removing files from it at the moment. I've had a few Input/Output errors, but i'm going to get as much from it as I can. The only valuable files that i've got are my music ones. At the moment it will take another 4 hours.

I think I will reformat the drive to a linux filesystem instead. Would it be a good idea to format both my 320GB boot drive and 2Tb hard drive to ext4? Would I see any performance benefits?
niowluka

Re: Hard drive.... bandwidth?

Post by niowluka »

Code: Select all

sudo apt-get install smartmontools
sudo smartctl -a /dev/sdb
This will run some basic diagnostics on the drive, should give us idea if the drive is failing (which I'm guessing it does). If it's the drive failing, changing filesystem won't make a difference.
deleted

Re: Hard drive.... bandwidth?

Post by deleted »

With that size drive, and if you aren't going to share it with say Windows or Mac, I'd put jfs on it.
-Hinto
fumbles22

Re: Hard drive.... bandwidth?

Post by fumbles22 »

I ran smartmontools, and this was the result:

Code: Select all

name@computer ~ $ sudo smartctl -a /dev/sdb
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.13.0-24-generic] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     ST2000DL004 HD204UI
Serial Number:    S2H7J90C305730
LU WWN Device Id: 5 0004cf 20728bf33
Firmware Version: 1AQ10001
User Capacity:    2,000,398,934,016 bytes [2.00 TB]
Sector Size:      512 bytes logical/physical
Rotation Rate:    5400 rpm
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ATA8-ACS T13/1699-D revision 6
SATA Version is:  SATA 2.6, 3.0 Gb/s
Local Time is:    Mon Nov 24 17:01:10 2014 GMT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00)	Offline data collection activity
					was never started.
					Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0)	The previous self-test routine completed
					without error or no self-test has ever 
					been run.
Total time to complete Offline 
data collection: 		(20940) seconds.
Offline data collection
capabilities: 			 (0x5b) SMART execute Offline immediate.
					Auto Offline data collection on/off support.
					Suspend Offline collection upon new
					command.
					Offline surface scan supported.
					Self-test supported.
					No Conveyance Self-test supported.
					Selective Self-test supported.
SMART capabilities:            (0x0003)	Saves SMART data before entering
					power-saving mode.
					Supports SMART auto save timer.
Error logging capability:        (0x01)	Error logging supported.
					General Purpose Logging supported.
Short self-test routine 
recommended polling time: 	 (   2) minutes.
Extended self-test routine
recommended polling time: 	 ( 349) minutes.
SCT capabilities: 	       (0x003f)	SCT Status supported.
					SCT Error Recovery Control supported.
					SCT Feature Control supported.
					SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   100   100   051    Pre-fail  Always       -       6529
  2 Throughput_Performance  0x0026   252   252   000    Old_age   Always       -       0
  3 Spin_Up_Time            0x0023   066   065   025    Pre-fail  Always       -       10494
  4 Start_Stop_Count        0x0032   096   096   000    Old_age   Always       -       4513
  5 Reallocated_Sector_Ct   0x0033   252   252   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   252   252   051    Old_age   Always       -       0
  8 Seek_Time_Performance   0x0024   252   252   015    Old_age   Offline      -       0
  9 Power_On_Hours          0x0032   100   100   000    Old_age   Always       -       4024
 10 Spin_Retry_Count        0x0032   252   252   051    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   252   252   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   097   097   000    Old_age   Always       -       3505
181 Program_Fail_Cnt_Total  0x0022   096   096   000    Old_age   Always       -       88583348
191 G-Sense_Error_Rate      0x0022   100   100   000    Old_age   Always       -       244
192 Power-Off_Retract_Count 0x0022   252   252   000    Old_age   Always       -       0
194 Temperature_Celsius     0x0002   064   064   000    Old_age   Always       -       21 (Min/Max 6/40)
195 Hardware_ECC_Recovered  0x003a   100   100   000    Old_age   Always       -       0
196 Reallocated_Event_Count 0x0032   252   252   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   100   100   000    Old_age   Always       -       60
198 Offline_Uncorrectable   0x0030   252   252   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0036   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x002a   100   100   000    Old_age   Always       -       41
223 Load_Retry_Count        0x0032   252   252   000    Old_age   Always       -       0
225 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       4569

SMART Error Log Version: 1
ATA Error Count: 212 (device log contains only the most recent five errors)
	CR = Command Register [HEX]
	FR = Features Register [HEX]
	SC = Sector Count Register [HEX]
	SN = Sector Number Register [HEX]
	CL = Cylinder Low Register [HEX]
	CH = Cylinder High Register [HEX]
	DH = Device/Head Register [HEX]
	DC = Device Command Register [HEX]
	ER = Error register [HEX]
	ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 212 occurred at disk power-on lifetime: 4024 hours (167 days + 16 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 08 1f 14 5c e6  Error: UNC 8 sectors at LBA = 0x065c141f = 106697759

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  c8 00 08 1f 14 5c e6 08      00:00:32.688  READ DMA
  27 00 00 00 00 00 e0 08      00:00:32.688  READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]
  ec 00 00 00 00 00 a0 08      00:00:32.688  IDENTIFY DEVICE
  ef 03 46 00 00 00 a0 08      00:00:32.688  SET FEATURES [Set transfer mode]
  27 00 00 00 00 00 e0 08      00:00:32.688  READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]

Error 211 occurred at disk power-on lifetime: 4024 hours (167 days + 16 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 08 1f 14 5c e6  Error: UNC 8 sectors at LBA = 0x065c141f = 106697759

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  c8 00 08 1f 14 5c e6 08      00:00:32.688  READ DMA
  25 00 08 a7 96 2c e0 08      00:00:32.688  READ DMA EXT
  25 00 08 9f 96 2c e0 08      00:00:32.688  READ DMA EXT
  25 00 08 97 96 2c e0 08      00:00:32.688  READ DMA EXT
  25 00 08 8f 96 2c e0 08      00:00:32.688  READ DMA EXT

Error 210 occurred at disk power-on lifetime: 4024 hours (167 days + 16 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 07 d0 b3 c5 e4  Error: UNC 7 sectors at LBA = 0x04c5b3d0 = 80065488

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  c8 00 08 cf b3 c5 e4 08      00:00:32.613  READ DMA
  25 00 08 f7 95 82 e0 08      00:00:32.613  READ DMA EXT
  25 00 08 ef 95 82 e0 08      00:00:32.613  READ DMA EXT
  25 00 08 e7 95 82 e0 08      00:00:32.613  READ DMA EXT
  25 00 08 df 95 82 e0 08      00:00:32.613  READ DMA EXT

Error 209 occurred at disk power-on lifetime: 4024 hours (167 days + 16 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 07 d0 b3 c5 e4  Error: UNC 7 sectors at LBA = 0x04c5b3d0 = 80065488

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  c8 00 08 cf b3 c5 e4 08      00:00:32.608  READ DMA
  27 00 00 00 00 00 e0 08      00:00:32.608  READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]
  ec 00 00 00 00 00 a0 08      00:00:32.608  IDENTIFY DEVICE
  ef 03 46 00 00 00 a0 08      00:00:32.608  SET FEATURES [Set transfer mode]
  27 00 00 00 00 00 e0 08      00:00:32.608  READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]

Error 208 occurred at disk power-on lifetime: 4024 hours (167 days + 16 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 07 d0 b3 c5 e4  Error: UNC 7 sectors at LBA = 0x04c5b3d0 = 80065488

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  c8 00 08 cf b3 c5 e4 08      00:00:32.603  READ DMA
  25 00 08 f7 93 82 e0 08      00:00:32.603  READ DMA EXT
  25 00 08 ef 93 82 e0 08      00:00:32.603  READ DMA EXT
  25 00 08 e7 93 82 e0 08      00:00:32.603  READ DMA EXT
  25 00 08 df 93 82 e0 08      00:00:32.603  READ DMA EXT

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]


SMART Selective self-test log data structure revision number 0
Note: revision number not 1 implies that no selective self-test has ever been run
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Completed [00% left] (0-65535)
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
I'm not sure what it all means, but the "SMART overall-health self-assessment test result: PASSED" tells me that it's good news (?).
With that size drive, and if you aren't going to share it with say Windows or Mac, I'd put jfs on it.
-Hinto
I'm going to reformat my current hard drive as well (my 320Gb one). The main reason i'd like to do this is to experiment with some of the linux filesystems.

Should I also format my 320Gb boot drive to jfs?
deleted

Re: Hard drive.... bandwidth?

Post by deleted »

You can, but not really necessary.
JFS just allows fast deletes of very large files (like movies, etc)

GParted has some jfs support, most distros will ship with it for live sessions.
The main thing, for the external drive, is you can apply permissions to the files whereas ntfs will not.
ntfs drive permissions are set during mount (command line or say /etc/fstab)
-Hinto

Edit:
the command line has a whole set of jfs utilities. It's been around for quite a while.
Last edited by deleted on Mon Nov 24, 2014 1:34 pm, edited 1 time in total.
niowluka

Re: Hard drive.... bandwidth?

Post by niowluka »

fumbles22 wrote:I'm not sure what it all means, but the "SMART overall-health self-assessment test result: PASSED" tells me that it's good news (?).
Yeah, 209 errors say it's not, though. This drive won't run for much longer.
fumbles22 wrote:Should I also format my 320Gb boot drive to jfs?
By all means. JFS is my favourite filesystem, I use it for both OS and data. I also use Btrfs on some SSD partitions.
fumbles22

Re: Hard drive.... bandwidth?

Post by fumbles22 »

Is there any way to know whether the errors are logical or physical? Is reformatting it and copying and pasting files onto it the only way to know? I'm just asking since it would be a shame to buy a new hard drive, only to realise that my old one was repairable.

Also, i'm definitely going to have a JFS boot and storage drive even if I need to buy a new storage drive. Since i'm using linux, it makes sense to use a linux filesystem.
niowluka

Re: Hard drive.... bandwidth?

Post by niowluka »

fumbles22 wrote:Is there any way to know whether the errors are logical or physical?
smartctl reads disk's S.M.A.R.T. data, it's hardware and your disk is going down.
Mute Ant

Re: Hard drive.... bandwidth?

Post by Mute Ant »

''I'm not sure what it all means,'' http://www.thomas-krenn.com/en/wiki/Ana ... g_Smartctl They don't make it obvious do they? SMART uses the French-Dictation scoring system. Each parameter starts with a full score, each confirmed failure causes it to lose-some-score and reduce towards zero, until eventually the accumulated errors trigger a warning. I just look at the Raw Value...
  • 197 Current_Pending_Sector 60 :shock:
There's 60 sectors the drive has blacklisted as unreadable [now] even though they were writable [then]. So there's your I/O errors explained. It's internal to the drive.

With your data backed up, this is worth a try before you discard it as scrap...don't do this as a hobby...never do this on a SSD...

Code: Select all

sudo su -c 'x=/dev/sdb ; dd if=$x conv=sync,noerror | pv > $x'
  • sudo su -c..................Run the string as a command with root privilege
    x=/dev/sdb.................The handle of the device to be scandisk'ed
    dd if=$x......................Read the whole of the device
    conv=sync,noerror.......Ignore errors, pad missing data with zeros
    | pv.............................Show progress
    > $x............................Write the data back where it came from
This rewrites every sector on a drive, giving the firmware another chance at writing to the bad ones or remapping them. Bad sectors are replaced with zeros, probably no worse than them being unreadable.
Locked

Return to “Beginner Questions”