Hi Yogi
I had the strangest thing happen yesterday, which cost me a couple of hours, and thinking my hard drive failed.
To make a long story short, and to set the basis for the problem, the on-site computer I use for my main storage, is separate from the computer I do my work on. I have a 1 T Sata Drive, with a larger partition for my on-site backup.
No matter which OS I have up and running on this Computer B, because the storage area is a separate partition, I can mount the partition on the desktop using Samba to grab a file. Since this is a backup, I don't write a changed file back to it.
OK, let's move on to my daily work Computer A. It too has it's own Data Partition from which I do my daily work. It is THIS Data Partition, which is RSynced to the Data Storage Partition on Computer B, and to a Data Storage External Hard Drive down at the house. OK, AND to an External Hard Drive connected to Computer A as well. Gotta be Redundant you know, hi hi...
RSync has messed me up big time, and I really don't know why. I'll explain what is happening, and maybe you might have an idea of what went wrong.
My Normal Set-up which has never given a problem: RSync Source: /media/DataPartition RSync Destination: /OnSiteCompaq /MountCompaq
On Computer A, I created the Mountpoint "MountCompaq" and within this folder I have the folder "OnSiteCompaq"
On Computer B, The folder in the Data Storage Partition is named "OnSiteCompaq" and Shared as User, Group, and Guest, create and delete. I've NEVER had a problem writing to this folder before.
I KNOW the folder MOUNTED on Computer A is LINKED to the partition on Computer B, and as such is also Linked to the OS which created the SHARE. I run Debian 7 on all of my computers.
Now for the Problem. I installed Debian 8 on Computer B. Through the Samba Network naturally the Debian 7 shared folder on the DataStorage Partition is not available because the share source is not available.
No Biggie:
I open the Data Storage Partition on the Debian 8 desktop using Computer Places, because it is a local Partition.
Inside the Partition I open the "OnSiteCompaq" folder, NO PROBLEM.
Disaster is ready to strike.
In order to run my daily backup to "OnSiteCompaq" while on Debian 8 I SHARE the "OnSiteCompaq" folder.
I go to my Computer A and create a NEW Mount Point Folder named MountCompaqD8, and inside this new folder I make a folder named "OnSiteCompaq"
I set the RSync Destination as /OnSiteCompaq /MountCompaqD8
This way RSync should read the destination folder files and copy changed files to it.
WRONG WRONG WRONG - SOMETHING WENT HAYWIRE
RSync appeared to completely rewrite every file. However, when I went to the folder "OnSiteCompaq" on Computer B, what was written there was a three day old copy of my DataPartition. The files were there as expected, but three days old. In other words, how the "OnSiteCompaq" folder looked the last time I wrote to it while it was on the Debian 7 OS.
I checked my Computer A Mount Points and found my Original "MountCompaq" folder was MISSING. In it's place is a Document, not a folder, which I cannot open, and cannot delete, not even as root. I assume this was my original folder.
I studied the RSync man pages and on-line tutorials, and it appears you should be able to write to the same Folder using RSync from any computer.
I tried to CHOWN this oddball file so I could get rid of it.
Rebooted Computer B back into Debian 7 to see if perhaps then it would work, using the RSync Source and Destination, First I got no such folder. I tried to make a new folder, No Can Do, Folder Name Already In Use. So I created yet another new Mount Point Folder named "MountCompaqD7" and added the "OnSiteCompaq" folder in it.
Ran RSync and it appeared all the files copied, just as before. But NOTHING was in the new mount point folder this time.
Now, If I go directly to the Actual Mounted Folder on my local Computer A and look there, what was supposed to be written in those folders is there. But the folder they are supposed to be writing to remain unchanged.
Somehow I've also messed up the hard drive in the process. Can no longer boot from it, Computer Bios says SATA 0 Not Found. I swapped the SATA 0 port with another bootable drive, and it boots just fine. Swapped the wires back so my main drive is in the SATA 0 port and it says SATA 0 Drive Empty or Not found. Which is something else to fix now.
Am I missing something about RSync's ability to write to a non-OS partition? Because I use RSync through Samba instead of SSH, I DO HAVE to have the folder I want to write to SHARED, but I do not have to have it mounted in Samba on the machine I'm writing to. Provided it is shared via the OS in which I set up the SHARE. Naturally it is not accessible if I have a different OS running. But sharing the folder with the second OS seems to have messed things up royally.
Lesson Learned I suppose. Thank goodness I don't have this problem with External Hard Drives I use with RSync.
All I need to do with them is plug them in, give them a Share Name, and set RSync to find the share name and it just works as it should.
Have any Idea of why RSync says it works, No Error Message, but the files are not on the destination as they should be.
Here is the line I use to Mount the Mountpoint, which works on all but one machine.
sudo mount -t cifs //192.168.1.10/OnSiteCompaq /MountCompaq -o username=gary,iocharset=utf8,file_mode=0777,dir_mode=0777
On the External Drive which is connected to a Windows Machine, I have to add to the above
uid=gary,gid=gary,
in front of iocharset.
I'm thinking, due to the problems, that you CANNOT write to the same Folder Name on the remote computer, if it is shared by more than one OS. Even though this is not directly addressed by the man-pages or on-line tutorials.
They say you can write TO the same file on the Remote computer from any computer via Rsync.
Doesn't actually say you can share the same folder from more than one OS, although I don't see why that would matter, because it loses the sharing when you change OSs anyhow.
But using different Mount Point Folders did not correct the problem, and actually has now made matters worse.
OK, scratch your head over this one, hi hi...
TTUL
Gary
Mounted Folders from Remote Hard Drives.
Re: Mounted Folders from Remote Hard Drives.
To be perfectly honest, I don't know what you are doing there, and can't offer any sage advice. I am familiar with the theory, but you are stretching the laws of Unix Physics to their limits. rsync is one of those complex Unix/Linux commands that is a disaster waiting to happen in the hands of inexperienced users. I fall into that category and would not trust myself to backup anything important using rsync. As you know Windows is my default operating system and Microsoft rightly figures guys like me could be dangerous due to their own ignorance. Their backup programs assume idiots are running them and they take care of all the details like sharing, mounting, ownership, and permissions. Plus they don't do Linux. Fortunately, third party software writers do cater to us Windows/Linux hybrid types, but they make the same assumptions as the Redmond crowd. They figure I'm stupid and leave nothing to chance.
Having said all that, I might attempt to backup something from the command line using the dd instruction. The backup storage would naturally be shared something like your NAS with it's native Linux OS does it. It can be set up for any OS to write to the remote storage device. You might have to have server software on that remote backup device, but I don't see a problem writing to it from any kind of machine in existence. Think how Cloud storage does the same thing.
I don't know all what the dd command can do, but it's damned straight forward in its simplest form.
Also, I think you are on to something when you speculate that there may be problems sharing a single folder from two different OS's. I'd recommend you pick a server OS you like and stick to it. Don't go mixing and matching unless you like to solve problems of Einstein proportions.
As far as the stubborn folder with no name goes, I recall experiencing that exact problem. It was a long long time ago but I did eventually find a way to get rid of it. I'm thinking I booted from a USB memory stick and then was able to remove the offending file using the portable operating system. That doesn't seem logical, but I did get rid of the file eventually. You might try fsck if nothing else works.
Having said all that, I might attempt to backup something from the command line using the dd instruction. The backup storage would naturally be shared something like your NAS with it's native Linux OS does it. It can be set up for any OS to write to the remote storage device. You might have to have server software on that remote backup device, but I don't see a problem writing to it from any kind of machine in existence. Think how Cloud storage does the same thing.
I don't know all what the dd command can do, but it's damned straight forward in its simplest form.
- dd if<input file> of <output file>
Also, I think you are on to something when you speculate that there may be problems sharing a single folder from two different OS's. I'd recommend you pick a server OS you like and stick to it. Don't go mixing and matching unless you like to solve problems of Einstein proportions.
As far as the stubborn folder with no name goes, I recall experiencing that exact problem. It was a long long time ago but I did eventually find a way to get rid of it. I'm thinking I booted from a USB memory stick and then was able to remove the offending file using the portable operating system. That doesn't seem logical, but I did get rid of the file eventually. You might try fsck if nothing else works.
Re: Mounted Folders from Remote Hard Drives.
Hi Yogi
Now if there is a way to break something, I'm your man to call!
Although I have found some websites which did say you can do what I was trying to do, they were not exactly what I was trying to do.
If you are changing from one OS to another OS on the same machine, and want to be able to RSync to those folders previously Rsynced to, this creates a whole new can of worms. RSync builds and maintains a list of what files it transferred, and although it does read the remote drive to compare, it only does so on external or removable drives. This is why Rsync reads the entire drive before each comparison and transfer. But on internal hard drives, it assumes the last file it created when reading it is correct. Also, when you change OSs, the OS assigns a new UUID???? to the drive and folder.
What to do? If you change OSs on the storage computer, you MUST make a copy of the former folder, even though you can use it without doing so, you can't Rsync to it. Make a new copy of the folder and delete the old copy. Then on the machine sending the data, delete the Mount Point and create a new one. You cannot have the same folder appear in more than one Mount Point. ( I don't know why not, but that is what I found out. )
At least I learned one thing. I now know why when writing from Linux to Linux vs Linux to Windows, when writing to a Windows drive it rereads the entire file each and every time. Windows does not maintain permissions outside of the machines permission settings. So Rsync must maintain a permissions file on the sending computer and place a copy of this on the receiving computers Rsynced file, to make up for windows shortfalls.
I know you like Windows better, but Linux has much more powerful tools (in the hands of someone who knows how to use them that is, which excludes me, hi hi).
As far as copy and paste (DD), I can do that to any drive on any machine, if I have access to the drive or folder.
Because everyone uses Windows, I keep all of my Stored Files on an NTFS Drive, so when I croak, they are all readable on a Windows computer. Important document files I store in both odt and doc formats, and some in pdf format too.
I think what befuddled me the most was simply. If I have a Partition on a Hard Drive, technically it has nothing to do with whatever OS I am using. Such as those who use a separate /home partition and all OSs access it.
No matter what OS I'm using, Windows XP, Debian 7, Mint, Ubuntu, I can access ALL the files on the Partition I use for Storage.
Now if I was using SSH, perhaps I would not have had a problem. But I was using Samba to share the folder. This makes the shared folder as owned by the OS which shared it. Even so, if I reboot into a different OS, all shared folders are no longer shared. So I can share them on the OS I'm currently on, one would think, but apparently not. Because as I mentioned, I found where I must copy the contents of a previously shared folder from another OS to a new folder made by the current OS, especially if I want to Rsync to it from another computer.
Apparently there is some hidden code applied to all shared folders which does not go away, hi hi...
I do find ways to create dilemmas don't I, hi hi...
TTUL
Gary
Now if there is a way to break something, I'm your man to call!
Although I have found some websites which did say you can do what I was trying to do, they were not exactly what I was trying to do.
If you are changing from one OS to another OS on the same machine, and want to be able to RSync to those folders previously Rsynced to, this creates a whole new can of worms. RSync builds and maintains a list of what files it transferred, and although it does read the remote drive to compare, it only does so on external or removable drives. This is why Rsync reads the entire drive before each comparison and transfer. But on internal hard drives, it assumes the last file it created when reading it is correct. Also, when you change OSs, the OS assigns a new UUID???? to the drive and folder.
What to do? If you change OSs on the storage computer, you MUST make a copy of the former folder, even though you can use it without doing so, you can't Rsync to it. Make a new copy of the folder and delete the old copy. Then on the machine sending the data, delete the Mount Point and create a new one. You cannot have the same folder appear in more than one Mount Point. ( I don't know why not, but that is what I found out. )
At least I learned one thing. I now know why when writing from Linux to Linux vs Linux to Windows, when writing to a Windows drive it rereads the entire file each and every time. Windows does not maintain permissions outside of the machines permission settings. So Rsync must maintain a permissions file on the sending computer and place a copy of this on the receiving computers Rsynced file, to make up for windows shortfalls.
I know you like Windows better, but Linux has much more powerful tools (in the hands of someone who knows how to use them that is, which excludes me, hi hi).
As far as copy and paste (DD), I can do that to any drive on any machine, if I have access to the drive or folder.
Because everyone uses Windows, I keep all of my Stored Files on an NTFS Drive, so when I croak, they are all readable on a Windows computer. Important document files I store in both odt and doc formats, and some in pdf format too.
I think what befuddled me the most was simply. If I have a Partition on a Hard Drive, technically it has nothing to do with whatever OS I am using. Such as those who use a separate /home partition and all OSs access it.
No matter what OS I'm using, Windows XP, Debian 7, Mint, Ubuntu, I can access ALL the files on the Partition I use for Storage.
Now if I was using SSH, perhaps I would not have had a problem. But I was using Samba to share the folder. This makes the shared folder as owned by the OS which shared it. Even so, if I reboot into a different OS, all shared folders are no longer shared. So I can share them on the OS I'm currently on, one would think, but apparently not. Because as I mentioned, I found where I must copy the contents of a previously shared folder from another OS to a new folder made by the current OS, especially if I want to Rsync to it from another computer.
Apparently there is some hidden code applied to all shared folders which does not go away, hi hi...
I do find ways to create dilemmas don't I, hi hi...
TTUL
Gary