how to recover filesystem after clobbering array?

Hello people.

I've messed up good, and now I need you nice folks to help me recover
500GB of irreplaceable home video, the full 7 years of my family
trove. :(

I mistakenly used mdadm to *create* an array instead of *starting* the
array (big Duh!). Now the array has no partition table. I shudder to
think I may have clobbered not only the array but also the file
system. I hardly slept last night. Acceptance is a process. :/

I've learned my lesson there, no more fiddling with "--assume-clean"
(which was a stupid idea in the first place), but the issue remains:
how do I get to the file system that I know is still there? The array
is up but obviously it was never mounted.

Can I just recreate the partition table and it works?.... or do I have
to use some complex form of forensics to recover the data?

Thank you very much,
Vasco.


--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: how to recover filesystem after clobbering array?

Oh, the type of array is Raid 1.

Citando Vasco N=E9voa :

>
> Hello people.
>
> I've messed up good, and now I need you nice folks to help me =20
> recover 500GB of irreplaceable home video, the full 7 years of my =20
> family trove. :(
>
> I mistakenly used mdadm to *create* an array instead of *starting* =20
> the array (big Duh!). Now the array has no partition table. I =20
> shudder to think I may have clobbered not only the array but also =20
> the file system. I hardly slept last night. Acceptance is a process. =
=20
> :/
>
> I've learned my lesson there, no more fiddling with "--assume-clean" =
=20
> (which was a stupid idea in the first place), but the issue remains: =
=20
> how do I get to the file system that I know is still there? The =20
> array is up but obviously it was never mounted.
>
> Can I just recreate the partition table and it works?.... or do I =20
> have to use some complex form of forensics to recover the data?
>
> Thank you very much,
> Vasco.
>
>
>


--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: how to recover filesystem after clobbering array?

On 19/07/2011 11:43, Vasco N=E9voa wrote:
>
> Hello people.
>
> I've messed up good, and now I need you nice folks to help me recover
> 500GB of irreplaceable home video, the full 7 years of my family trov=
e. :(
>
> I mistakenly used mdadm to *create* an array instead of *starting* th=
e
> array (big Duh!). Now the array has no partition table. I shudder to
> think I may have clobbered not only the array but also the file syste=
m.
> I hardly slept last night. Acceptance is a process. :/
>
> I've learned my lesson there, no more fiddling with "--assume-clean"
> (which was a stupid idea in the first place), but the issue remains: =
how
> do I get to the file system that I know is still there? The array is =
up
> but obviously it was never mounted.
>
> Can I just recreate the partition table and it works?.... or do I hav=
e
> to use some complex form of forensics to recover the data?
>
> Thank you very much,
> Vasco.
>

The first thing you should do is write out "RAID is not a backup=20
solution" 100 times!

Then write down all the information you know - exactly how was the arra=
y=20
built up, how were the disks partitioned, what filesystems, etc. And=20
what was the exact command you used to mess up the array, and how far=20
did you let it go with resynchronisation, etc. Make good notes here=20
before you forget.

The next step is to get a couple spare disks that are bigger than your=20
original disks (get 2 TB disks - they cost almost the same as anything=20
smaller). Make a direct copy of the entire original disks to a file on=
=20
the new disks with something like:

dd if=3D/dev/sda of=3D/mnt/backup/disk1.img bs=3D1M

Once you have got image files for each of your disks, make copies of=20
these image files to another spare disk. Keep careful notes of exactly=
=20
what you have done here, and which files are which. And put your=20
original disks, carefully labelled, on a shelf somewhere.

Now you are in a position to attempt data recovery on your copied files=
.
If you do something wrong, you can simply re-copy the image files and=
=20
try again. You still have absolutely no guarantees that you'll get=20
anything back - but at least you can be sure you are not going to make=20
anything worse.

If you have started re-syncing the two disks as a RAID1 pair, there is =
a=20
good chance that one of the disks contains the original data and=20
filesystem, except where it was overwritten by the new metadata for the=
=20
newly created array.

Then you sit back and hope that someone on this list can give you ideas=
=20
about getting the data back from the image file(s).


--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: how to recover filesystem after clobbering array?

On 2011-07-19 11:17, David Brown wrote:
> Once you have got image files for each of your disks, make copies of
> these image files to another spare disk. Keep careful notes of exactl=
y
> what you have done here, and which files are which. And put your
> original disks, carefully labelled, on a shelf somewhere.
>
> Now you are in a position to attempt data recovery on your copied
> files. If you do something wrong, you can simply re-copy the image
> files and try again. You still have absolutely no guarantees that
> you'll get anything back - but at least you can be sure you are not
> going to make anything worse.

=46ollow David's advice.

What was the filesystem on the array?

Now, use testdisk, photorec, and foremost to seek through the raw image=
s
and extract files. The good news is, most video formats are detectable
by these tools. All can be installed with your package manager.

Regards,
Tyler

--=20
"Offending fundamentalists isn't my goal â€=93 but if it is an inev=
itable
side-effect of defending human rights, so be it."
-- Johann Hari

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: how to recover filesystem after clobbering array?

Thank you very much, Tyler and David.
Good advice.

The array is level 1, built upon primary partitions of the devices.
The file system is EXT4.
The mdadm command that I stupidly used to create instead of starting
the array included "--assume-clean" but no "--build".
I checked via /proc/mdadm that the array came up without syncing
anything (or at least it was ultra-fast, less than 5 seconds).
So I firmly believe the data is all there, on both disks.
I just need to rebuild the metadata to point to the data again somehow.
Right?...

Citando "Tyler J. Wagner" :

> On 2011-07-19 11:17, David Brown wrote:
>> Once you have got image files for each of your disks, make copies of
>> these image files to another spare disk. Keep careful notes of exactly
>> what you have done here, and which files are which. And put your
>> original disks, carefully labelled, on a shelf somewhere.
>>
>> Now you are in a position to attempt data recovery on your copied
>> files. If you do something wrong, you can simply re-copy the image
>> files and try again. You still have absolutely no guarantees that
>> you'll get anything back - but at least you can be sure you are not
>> going to make anything worse.
>
> Follow David's advice.
>
> What was the filesystem on the array?
>
> Now, use testdisk, photorec, and foremost to seek through the raw images
> and extract files. The good news is, most video formats are detectable
> by these tools. All can be installed with your package manager.
>
> Regards,
> Tyler
>
> --
> "Offending fundamentalists isn't my goal - but if it is an inevitable
> side-effect of defending human rights, so be it."
> -- Johann Hari
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>


--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: how to recover filesystem after clobbering array?

On 2011-07-19 12:01, Vasco N=E9voa wrote:
> So I firmly believe the data is all there, on both disks.
> I just need to rebuild the metadata to point to the data again someho=
w.
> Right?...

Yes. Or no. Do you want to risk losing your data? Clone one of the disk=
s
now, then experiment with trying to re-assemble the array.

Regards,
Tyler

--=20
"It is an interesting and demonstrable fact, that all children are athe=
ists
and were religion not inculcated into their minds, they would remain so=
"
-- Ernestine Rose

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: how to recover filesystem after clobbering array?

I've successfully recovered my array data. :)
All it took was, as I expected, to rebuild the right pointers. The data=
=20
was always untouched.
Here it goes, for the sake of completeness.

I initially used "mdadm --create --assume-clean ..." on a level1 array=20
of 2 disks that came from another machine. I didn't know any other way=20
of starting an array in this situation. While this is an acceptable=20
practice in some cases, it is better to use "--assemble" and pass the=20
necessary info (like uuid).

Unfortunately I apparently lost all the data because the newly created=20
metadata superblock was the default 0.9 version, and the original array=
=20
metadata version was 1.2, and this resulted in a missing partition tabl=
e=20
once the array was run.

Then I retrieved the mdadm.conf file from the original machine, and=20
there I found the correct metadata version, uuid, array name, array=20
device name. I also double-checked the volume partition and filesystem=20
type from that machine's /etc/fstab.

So, I zeroed-out the metadata superblocks with "mdadm --zero-superblock=
=20
..", and then proceeded to restore the original metadata superblock=20
with "mdadm --create --assume-clean --metadata=3D1.2 --uuid=3D...=20
--name=3D... --level=3D1 --raid-devices=3D2 /dev/sde1 missing", and it=
worked=20
just fine. The array was started and I could see the original data=20
partition with fdisk and actually mount it. After this successful test,=
=20
I just added the other disk to the array.

The one information I expected from this list was: "No problem, your=20
data is still there. If you can recreate the superblock with the same=20
metadata version as it used to be, everything reverts to normal."=20
Unfortunately I had no such support.

Cheers,
Vasco.

On 19-07-2011 12:01, Vasco N=E9voa wrote:
> Thank you very much, Tyler and David.
> Good advice.
>
> The array is level 1, built upon primary partitions of the devices.
> The file system is EXT4.
> The mdadm command that I stupidly used to create instead of starting=20
> the array included "--assume-clean" but no "--build".
> I checked via /proc/mdadm that the array came up without syncing=20
> anything (or at least it was ultra-fast, less than 5 seconds).
> So I firmly believe the data is all there, on both disks.
> I just need to rebuild the metadata to point to the data again someho=
w.
> Right?...
>
> Citando "Tyler J. Wagner" :
>
>> On 2011-07-19 11:17, David Brown wrote:
>>> Once you have got image files for each of your disks, make copies o=
f
>>> these image files to another spare disk. Keep careful notes of exac=
tly
>>> what you have done here, and which files are which. And put your
>>> original disks, carefully labelled, on a shelf somewhere.
>>>
>>> Now you are in a position to attempt data recovery on your copied
>>> files. If you do something wrong, you can simply re-copy the image
>>> files and try again. You still have absolutely no guarantees that
>>> you'll get anything back - but at least you can be sure you are not
>>> going to make anything worse.
>>
>> Follow David's advice.
>>
>> What was the filesystem on the array?
>>
>> Now, use testdisk, photorec, and foremost to seek through the raw im=
ages
>> and extract files. The good news is, most video formats are detectab=
le
>> by these tools. All can be installed with your package manager.
>>
>> Regards,
>> Tyler
>>
>> --=20
>> "Offending fundamentalists isn't my goal - but if it is an inevitabl=
e
>> side-effect of defending human rights, so be it."
>> -- Johann Hari
>>
>> --=20
>> To unsubscribe from this list: send the line "unsubscribe linux-raid=
" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>
>

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: how to recover filesystem after clobbering array?

This message is in MIME format. The first part should be readable text,
while the remaining parts are likely unreadable without MIME-aware tools.

---137064504-1673026807-1311410552=:20159
Content-Type: TEXT/PLAIN; CHARSET=ISO-8859-15; FORMAT=flowed
Content-Transfer-Encoding: 8BIT

On Sat, 23 Jul 2011, Vasco Névoa wrote:

> I initially used "mdadm --create --assume-clean ..." on a level1 array
> of 2 disks that came from another machine. I didn't know any other way
> of starting an array in this situation. While this is an acceptable
> practice in some cases, it is better to use "--assemble" and pass the
> necessary info (like uuid).

In case someone finds this from the archives:

mdadm --create --assume-clean is a LAST RESORT. It's to be used when
everything else fails. I've seen so many start doing this way too early in
the process and who then subsequently lost their data because of
superblock version and mdadm changes over time (especially when they
forgot --assume-clean and got the layout wrong).

--
Mikael Abrahamsson email: swmike@swm.pp.se
---137064504-1673026807-1311410552=:20159--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: how to recover filesystem after clobbering array?

Just to make data restoring a bit safer - one can use device mappers's
snapshot feature as it can snapshot _any_ block device, you can create
snapshots of say /dev/sda1 & /dev/sdb1 , then do your raid experiments
on that snapshotted devices.

More info http://www.mjmwired.net/kernel/Documentation/device-mapper/s n=
apshot.txt
https://www.redhat.com/archives/dm-devel/2004-July/msg00071. html

2011/7/23 Mikael Abrahamsson :
> On Sat, 23 Jul 2011, Vasco N=E9voa wrote:
>
>> I initially used "mdadm --create --assume-clean ..." on a level1 arr=
ay of
>> 2 disks that came from another machine. I didn't know any other way =
of
>> starting an array in this situation. While this is an acceptable pra=
ctice in
>> some cases, it is better to use "--assemble" and pass the necessary =
info
>> (like uuid).
>
> In case someone finds this from the archives:
>
> mdadm --create --assume-clean is a LAST RESORT. It's to be used when
> everything else fails. I've seen so many start doing this way too ear=
ly in
> the process and who then subsequently lost their data because of supe=
rblock
> version and mdadm changes over time (especially when they forgot
> --assume-clean and got the layout wrong).
>
> --
> Mikael Abrahamsson =A0 =A0email: swmike@swm.pp.se



--=20
Best regards,
[COOLCOLD-RIPN]
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html