parallel option in pg_restore

This is a multi-part message in MIME format.

------_=_NextPart_001_01CB120F.2DE4C404
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

I'm testing 8.4.4 (on Windows) before upgrading our app to this PG
version.
=20
When running pg_restore with "-j 2" parallel option, I'm getting the
following error:
=20
"pg_restore: [custom archiver] dumping a specific TOC data block out of
order is not supported without ID on this input stream (fseek required)"
=20
in the log file.
=20
Mind you, the backup (which I'm restoring here) was done in "custom"
mode ( -F c) using pg_dump version 8.2.5.
Is this error results from version differences between pg_dump and
pg_restore?
=20
The reason I'm using "old" backups (created with older pg_dump version)
is that I'm trying to save time during upgrade, and I have these big
backup files already created.
=20
TIA,
Igor Neyman


------_=_NextPart_001_01CB120F.2DE4C404
Content-Type: text/html;
charset="us-ascii"
Content-Transfer-Encoding: quoted-printable



charset=3Dus-ascii">

Re: parallel option in pg_restore

"Igor Neyman" wrote:

> I'm testing 8.4.4

> pg_restore with "-j 2" parallel option

> using pg_dump version 8.2.5.

> Is this error results from version differences between pg_dump and
> pg_restore?

Yeah, probably.

I suspect that you have the choice of dumping with the newer
pg_dump, or not using the new "-j 2" option on pg_restore.

-Kevin

--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

Re: parallel option in pg_restore

"Igor Neyman" writes:
> I'm testing 8.4.4 (on Windows) before upgrading our app to this PG
> version.
> When running pg_restore with "-j 2" parallel option, I'm getting the
> following error:
> "pg_restore: [custom archiver] dumping a specific TOC data block out of
> order is not supported without ID on this input stream (fseek required)"

We have gotten several reports of this, but none of the developers have
been able to reproduce it. Can you provide an exact test case?

regards, tom lane

--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

Re: parallel option in pg_restore

This is a multi-part message in MIME format.

------_=_NextPart_001_01CB121C.4A38CB2A
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

> -----Original Message-----
> From: Tom Lane [mailto:tgl@sss.pgh.pa.us]=20
> Sent: Tuesday, June 22, 2010 10:37 AM
> To: Igor Neyman
> Cc: pgsql-admin@postgresql.org
> Subject: Re: [ADMIN] parallel option in pg_restore=20
>=20
> "Igor Neyman" writes:
> > I'm testing 8.4.4 (on Windows) before upgrading our app to this PG=20
> > version.
> > When running pg_restore with "-j 2" parallel option, I'm=20
> getting the=20
> > following error:
> > "pg_restore: [custom archiver] dumping a specific TOC data=20
> block out=20
> > of order is not supported without ID on this input stream=20
> (fseek required)"
>=20
> We have gotten several reports of this, but none of the=20
> developers have been able to reproduce it. Can you provide=20
> an exact test case?
>=20
> regards, tom lane
>=20
>=20

Tom,

Backup files I'm trying to restore "in parallel" contain partitions of
several partitioned tables.
Tables partitioned "by month", each backup file contains 1 month worth
of data for all partitioned tables.

Before restoring backed up partitions, I'm restoring from another backup
file (not using "-j"), which contains "base" (empty) tables, from which
partitions inherited. And this restore runs fine.

Is that the information you asked for, or you want a sample of small
backup file attached?
I'm attaching pg_restore log file, if it's of any help.

Regards,
Igor Neyman

------_=_NextPart_001_01CB121C.4A38CB2A
Content-Type: application/octet-stream;
name="CM_200608_Restore.log"
Content-Transfer-Encoding: base64
Content-Description: CM_200608_Restore.log
Content-Disposition: attachment;
filename="CM_200608_Restore.log"

cGdfcmVzdG9yZTogY29ubmVjdGluZyB0byBkYXRhYmFzZSBmb3IgcmVzdG9y
ZQ0KcGdfcmVzdG9yZTogcHJvY2Vzc2luZyBpdGVtIDM2ODMgRU5DT0RJTkcg
RU5DT0RJTkcNCnBnX3Jlc3RvcmU6IHByb2Nlc3NpbmcgaXRlbSAzNjg0IFNU
RFNUUklOR1MgU1REU1RSSU5HUw0KcGdfcmVzdG9yZTogcHJvY2Vzc2luZyBp
dGVtIDI3MDEgVEFCTEUgZ3BfY3ljbGVfMjAwNjA4DQpwZ19yZXN0b3JlOiBj
cmVhdGluZyBUQUJMRSBncF9jeWNsZV8yMDA2MDgNCnBnX3Jlc3RvcmU6IHBy
b2Nlc3NpbmcgaXRlbSAyNzAyIFRBQkxFIGdwX2N5Y2xlX2FsYXJtXzIwMDYw
OA0KcGdfcmVzdG9yZTogY3JlYXRpbmcgVEFCTEUgZ3BfY3ljbGVfYWxhcm1f
MjAwNjA4DQpwZ19yZXN0b3JlOiBwcm9jZXNzaW5nIGl0ZW0gMjcwMyBUQUJM
RSBncF9jeWNsZV9wYXJ0X2lkXzIwMDYwOA0KcGdfcmVzdG9yZTogY3JlYXRp
bmcgVEFCTEUgZ3BfY3ljbGVfcGFydF9pZF8yMDA2MDgNCnBnX3Jlc3RvcmU6
IHByb2Nlc3NpbmcgaXRlbSAyNzA0IFRBQkxFIGdwX2N5Y2xlX3Byb2Nlc3Nf
aWRfMjAwNjA4DQpwZ19yZXN0b3JlOiBjcmVhdGluZyBUQUJMRSBncF9jeWNs
ZV9wcm9jZXNzX2lkXzIwMDYwOA0KcGdfcmVzdG9yZTogcHJvY2Vzc2luZyBp
dGVtIDI3MDUgVEFCTEUgZ3BfY3ljbGVfdmZfc2VjX3BvaW50c18yMDA2MDgN
CnBnX3Jlc3RvcmU6IGNyZWF0aW5nIFRBQkxFIGdwX2N5Y2xlX3ZmX3NlY19w
b2ludHNfMjAwNjA4DQpwZ19yZXN0b3JlOiBwcm9jZXNzaW5nIGl0ZW0gMjcw
NiBUQUJMRSBncF9jeWNsZV92aXN1YWxfZml4dHVyZV8yMDA2MDgNCnBnX3Jl
c3RvcmU6IGNyZWF0aW5nIFRBQkxFIGdwX2N5Y2xlX3Zpc3VhbF9maXh0dXJl
XzIwMDYwOA0KcGdfcmVzdG9yZTogcHJvY2Vzc2luZyBpdGVtIDI3MDcgVEFC
TEUgZ3BfbWVhc3VyZW1lbnRfMjAwNjA4DQpwZ19yZXN0b3JlOiBjcmVhdGlu
ZyBUQUJMRSBncF9tZWFzdXJlbWVudF8yMDA2MDgNCnBnX3Jlc3RvcmU6IGVu
dGVyaW5nIG1haW4gcGFyYWxsZWwgbG9vcA0KcGdfcmVzdG9yZTogbGF1bmNo
aW5nIGl0ZW0gMzY3NiBUQUJMRSBEQVRBIGdwX2N5Y2xlXzIwMDYwOA0KcGdf
cmVzdG9yZTogbGF1bmNoaW5nIGl0ZW0gMzY3NyBUQUJMRSBEQVRBIGdwX2N5
Y2xlX2FsYXJtXzIwMDYwOA0KcGdfcmVzdG9yZTogbGF1bmNoaW5nIGl0ZW0g
MzY3OCBUQUJMRSBEQVRBIGdwX2N5Y2xlX3BhcnRfaWRfMjAwNjA4DQpwZ19y
ZXN0b3JlOiBsYXVuY2hpbmcgaXRlbSAzNjc5IFRBQkxFIERBVEEgZ3BfY3lj
bGVfcHJvY2Vzc19pZF8yMDA2MDgNCnBnX3Jlc3RvcmU6IHJlc3RvcmluZyBk
YXRhIGZvciB0YWJsZSAiZ3BfY3ljbGVfYWxhcm1fMjAwNjA4Ig0KcGdfcmVz
dG9yZTogcmVzdG9yaW5nIGRhdGEgZm9yIHRhYmxlICJncF9jeWNsZV9wYXJ0
X2lkXzIwMDYwOCINCnBnX3Jlc3RvcmU6IFtjdXN0b20gYXJjaGl2ZXJdIGR1
bXBpbmcgYSBzcGVjaWZpYyBUT0MgZGF0YSBibG9jayBvdXQgb2Ygb3JkZXIg
aXMgbm90IHN1cHBvcnRlZCB3aXRob3V0IElEIG9uIHRoaXMgaW5wdXQgc3Ry
ZWFtIChmc2VlayByZXF1aXJlZCkNCnBnX3Jlc3RvcmU6ICoqKiBhYm9ydGVk
IGJlY2F1c2Ugb2YgZXJyb3INCg==

------_=_NextPart_001_01CB121C.4A38CB2A
Content-Type: text/plain
Content-Disposition: inline
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable


--=20
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

------_=_NextPart_001_01CB121C.4A38CB2A--

Re: parallel option in pg_restore

> From: Igor Neyman
> Subject: Re: [ADMIN] parallel option in pg_restore
> To: "Tom Lane"
> Cc: pgsql-admin@postgresql.org
> Date: Tuesday, 22 June, 2010, 16:05
> > -----Original Message-----
> > From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
>=20
> > Sent: Tuesday, June 22, 2010 10:37 AM
> > To: Igor Neyman
> > Cc: pgsql-admin@postgresql.org
> > Subject: Re: [ADMIN] parallel option in pg_restore=20
> >=20
> > "Igor Neyman"
> writes:
> > > I'm testing 8.4.4 (on Windows) before upgrading
> our app to this PG=20
> > > version.
> > > When running pg_restore with "-j 2" parallel
> option, I'm=20
> > getting the=20
> > > following error:
> > > "pg_restore: [custom archiver] dumping a specific
> TOC data=20
> > block out=20
> > > of order is not supported without ID on this
> input stream=20
> > (fseek required)"
> >=20
> > We have gotten several reports of this, but none of
> the=20
> > developers have been able to reproduce it.=A0 Can
> you provide=20
> > an exact test case?
> >=20
> >   =A0   =A0
>   =A0 regards, tom lane
> >=20
> >=20
>=20
> Tom,
>=20
> Backup files I'm trying to restore "in parallel" contain
> partitions of
> several partitioned tables.
> Tables partitioned "by month", each backup file contains 1
> month worth
> of data for all partitioned tables.
>=20
> Before restoring backed up partitions, I'm restoring from
> another backup
> file (not using "-j"), which contains "base" (empty)
> tables, from which
> partitions inherited. And this restore runs fine.
>=20
> Is that the information you asked for, or you want a sample
> of small
> backup file attached?
> I'm attaching pg_restore log file, if it's of any help.
>=20

In my experiments the error went away when I reduced the amount of data in =
the tables being restored/size of the dump.

This is as far as I got, but I let it rest for a while due to lack of respo=
nse on the list.

http://archives.postgresql.org/pgsql-general/2010-05/msg0077 8.php






--=20
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

Re: parallel option in pg_restore

"Igor Neyman" writes:
> Is that the information you asked for, or you want a sample of small
> backup file attached?
> I'm attaching pg_restore log file, if it's of any help.

If you can make a small archive file that provokes the problem, yes
please send it. Also, please show the exact pg_restore command line
you're using.

regards, tom lane

--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

Re: parallel option in pg_restore

On Tue, Jun 22, 2010 at 11:05:02AM -0400, Igor Neyman wrote:
> > "Igor Neyman" writes:
> > > I'm testing 8.4.4 (on Windows) before upgrading our app to this PG
> > > version.
> > > When running pg_restore with "-j 2" parallel option, I'm getting the
> > > following error:
> > > "pg_restore: [custom archiver] dumping a specific TOC data block out
> > > of order is not supported without ID on this input stream
> > > (fseek required)"
> >
> > We have gotten several reports of this, but none of the
> > developers have been able to reproduce it. Can you provide
> > an exact test case?
> > regards, tom lane

> Backup files I'm trying to restore "in parallel" contain partitions of
> several partitioned tables.
> Tables partitioned "by month", each backup file contains 1 month worth
> of data for all partitioned tables.
>
> Before restoring backed up partitions, I'm restoring from another backup
> file (not using "-j"), which contains "base" (empty) tables, from which
> partitions inherited. And this restore runs fine.

I realise this may be a silly question (especially for windows), but
the fseek complaint has me wondering.

Are you running a pipleine reatore? E.G:

type dumpfile | pg_restore -j 2

or are you running:

pg_restore -j 2 dumpfile

in the latter case it should be fseekable, but in the former case I
don't think you can fseek stdin on either windows or *nix..

--
-- rouilj

John Rouillard System Administrator
Renesys Corporation 603-244-9084 (cell) 603-643-9300 x 111

--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

Re: parallel option in pg_restore

=20

> -----Original Message-----
> From: John Rouillard [mailto:rouilj@renesys.com]=20
> Sent: Tuesday, June 22, 2010 11:52 AM
> To: Igor Neyman
> Cc: pgsql-admin@postgresql.org
> Subject: Re: [ADMIN] parallel option in pg_restore
>=20
> On Tue, Jun 22, 2010 at 11:05:02AM -0400, Igor Neyman wrote:
> > > "Igor Neyman" writes:
> > > > I'm testing 8.4.4 (on Windows) before upgrading our app=20
> to this PG=20
> > > > version.
> > > > When running pg_restore with "-j 2" parallel option,=20
> I'm getting=20
> > > > the following error:
> > > > "pg_restore: [custom archiver] dumping a specific TOC=20
> data block=20
> > > > out of order is not supported without ID on this input stream=20
> > > > (fseek required)"
> > >=20
> > > We have gotten several reports of this, but none of the=20
> developers=20
> > > have been able to reproduce it. Can you provide an exact=20
> test case?
> > > regards, tom lane
>=20
> > Backup files I'm trying to restore "in parallel" contain=20
> partitions of=20
> > several partitioned tables.
> > Tables partitioned "by month", each backup file contains 1=20
> month worth=20
> > of data for all partitioned tables.
> >=20
> > Before restoring backed up partitions, I'm restoring from another=20
> > backup file (not using "-j"), which contains "base" (empty) tables,=20
> > from which partitions inherited. And this restore runs fine.
>=20
> I realise this may be a silly question (especially for=20
> windows), but the fseek complaint has me wondering.
>=20
> Are you running a pipleine reatore? E.G:
>=20
> type dumpfile | pg_restore -j 2
>=20
> or are you running:
>=20
> pg_restore -j 2 dumpfile
>=20
> in the latter case it should be fseekable, but in the former=20
> case I don't think you can fseek stdin on either windows or *nix..
>=20
> --=20
> -- rouilj
>=20
> John Rouillard System Administrator
> Renesys Corporation 603-244-9084 (cell) 603-643-9300 x 111
>=20
>=20

No piping, just regular restore from the backup file.

Regards,
Igor Neyman

--=20
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

Re: parallel option in pg_restore

--- On Tue, 22/6/10, Igor Neyman wrote:

> From: Igor Neyman
> Subject: Re: [ADMIN] parallel option in pg_restore
> To: "John Rouillard"
> Cc: pgsql-admin@postgresql.org
> Date: Tuesday, 22 June, 2010, 17:34
>
> No piping, just regular restore from the backup file.
>=20

Same here. If only I could get a small sample which exhibited the issues -=
so far I can only get the same error with large dump files.





--=20
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

Re: parallel option in pg_restore

> -----Original Message-----
> From: Glyn Astill [mailto:glynastill@yahoo.co.uk]=20
> Sent: Tuesday, June 22, 2010 12:36 PM
> To: John Rouillard; Igor Neyman
> Cc: pgsql-admin@postgresql.org
> Subject: Re: [ADMIN] parallel option in pg_restore
>=20
> --- On Tue, 22/6/10, Igor Neyman wrote:
>=20
> > From: Igor Neyman
> > Subject: Re: [ADMIN] parallel option in pg_restore
> > To: "John Rouillard"
> > Cc: pgsql-admin@postgresql.org
> > Date: Tuesday, 22 June, 2010, 17:34
> >
> > No piping, just regular restore from the backup file.
> >=20
>=20
> Same here. If only I could get a small sample which=20
> exhibited the issues - so far I can only get the same error=20
> with large dump files.
>=20

I just sent some samples in reply to Tom's request.

Regards,
Igor Neyman

--=20
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

Re: parallel option in pg_restore

Glyn Astill wrote:

> so far I can only get the same error with large dump files.

"Large" being a relative term --
ever see it on a file smaller than 2GB?

-Kevin

--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

Re: parallel option in pg_restore

=20

> -----Original Message-----
> From: Kevin Grittner [mailto:Kevin.Grittner@wicourts.gov]=20
> Sent: Tuesday, June 22, 2010 12:40 PM
> To: Igor Neyman; John Rouillard; Glyn Astill
> Cc: pgsql-admin@postgresql.org
> Subject: Re: [ADMIN] parallel option in pg_restore
>=20
> Glyn Astill wrote:
>
> > so far I can only get the same error with large dump files.
>
> "Large" being a relative term --
> ever see it on a file smaller than 2GB?
>
> -Kevin
>=20
>=20

Yes, just sent couple to the list.

Igor

--=20
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

Re: parallel option in pg_restore

"Igor Neyman" writes:
> Attached are couple smallish files (I suspect, CM_200909.bac might have
> just empty tables, no data - but it still produces an errror).

Hmm. I get

pg_restore: [archiver (db)] Error while PROCESSING TOC:
pg_restore: [archiver (db)] Error from TOC entry 2741; 1259 30866 TABLE gp_cycle_200907 vec_dba
pg_restore: [archiver (db)] could not execute query: ERROR: relation "gp_cycle" does not exist
Command was:
CREATE TABLE gp_cycle_200907 (CONSTRAINT gp_cycle_200907_cycle_date_time_check CHECK (((cycle_date_time >= '2009-07-01 00:0...

The tables all seem to inherit from tables you omitted from the dump,
so of course it's not restorable for anyone else.

Now I do see

pg_restore: [custom archiver] dumping a specific TOC data block out of order is not supported without ID on this input stream (fseek required)

after that, but I'm wondering if this is just a problem in error
recovery rather than the bug we thought we were looking for.

regards, tom lane

--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

Re: parallel option in pg_restore

This is a multi-part message in MIME format.

------_=_NextPart_001_01CB1231.22DBC874
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: quoted-printable


> -----Original Message-----
> From: Tom Lane [mailto:tgl@sss.pgh.pa.us]=20
> Sent: Tuesday, June 22, 2010 1:10 PM
> To: Igor Neyman
> Cc: pgsql-admin@postgresql.org
> Subject: Re: [ADMIN] parallel option in pg_restore=20
>=20
> "Igor Neyman" writes:
> > Attached are couple smallish files (I suspect, CM_200909.bac might=20
> > have just empty tables, no data - but it still produces an errror).
>=20
> Hmm. I get
>=20
> pg_restore: [archiver (db)] Error while PROCESSING TOC:
> pg_restore: [archiver (db)] Error from TOC entry 2741; 1259=20
> 30866 TABLE gp_cycle_200907 vec_dba
> pg_restore: [archiver (db)] could not execute query: ERROR:
> relation "gp_cycle" does not exist
> Command was:=20
> CREATE TABLE gp_cycle_200907 (CONSTRAINT=20
> gp_cycle_200907_cycle_date_time_check CHECK=20
> (((cycle_date_time >=3D '2009-07-01 00:0...
>=20
> The tables all seem to inherit from tables you omitted from=20
> the dump, so of course it's not restorable for anyone else.
>=20
> Now I do see
>=20
> pg_restore: [custom archiver] dumping a specific TOC data=20
> block out of order is not supported without ID on this input=20
> stream (fseek required)
>=20
> after that, but I'm wondering if this is just a problem in=20
> error recovery rather than the bug we thought we were looking for.
>=20
> regards, tom lane
>=20
>=20

Right, like I mentioned, these are partitioned tables.

Attached is script that could be used to pre-create "parent" tables
(from which partitions were inherited).
You run it before restoring backed up partition.

Thank you for taking time to look into this issue.
Regards,
Igor Neyman

------_=_NextPart_001_01CB1231.22DBC874
Content-Type: application/octet-stream;
name="parent_tables.sql"
Content-Transfer-Encoding: base64
Content-Description: parent_tables.sql
Content-Disposition: attachment;
filename="parent_tables.sql"

Lyo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PSovDQovKiBUYWJsZTogR1BfQ1lDTEUgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgKi8N
Ci8qPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT0qLw0KY3JlYXRlIHRhYmxlIEdQX0NZQ0xF
ICgNCkNZQ0xFX0RBVEVfVElNRSAgICAgIFRJTUVTVEFNUCAgICAgICAgICAg
IG5vdCBudWxsLA0KQ0VMTF9JRCAgICAgICAgICAgICAgSU5UNCAgICAgICAg
ICAgICAgICAgbm90IG51bGwsDQpQQVJUX1RZUEVfSUQgICAgICAgICBJTlQ0
ICAgICAgICAgICAgICAgICBub3QgbnVsbCwNCkNZQ0xFX01PREVfQ09ERV9J
RCAgIElOVDQgICAgICAgICAgICAgICAgIG5vdCBudWxsLA0KRVJST1JfQ09V
TlQgICAgICAgICAgSU5UNCAgICAgICAgICAgICAgICAgbnVsbCwNClFVQUxJ
VFlfQ09ERV9JRCAgICAgIElOVDQgICAgICAgICAgICAgICAgIG51bGwsDQpB
TEFSTV9DT1VOVCAgICAgICAgICBJTlQ0ICAgICAgICAgICAgICAgICBudWxs
LA0KQ1lDTEVfRUxBUFNFRF9USU1FICAgRkxPQVQ4ICAgICAgICAgICAgICAg
bnVsbCwNCkdPT0RfQ09VTlQgICAgICAgICAgIElOVDQgICAgICAgICAgICAg
ICAgIG51bGwsDQpUUkVORElOR19DT1VOVCAgICAgICBJTlQ0ICAgICAgICAg
ICAgICAgICBudWxsLA0KT1VUX09GX1NQRUNfQ09VTlQgICAgSU5UNCAgICAg
ICAgICAgICAgICAgbnVsbCwNClJFSkVDVF9DT1VOVCAgICAgICAgIElOVDQg
ICAgICAgICAgICAgICAgIG51bGwsDQpTREFfUkVTVUxUX0NPREVfSUQgICBJ
TlQ0ICAgICAgICAgICAgICAgICBudWxsLA0KQ1JJVF9FUlJPUl9DT1VOVCAg
ICAgSU5UNCAgICAgICAgICAgICAgICAgbnVsbCwNCkNSSVRfQUxBUk1fQ09V
TlQgICAgIElOVDQgICAgICAgICAgICAgICAgIG51bGwsDQpDUklUX0dPT0Rf
Q09VTlQgICAgICBJTlQ0ICAgICAgICAgICAgICAgICBudWxsLA0KQ1JJVF9U
UkVORElOR19DT1VOVCAgSU5UNCAgICAgICAgICAgICAgICAgbnVsbCwNCkNS
SVRfT1VUX09GX1NQRUNfQ09VTlQgSU5UNCAgICAgICAgICAgICAgICAgbnVs
bCwNCkNSSVRfUkVKRUNUX0NPVU5UICAgIElOVDQgICAgICAgICAgICAgICAg
IG51bGwsDQpDUklUX1FVQUxJVFlfQ09ERV9JRCBJTlQ0ICAgICAgICAgICAg
ICAgICBudWxsLA0KY29uc3RyYWludCBQS19HUF9DWUNMRSBwcmltYXJ5IGtl
eSAoQ0VMTF9JRCwgUEFSVF9UWVBFX0lELCBDWUNMRV9EQVRFX1RJTUUpDQop
Ow0KDQovKj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09Ki8NCi8qIFRhYmxlOiBHUF9DWUNM
RV9BTEFSTSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAqLw0KLyo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PSovDQpjcmVhdGUgdGFibGUgR1Bf
Q1lDTEVfQUxBUk0gKA0KQ0VMTF9JRCAgICAgICAgICAgICAgSU5UNCAgICAg
ICAgICAgICAgICAgbm90IG51bGwsDQpQQVJUX1RZUEVfSUQgICAgICAgICBJ
TlQ0ICAgICAgICAgICAgICAgICBub3QgbnVsbCwNCkNZQ0xFX0RBVEVfVElN
RSAgICAgIFRJTUVTVEFNUCAgICAgICAgICAgIG5vdCBudWxsLA0KQUxBUk1f
SUQgICAgICAgICAgICAgSU5UNCAgICAgICAgICAgICAgICAgbm90IG51bGws
DQpDSF9JRCAgICAgICAgICAgICAgICBJTlQ0ICAgICAgICAgICAgICAgICBu
b3QgbnVsbCwNCkxJTUlUX1RZUEVfQ09ERV9JRCAgIElOVDQgICAgICAgICAg
ICAgICAgIG5vdCBudWxsLA0KY29uc3RyYWludCBQS19HUF9DWUNMRV9BTEFS
TSBwcmltYXJ5IGtleSAoQ0VMTF9JRCwgUEFSVF9UWVBFX0lELCBDWUNMRV9E
QVRFX1RJTUUsIEFMQVJNX0lELCBDSF9JRCwgTElNSVRfVFlQRV9DT0RFX0lE
KQ0KKTsNCg0KLyo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PSovDQovKiBUYWJsZTogR1Bf
Q1lDTEVfUEFSVF9JRCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgKi8NCi8qPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0qLw0KY3JlYXRlIHRhYmxl
IEdQX0NZQ0xFX1BBUlRfSUQgKA0KQ1lDTEVfREFURV9USU1FICAgICAgVElN
RVNUQU1QICAgICAgICAgICAgbm90IG51bGwsDQpDRUxMX0lEICAgICAgICAg
ICAgICBJTlQ0ICAgICAgICAgICAgICAgICBub3QgbnVsbCwNClBBUlRfVFlQ
RV9JRCAgICAgICAgIElOVDQgICAgICAgICAgICAgICAgIG5vdCBudWxsLA0K
UEFSVF9JRF9OQU1FICAgICAgICAgVkFSQ0hBUigxMDApICAgICAgICAgbm90
IG51bGwsDQpQQVJUX0lEX1ZBTFVFICAgICAgICBWQVJDSEFSKDI1NSkgICAg
ICAgICBudWxsLA0KY29uc3RyYWludCBQS19HUF9DWUNMRV9QQVJUX0lEIHBy
aW1hcnkga2V5IChDRUxMX0lELCBQQVJUX1RZUEVfSUQsIENZQ0xFX0RBVEVf
VElNRSwgUEFSVF9JRF9OQU1FKQ0KKTsNCg0KLyo9PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PSovDQovKiBUYWJsZTogR1BfQ1lDTEVfUFJPQ0VTU19JRCAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgKi8NCi8qPT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PT0qLw0KY3JlYXRlIHRhYmxlIEdQX0NZQ0xFX1BST0NFU1NfSUQgKA0KQ1lD
TEVfREFURV9USU1FICAgICAgVElNRVNUQU1QICAgICAgICAgICAgbm90IG51
bGwsDQpDRUxMX0lEICAgICAgICAgICAgICBJTlQ0ICAgICAgICAgICAgICAg
ICBub3QgbnVsbCwNClBBUlRfVFlQRV9JRCAgICAgICAgIElOVDQgICAgICAg
ICAgICAgICAgIG5vdCBudWxsLA0KUFJPQ0VTU19JRF9OQU1FICAgICAgVkFS
Q0hBUigxMDApICAgICAgICAgbm90IG51bGwsDQpQUk9DRVNTX0lEX1ZBTFVF
ICAgICBWQVJDSEFSKDI1NSkgICAgICAgICBub3QgbnVsbCwNCmNvbnN0cmFp
bnQgUEtfR1BfQ1lDTEVfUFJPQ0VTU19JRCBwcmltYXJ5IGtleSAoQ0VMTF9J
RCwgUEFSVF9UWVBFX0lELCBDWUNMRV9EQVRFX1RJTUUsIFBST0NFU1NfSURf
TkFNRSkNCik7DQoNCi8qPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0qLw0KLyogVGFibGU6
IEdQX0NZQ0xFX1ZGX1NFQ19QT0lOVFMgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICovDQovKj09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ki8NCmNyZWF0ZSB0
YWJsZSBHUF9DWUNMRV9WRl9TRUNfUE9JTlRTICgNCkNZQ0xFX0RBVEVfVElN
RSAgICAgIFRJTUVTVEFNUCAgICAgICAgICAgIG5vdCBudWxsLA0KQ0VMTF9J
RCAgICAgICAgICAgICAgSU5UNCAgICAgICAgICAgICAgICAgbm90IG51bGws
DQpQQVJUX1RZUEVfSUQgICAgICAgICBJTlQ0ICAgICAgICAgICAgICAgICBu
b3QgbnVsbCwNClZJU1VBTF9GSVhUVVJFX0lEICAgIElOVDQgICAgICAgICAg
ICAgICAgIG5vdCBudWxsLA0KSU5TUF9QT0lOVF9JRCAgICAgICAgSU5UNCAg
ICAgICAgICAgICAgICAgbm90IG51bGwsDQpjb25zdHJhaW50IFBLX0dQX0NZ
Q0xFX1ZGX1NFQ19QT0lOVFMgcHJpbWFyeSBrZXkgKENFTExfSUQsIFBBUlRf
VFlQRV9JRCwgQ1lDTEVfREFURV9USU1FLCBWSVNVQUxfRklYVFVSRV9JRCwg
SU5TUF9QT0lOVF9JRCkNCik7DQoNCi8qPT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0qLw0K
LyogVGFibGU6IEdQX0NZQ0xFX1ZJU1VBTF9GSVhUVVJFICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICovDQovKj09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ki8N
CmNyZWF0ZSB0YWJsZSBHUF9DWUNMRV9WSVNVQUxfRklYVFVSRSAoDQpDWUNM
RV9EQVRFX1RJTUUgICAgICBUSU1FU1RBTVAgICAgICAgICAgICBub3QgbnVs
bCwNCkNFTExfSUQgICAgICAgICAgICAgIElOVDQgICAgICAgICAgICAgICAg
IG5vdCBudWxsLA0KUEFSVF9UWVBFX0lEICAgICAgICAgSU5UNCAgICAgICAg
ICAgICAgICAgbm90IG51bGwsDQpWSVNVQUxfRklYVFVSRV9JRCAgICBJTlQ0
ICAgICAgICAgICAgICAgICBub3QgbnVsbCwNClJPQk9UX0lEICAgICAgICAg
ICAgIElOVDQgICAgICAgICAgICAgICAgIG5vdCBudWxsLA0KVFJBTlNGT1JN
X1RZUEUgICAgICAgSU5UNCAgICAgICAgICAgICAgICAgbm90IG51bGwsDQpW
SVNVQUxfRklYVFVSRV9UUkFOU0ZPUk0gQllURUEgICAgICAgICAgICAgICAg
bnVsbCwNCkJPVU5EQVJZX0JPWF9UUkFOU0ZPUk0gQllURUEgICAgICAgICAg
ICAgICAgbnVsbCwNClBPU0lUSU9OX0lEICAgICAgICAgIElOVDQgICAgICAg
ICAgICAgICAgIG51bGwsDQpCQUNLVVBfU0NIRU1FICAgICAgICBJTlQ0ICAg
ICAgICAgICAgICAgICBudWxsLA0KVkZfUVVBTElUWV9DT0RFX0lEICAgSU5U
NCAgICAgICAgICAgICAgICAgbm90IG51bGwsDQpTRUNPTkRBUllfUE9JTlRT
X1VTRUQgU01BTExJTlQgICAgICAgICAgICAgbm90IG51bGwsDQpDWUNMRV9N
T0RFX0NPREVfSUQgICBJTlQ0ICAgICAgICAgICAgICAgICBub3QgbnVsbCwN
CmNvbnN0cmFpbnQgUEtfR1BfQ1lDTEVfVklTVUFMX0ZJWFRVUkUgcHJpbWFy
eSBrZXkgKENFTExfSUQsIFBBUlRfVFlQRV9JRCwgQ1lDTEVfREFURV9USU1F
LCBWSVNVQUxfRklYVFVSRV9JRCwgUk9CT1RfSUQsIFRSQU5TRk9STV9UWVBF
KQ0KKTsNCg0KLyo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PSovDQovKiBUYWJsZTogR1Bf
TUVBU1VSRU1FTlQgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgKi8NCi8qPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0qLw0KY3JlYXRlIHRhYmxl
IEdQX01FQVNVUkVNRU5UICgNCkNZQ0xFX0RBVEVfVElNRSAgICAgIFRJTUVT
VEFNUCAgICAgICAgICAgIG5vdCBudWxsLA0KQ0hfSUQgICAgICAgICAgICAg
ICAgSU5UNCAgICAgICAgICAgICAgICAgbm90IG51bGwsDQpDRUxMX0lEICAg
ICAgICAgICAgICBJTlQ0ICAgICAgICAgICAgICAgICBub3QgbnVsbCwNCk1F
QVNVUkVNRU5UICAgICAgICAgIEZMT0FUOCAgICAgICAgICAgICAgIG51bGws
DQpRVUFMSVRZX0NPREVfSUQgICAgICBJTlQ0ICAgICAgICAgICAgICAgICBu
dWxsLA0KRVJST1JfQ09ERV9JRCAgICAgICAgSU5UNCAgICAgICAgICAgICAg
ICAgbnVsbCwNCkxJTUlUX0FMQVJNX01BU0sgICAgIElOVDQgICAgICAgICAg
ICAgICAgIG51bGwsDQpWSVNVQUxfRklYVFVSRV9JRCAgICBJTlQ0ICAgICAg
ICAgICAgICAgICBub3QgbnVsbCwNCmNvbnN0cmFpbnQgUEtfR1BfTUVBU1VS
RU1FTlQgcHJpbWFyeSBrZXkgKENZQ0xFX0RBVEVfVElNRSwgQ0hfSUQsIFZJ
U1VBTF9GSVhUVVJFX0lEKQ0KKTsNCg0KDQoNCg0K

------_=_NextPart_001_01CB1231.22DBC874
Content-Type: text/plain
Content-Disposition: inline
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable


--=20
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

------_=_NextPart_001_01CB1231.22DBC874--

Re: parallel option in pg_restore

"Igor Neyman" writes:
> Attached is script that could be used to pre-create "parent" tables
> (from which partitions were inherited).

Thanks. Now that I dig into it, it looks like the actual trigger for
the problem is that pg_dump, not pg_restore, couldn't seek while it
was creating the dump file --- so it didn't seek back and update the
file's table-of-contents with exact dump offsets. What command did
you use to create the dump file, exactly?

regards, tom lane

--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

Re: parallel option in pg_restore

> -----Original Message-----
> From: Tom Lane [mailto:tgl@sss.pgh.pa.us]=20
> Sent: Tuesday, June 22, 2010 2:41 PM
> To: Igor Neyman
> Cc: pgsql-admin@postgresql.org
> Subject: Re: [ADMIN] parallel option in pg_restore=20
>=20
> "Igor Neyman" writes:
> > Attached is script that could be used to pre-create "parent" tables=20
> > (from which partitions were inherited).
>=20
> Thanks. Now that I dig into it, it looks like the actual=20
> trigger for the problem is that pg_dump, not pg_restore,=20
> couldn't seek while it was creating the dump file --- so it=20
> didn't seek back and update the file's table-of-contents with=20
> exact dump offsets. What command did you use to create the=20
> dump file, exactly?
>=20
> regards, tom lane
>=20
>=20

Here is the backup script to backup all partitions for specific month
(200907) in one backup file:

SETLOCAL
set PGPASSFILE=3D%PGINSTALL%\DB_scripts\postgres.pgpass
SET PGBACKUPDRIVE=3D%PGBACKUP%

pg_dump -U vec_dba -F c -f
%PGBACKUPDRIVE%\PartitionedBackup\CM_200907.bac -v -Z 9 -t *200907
vector 2>> %PGBACKUPDRIVE%\Backup\Log\DB_Backup.log

ENDLOCAL


This script is a part of "bigger" backup, which backs up other
non-partitioned tables as well.


Regards,
Igor Neyman

--=20
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

Re: parallel option in pg_restore

--- On Tue, 22/6/10, Kevin Grittner wrote:
> Glyn Astill
> wrote:
>
> > so far I can only get the same error with large dump
> files.
>
> "Large" being a relative term --
> ever see it on a file smaller than 2GB?
>

Good point. No I've not seen it on a file smaller than 2GB, but the test I=
did was pretty basic - I just trimmed down the size of all of my tables to=
create a dump that was only 50Mb or so. It looks like Igor has a reproduc=
eable case now though, so hopefully Tom can figure out what's going off.




--=20
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

Re: parallel option in pg_restore

Glyn Astill writes:
> Good point. No I've not seen it on a file smaller than 2GB, but the test I did was pretty basic - I just trimmed down the size of all of my tables to create a dump that was only 50Mb or so. It looks like Igor has a reproduceable case now though, so hopefully Tom can figure out what's going off.

I neglected to follow up to this -admin thread, but see
http://archives.postgresql.org/pgsql-hackers/2010-06/msg0122 7.php

regards, tom lane

--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

Re: parallel option in pg_restore

--- On Wed, 23/6/10, Tom Lane wrote:
> Glyn Astill
> writes:
> > Good point.=A0 No I've not seen it on a file
> smaller than 2GB, but the test I did was pretty basic - I
> just trimmed down the size of all of my tables to create a
> dump that was only 50Mb or so.=A0 It looks like Igor has
> a reproduceable case now though, so hopefully Tom can figure
> out what's going off.
>=20
> I neglected to follow up to this -admin thread, but see
> http://archives.postgresql.org/pgsql-hackers/2010-06/msg0122 7.php

Thanks Tom, that's sufficient information to solve our problems here.





--=20
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin