Mailbox size limits, gzip in procmail

I ran into some behaviour on my postfix+procmail setup which really
confused me. Postfix has a maximum mailbox size limit (50 M default) which
can be easily changed, but I thought that if using procmail to store
gzipped mail to a file then I wouldn't deal with the limit at all.

Using a rule like
:0:
| gzip >> archive.gz

It seems like archive.gz grows to 50 M and then stops. And it's corrupted,
gunzip fails. The reason I was trying to go through gzip is to get around
the mailbox maximum. And the limit kicking in and corrupting the gz file is
really bad :( Any tips?

Re: Mailbox size limits, gzip in procmail

Jem Berkes wrote:
> I ran into some behaviour on my postfix+procmail setup which really
> confused me. Postfix has a maximum mailbox size limit (50 M default) which
> can be easily changed, but I thought that if using procmail to store
> gzipped mail to a file then I wouldn't deal with the limit at all.
>
> Using a rule like
> :0:
> | gzip >> archive.gz
>
> It seems like archive.gz grows to 50 M and then stops. And it's corrupted,
> gunzip fails. The reason I was trying to go through gzip is to get around
> the mailbox maximum. And the limit kicking in and corrupting the gz file is
> really bad :( Any tips?

What I would suggest is to save your mail uncompressed, then compress
them later using either a cron job or logrotate functions to gzip or
bzip2 the files.

# Use date to create date stampped mailbox names.
# in the form archive-YYYY-MM

DATESTAMP=`date "+%Y-%m"`

# or archive-YYYY-MM-DD
# DATESTAMP=`date "+%Y-%m-%d"`

ARCHIVEFILENAME="archive-${DATESTAMP}"

:0:
${ARCHIVEFILENAME}


In a cronjob:
# crontab -e
# Once a day at 5:02 am
# search the archive directory for files starting with the name archive
# that are at least 35 days old. Compress any matches using gzip.

2 5 * * * find /path/to/archivedirectory -name 'archive*' -type f
-maxdepth 0 -mtime +35 -exec gzip {} \;

# You could do something similar to delete old archive files
# automatically that are at least 180 days old following a 6 month
# retention policy.

35 5 * * * find /path/to/archivedirectory -name 'archive*' -type f
-maxdepth 0 -mtime +180 -exec rm -f {} \;

Re: Mailbox size limits, gzip in procmail

Garen Erdoisa wrote in news:12or88ck7mrm812
@corp.supernews.com:

> What I would suggest is to save your mail uncompressed, then compress
> them later using either a cron job or logrotate functions to gzip or
> bzip2 the files.

I know what you're saying, but you would be shocked how much mail is coming
to this particular account. It needs to be compressed on the fly ;)

gzip works great on such a stream, bzip2 works very well for larger blocks
so it's not appropriate for the stream compression.

I figure another option is to launch a custom shell script that receives
the email and buffers it, then archives it.