[seqfan] Re: b-files

[Antti Karttunen]

On Mon, Nov 19, 2012 at 2:58 PM,  <seqfan-request at list.seqfan.eu> wrote:
----------------------------------------------------------------------
>
> Message: 1
> Date: Sun, 18 Nov 2012 08:03:31 +0200
> From: Georgi Guninski <guninski at guninski.com>
> To: Sequence Fanatics Discussion list <seqfan at list.seqfan.eu>
> Subject: [seqfan] Re: b-files, reply to Antti
> Message-ID: <20121118060331.GA2531 at sivokote.iziade.m$>
> Content-Type: text/plain; charset=us-ascii
>
> What about periodically making a single compressed archive of the
> b-files?
>
> This will save you both bandwidth and web server load.
>
> cron(8) can do it easily, though something like git/svn/hq would be
> better for me.
>
> Since you can't enforce not downloading via different IPs, there
> is no point to make it hard IMHO.
>
> On Sat, Nov 17, 2012 at 09:07:09PM -0500, Neil Sloane wrote:
>> There are about 46,000 b-files now. It is OK, I think, to download
>> a bunch of them (subject to the license agreements, of course) as long
>> as you don't clog the server.
>> Neil
>>
>> --
>> Dear Friends, I have now retired from AT&T. New coordinates:
>>
>> Neil J. A. Sloane, President, OEIS Foundation
>> 11 South Adelaide Avenue, Highland Park, NJ 08904, USA
>> Phone: 732 828 6098; home page: http://NeilSloane.com
>> Email: njasloane at gmail.com
>>
>> _______________________________________________
>>
>> Seqfan Mailing list - http://list.seqfan.eu/
>
>
> ------------------------------
>
> Message: 2
> Date: Sun, 18 Nov 2012 02:16:05 -0500
> From: Neil Sloane <njasloane at gmail.com>
> To: Sequence Fanatics Discussion list <seqfan at list.seqfan.eu>
> Subject: [seqfan] Re: b-files, reply to Antti
> Message-ID:
>         <CAAOnSgR4rpA=3kKAG6WZRm+C2jHq9_hDLkfdTWnkCHe2CoAwzg at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> If someone tried to download the b-file archive, wouldn't that
> really put the OEIS into a coma? And do more damage than someone
> downloading a few b-files at a time?

If you tell us what is the usual load on the OEIS server, e.g. average
number of b-files or "b-file-kilobytes"
downloaded per minute, then it is easy to set the sleeping-time
between wget-requests so long that
it would increase the total load just say by two percents. Then even a
few people bulk-downloading
at the same time wouldn´t increase the load excessively.

And we could adopt a gentleman´s agreement that we give an option like
--user-agent "Antti´s bulk-download of b-files, sleep/pause-time n
seconds, please contact <my-mail-address> if this causes problems." to
wget so that the administrators can see what´s going on.

How does this sound?

-- Antti


>
> Neil
>