Re: What's the best configuration for this setup?

Possibly the best way to throttle the connection is to let the TCP window
size handle the throttling. For example assume you have 2 connections, one
from Squid to a WWW server called the "network" connection. Another from
a browser to Squid called the "local" connection. You continue reading data
from the network connection until you have a small amount buffered ready
for the local connection to read. When the local connection reads the data,
you then read some more from the network connection until the buffer fills
again. You would see a small burst at the beginning of the transfer as the
buffer gets filled at a high rate, but then it would throttle back to the
speed that the client reads. Best of all no timers to manage. :-)

Currently Squid would just read from the network connection whenever it has
data via the select call. If Squid could know how much data it has that the
local connection hasn't yet read, and would not read the network connection
when the outstanding data is above a certain size, then it should work.

Any chance of doing this in a 1.1.beta version Duane?

Cheers.

Oskar Pearson (oskar@is.co.za) wrote:
> Daniel O'Callaghan wrote:
>
> > On Fri, 22 Nov 1996, Ross Wheeler wrote:
>
> > > I noticed a particularly unpleasant problem I'd appreciate ANY input on,
> > > and that is that when someone requests a page through our proxy, and it
> > > is NOT cached, it checks the "parent". IF upstream has it, (and here's
> > > the catch), it comes roaring down the 128K and consumes most of our
> > > bandwidth while it does so. If it's a large file (big web page, or worse,
> > > a large file that's being FTPd), this can go on for quite some time.
>
> > This was discussed on freebsd-hackers not so long ago. It is possible to
> > bandwidth limit connections by a simple
>
> > while () {
> > read();
> > sleep();
> > }
>
> > If course, sleep(), or usleep() are not appropriate for squid, but how
> > hard would it be to implement an algorithm which says "OK, I'm supposed
> > to read max 3000 chars/sec; I just read 1500 chars; I must not read again on
> > this connection for at lease 500 ms."??
> Or you could set a counter on each filehandle, decrease the number on the
> filehandle each time that you were "supposed" to transmit the data to them,
> and store data in a buffer until the counter=0 When it reaches 0, put X bytes
> of data in an outgoing queue and set counter=10 (for eg)
>
> Of course, if you have a T3 to the net and someone downloads Netscape via modem
> through the cache and is bandwidth limited, you have to hold the whole of
> netscape in those buffers (yes, the whole thing is anyway, but on a really big
> cache you are going to have to swap your current position in memory so fast
> that you are going to get latency from changing pages... Yes this is valid,
> a.root-servers.net has this problem)
>
> I think there are much more important things to be done. (I don't know if
> some of them have been done in 1.1bX - I haven't been watching)
>
> If (for eg) www.microsoft.com has 9 IP's, try connect to each one if you
> get something like a "cannot connect to remote host" message.
> Respect the TTL's that are returned with DNS responses
> ?Objects living forever in neighbor caches? (Must have been worked out?)
> Documentation (If someone has some kind of base framework, or chapter
> list that is reasonable, I am volunteering...)
>
> > My guess is that it would need a bandwidth and an "earliest time for next
> > read" to be stored for each (outbound?) tcp connection.
> > One could also limit write()s so that delivery is throttled.
>
> It wouldn't work if the slow person has the middle cache set as a perent
> (unless I am a fool ;) hey, no comments in the corner there!
>
> He is fed by ISP-A, he is ISP-B, he feeds ISP-C
>
> presumably he wants to feed ISP-C as fast as they can get, so he can't
> put his patch on himself. If he gets ISP-A to bandwidth limit, they are
> going to limit him too, as he is a PARENT to ISP-C.
>
> ie
> ISP-C requests doc from ISP-B
> ISP-B requests doc from ISP-A
> ISP-A limits ISP-B? WRONG!
>
> So ISP-C would have to have be set up with ISP-B as a neighbor and ISP-A
> as a parent. Squid on ISP-A would have to then bandwidth limit ISP-C directly.
>
> Oskar
>
> =============================================
> 'experience made art, but inexperience luck.'
> =============================================
>

-- 
--            +------------------------------------------------------------+
        .     | John Saunders  - mailto:john@nlc.net.au            (EMail) |
    ,--_|\    |                - http://www.nlc.net.au/              (WWW) |
   /  Oz  \   |                - 018-223-814 or 02-9477-2881       (Phone) |
   \_,--\_/   | NHJ NORTHLINK COMMUNICATIONS - Supplying a professional,   |
         v    | and above all friendly, internet connection service.       |
              +------------------------------------------------------------+