[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

An interesting little ftpd gotcha


The site I run has various downloads available via FTP and I run a set of
logs at the end of each day based on /var/log/ftpd

As any ftp site operator will attest to, download accelerators are the
work of the devil (<g>).  They open up tens of connections to your server
and because they're all hitting at once, put a massive load on the machine
if the file they're all attempting to retrieve is fairly large.

The "download accelerators" do their thing by doing the equivalent of
restarting a download at a certain point (REST) and then aborting it when
it reaches the start point of the next segment it's requested.  The end
result is, for example, 10 logged requests in /var/log/ftpd.  The only
problem is that instead of logging the amount of bytes transferred in that
particular retrieval, it logs the full data size of the file.

Say, for example, someone downloads a 30 meg file with a regular client...
You end up with a single log entry for a 30 meg file.  Say someone
downloads the file with a download accelerator using 10 (!) simultaneous
connections, instead of 10 log entries showing approx 3 megs downloaded
each, the OpenBSD ftpd logs the transaction as 10 downloads of 30 megs,
which means the log is a full order of magnitude out of whack (dependent
of course on how many clones the thing opens up...).

The critical bit in src/libexec/ftpd/ftpd.c appears to be here:

        send_data(fin, dout, st.st_blksize, st.st_size,
                  (restart_point == 0 && cmd == 0 && S_ISREG(st.st_mode)));
        if ((cmd == 0) && stats)
                logxfer(name, st.st_size, start);

First it does the send_data including the offset that the user has
requested (ie a restarted download) but then logs the transfer with the
full size of the file that's been downloaded.

I make no claims about being a coder, but can read it and (hopefully :P)
spot where the issue is.  Is this something someone would be willing to
look at changing/fixing?  I'm able to test privately if required...


Grant Bayley                         gbayley_(_at_)_ausmac_(_dot_)_net
-IT Manager @ Foster Nunn Loveder      (www.fnl.com.au)
-Admin @ AusMac Archive, Wiretapped.net, 2600 Australia
 www.ausmac.net   www.wiretapped.net   www.2600.org.au