NETBRAKE readme

  Copyright(C) 2001 Salvatore Sanfilippo
  All right reserved

  Distribution of this document is unlimited, providing that
  you include the above copyright notice.

OVERVIEW

  Netbrake is an utility to limit the bandwidth used by a process.
  It does not require to modify the kernel, nor to be root in order
  to work. It is useful when you need to download a very big file
  from a fast server to avoid a network congestion that will result
  in a too slow web/irc/... experience.
  I use it mostly to download big files (like the kernel source code)
  with wget.

  Netbrake also implements a very simple HTTP filesystem extension,
  so you can use the standard text utils against some URL, like:

     cat http://www.google.com
     grep -i href 'http://www.google.com/search?q=foo'

  and get the expected result. While this support is not complete
  it may be useful sometime.

  Netbrake consists of a little dynamic library (libnetbrake.so). This
  library provides wrappers for important system calls, this makes
  possible the bandwidth limitation. To override the libc syscalls
  wrappers with the netbrake's ones you need to use LD_PRELOAD. To
  improve the easy of use a simple program (netbrake), that exports the
  right LD_PRELOAD environ variable and run the specified command
  (the one you need to limit) is included in the distribution.

  Netbrake is free software under the terms of the X11 license,
  see the LICENSE file in the distribution for more information.

SUPPORTED SYSTEMS

  Linux/glibc under all archs (tested only on i386 and sparc)

  Probably other POSIX systems with glibc and a working LD_PRELOAD
  with minor changes.

INSTALL

 $ ./configure
 $ make
 $ make install

SIMPLE USAGE

  To run a program with the required bandwidth limitation just use
  the netbrake program. The usage is the following:

  netbrake wget http://www.kernel.org/....

  you can specify some option to change the behaviour:
  This is the help screen of the 'netbrake' utility.

--------------------------------------------------------------------------------
Usage: netbrake [options] program [program args]

List of available options:
+  --bps        -r <bps>       Set the max bytes per second limit
   --notcp      -T             Don't limit TCP sockets (usually limited)
   --noudp      -U             Don't limit UDP sockets (usually limited)
*  --rcvbuf     -b <bufsize>   Set the RCVBUF size (default is 8k)
*  --debug -d                  Enable debugging messages
*  --limit-getc -g             Limit getc() and fgetc() (default is NOT)
*  --httpfs     -w             Enable the HTTP filesystem.
   --httphdr    -H             Include the HTTP header with HTTP fs
   --bpsinfo    -p             Print on stderr info about bandwidth usage
   --limit      -l             Enable the bandwidth limitation. It is ON
                               for default but you can use this to override
                               the NETBRAKE_NETLIMIT=0 environ var.
   --help       -h             Show this help

+ = note that the default BPS limit is 2k for second (2048 bytes)
* = use it only if you know what you are doing
--------------------------------------------------------------------------------

  I'll try to explain better what this switchs can do:
  
  -r <bytes per second> to set the maximun bandwidth to use.
  This is probably the main option for the user, example:

  netbrake -r 2048 wget http://www.kernel.org/...

  will se the max bandwidth to 2k per second, that is the default
  bandwidth (if -b is not used).

  -d switch to turn on the debug mode.

  This is not useful if you aren't a programmer, but you should use
  it if you want to report a bug.

  -T to turn off the TCP sockets bandwidth limitation

  -U to turn off the UDP sockets bandwidth limitation

  -b to set the RCVBUF for the socket. Don't touch this unless you
     know what you are doing.

  -w Enable the HTTP filesystem. Example:

  $ netbrake -w md5sum http://www.google.com
  d6102f7fc57d4bd279c7fa24c7b99e86  http://www.google.com

  As shown the HTTP filesystem provide a way to use http
  URL with normal text utils. You need to specify the URL
  as http://HOST[:PORT]/PATH.

  This support is for default NOT compiled, ask yes to the right
  question when you run ./configure to enable it.

  -H includes the reply HTTP header with the HTTP filesystem,
     usually you don't want this.

  -p print the current and average bandwidth used by the limited
     process. Note that the information displaied is about the
     limited sockets and not about all the traffic generated
     by the application.

  -l Turn ON the bandwidth limitation in the case you exported the
     NETBRAKE_NETLIMIT to "0".

  -h shows the shown help screen and exit.

HOW IT WORKS

  Libnetbrake wrappers syscalls like socket(2), read(2), write(2)
  and so on. Once the first call to socket(2) is trapped
  libnetbrake intializes the internal stuff and starts tracing all
  the AF_INET family socket descriptors. Every socket that is relevant
  is set in a bitmap so we have a way to take the list of the socket
  descriptors we need to trace without to waste too CPU.

  For ever read/write operation a list with the few last time/bytes
  information is compiled. This allows libnetbrake to get an idea
  about the (almost) 'instantaneous' speed. this speed is for default
  computed using the last 10 samples. If the application breaks
  this limit it is put in sleep to adjust the download speed.

  If the other side of the connection is sending the data too fast
  for our 'bps' limit the process will be put in sleep often, this
  will fill the socket buffer and our system will publish a TCP
  window of 0 soon. When the application will read new data from
  the socket, the socket buffer will be again ready to accept
  new data, so the TCP stack will send some ACK with a non-zero
  window to the other and. And so on.

  To make the download speed very precise we also limit in the
  same way (putting the process in sleep) the average speed of
  the download. This will correct errors with the instantaneous
  BPS guessing.

  BTW the socket recv buffer can be very big, for example in linux is
  87380 bytes for default, so for little files the limitation is not
  effective since the buffer will be filled receiving the data ASAP anyway.
  To fix this problem libnetbrake will set a smaller socket recv buffer
  for every AF_INET socket() the application will open.

  The HTTP filesystem uses the same trick, it isn't a full
  implementation and may not work with many applications, but
  I tested it with many standard unix utils without to experiment
  too problems.

  The HTTP filesystem wrappers the open(2) and stat(2) syscalls.
  when the application try to open a file in read-only and that
  file contains a string like http://... netbrake open an HTTP
  connection against the specified web server/port and do an HTTP
  request, than pass the descriptor as it was returned by the real open(2).
  If the request was successful the application will read the HTTP
  server reply like a file, unless it try to do operations that aren't
  allowed for a socket, like lseek(2).

  The same for the stat(2) syscall. netbrake will do an HTTP 'HEAD'
  request to get the size and the last modification date of the remote
  file, to fill the "struct stat" structure. The inode is computed
  hashing the URL itself.

TESTED APPLICATIONS		COMMENTS

  wget				(this is my main target)
  linux ftp client
  lftp
  w3m 				(needs --limit-getc)
  lynx
  tcl

APPLICATION THAT SEEMS TO WORK WITH THE HTTP FILESYSTEM

  cat sed md5sum tail head grep vim cp ls diff xv

KNOWN PROBLEMS

  netbrake can't get a real bandwidth control. It reaches its
  goal using hacks, so probably it will not work with some
  application.

  It can't be used against multithread applications, the behaviour
  is undefined, but it shouldn't be hard to fix.

  The http filesystem is experimental and doesn't work with
  all the applications, expecially the one that instead to
  call open/stat /path/file do something like cd /path open file.

SEE ALSO

  RFC1945

CONTACT INFO

  For feedbacks and bug reports write to <antirez@invece.org>

NOTE

  The project is abandoned because I finally got ADSL, so 0.2
  (that fixes a compilation bug in new versions of libc) is probably
  the last version of netbrake from me.
