[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: anoncvs or cvsup?



shivak <anarkky_(_at_)_cyberwar_(_dot_)_com> wrote:

> I am trying to decide between anoncvs and cvsup. I've read the pages on
> openbsd.org, but I still cannot make the decision. According to the
> website, cvsup seems to be faster, more flexible, and more advanced.

That's not the real picture.  CVS and CVSup are entirely different
applications that have only minimal functional overlap.  They're
different tools for different jobs.

CVS is a revision control system for collaborative development.
It allows for basic remote access operation, which can be (ab)used
for network file distribution.

CVSup is a dedicated file distribution tool, highly optimized for
synchronizing directory trees over the network.  CVSup has special
support for CVS repositories, including the ability to check out
particular revisions of a tree from the repository.

Unsurprisingly, CVSup is substantially more efficient than CVS for
mirroring source trees over the network.  For the FreeBSD community,
this issue is closed.  CVSup is used almost exclusively there for
source distribution.  In fact, I know only a grand total of two
public CVS servers for FreeBSD:  anoncvs.freebsd.org (which is
constantly down or overloaded) and grappa.unix-ag.uni-kl.de (which
I'm a co-admin of).

The situation is somewhat different for OpenBSD, because
- there is no native build of CVSup, which forces us to rely on
  executables for other platforms that run under a binary compatibilty
  layer ("emulation"), and which are only available for i386 and
  sparc;
- there are no native OpenBSD CVSup servers, since we have only a
  working client but no server daemon.  All the server machines
  listed on cvsup.html run under operating systems other than
  OpenBSD, most likely FreeBSD.

That much advance information.

> But I don't like the compile issues with cvsup (modula 3...) and
> I've even heard about some obscure bug (like cvsup not being able
> to get some gcc source file?).

Right, CVSup in checkout mode will fetch a wrong revision of
src/gnu/egcs/libiberty/hashtab.c from the OpenBSD repository.
I blame espie ;-) because he didn't do a proper merge after a vendor
branch import, but yeah, it's a bug in cvsupd (that will be fixed
in the next release).

This bug, which is only triggered by an exceedingly rare condition
or it wouldn't have gone unnoticed for so long, should not detract
from the fact that CVSup is well-tested, rock-solid, and very
robust.

> I'm not absolutely sure about the latter, but the fact is I need a
> reliable, single way to get my sources. Specifically, I will download both
> openBSD and freeBSD sources to a central server, and the sources of my BSD
> machines will be synchronized from that machine

Now, before we worry about the exact tools to use, let's first
consider just what you want to fetch.  There are two alternatives:

- You can fetch the source trees of a particular branch, say,
  FreeBSD 4.x-STABLE (RELENG_4) and OpenBSD-stable (OPENBSD_2_8),
  respectively.  You could then distribute the source trees by
  making them available over NFS or maintaining local copies with
  rsync.

- Or you could fetch copies of the FreeBSD and OpenBSD CVS
  repositories (~1GB each).  These contain all branches and the
  complete development history.  This would allow you to check out
  different versions on different machines (say, -STABLE and
  -CURRENT) and to easily access the revision histories.  For
  example, you could trivially set up CVSweb to allow browsing of
  the repository.

  Distribution from the central server to your other machines could
  be accomplished by your own CVSup server if the central machine
  was a FreeBSD box, or by remote CVS over rsh, or by making the
  repositories available over NFS.

Your call.

Now, how to fetch the data on your central server?  The protocol choices
are:
- Repository:  CVSup, SUP, Rsync
- Source Tree: CVSup, CVS, Rsync

FreeBSD is readily available by CVSup, both the repository and
checked out trees.  If you want to insist on avoiding CVSup, then
you will find that alternative resources are rare.  You could
mirror the repository by rsync or check out a tree by CVS from
grappa.unix-ag.uni-kl.de.

If you already use CVSup to fetch FreeBSD, you might as well use
it for OpenBSD.  If you don't want to, you can mirror the repository
by rsync from grappa or by anonymous SUP from anoncvs.usa.openbsd.org.
If you only want to get a checked out source tree, there are plenty
of public CVS servers around.

> (I will have about 20 simultaneous users on an 800mhz i386 w/ a
> 6 drive raid 10 and 1gb of ram all on a 1gb ethernet - will the
> efficiency gains by cvsup be noticeable at this point? is cvsup
> faster only on the client side, or does it put less strain on the
> server as well?).

I'm not sure what you mean by 20 simultaneous users here.  It
appears pretty silly to me to insist on updating 20 machines in
parallel if you could easily serialize this task.  Anyway, comparing
server load is difficult because of the many variables involved,
but average load should be lower for CVSup if you consider that
CVSup runs are much shorter.  Generally, CVSup is limited by disk
or network speed.  If you don't run "cvsup -s", client disk speed
will typically be the limiting factor.

> Which method would be the most reliable for my situation? thanks.

I don't think that reliability is an issue here.  Well, CVSup in
checkout mode will happily restore even a badly mangled source
tree, whereas with CVS you better throw away the whole mess and
start a fresh checkout, but that's the only thing I can think of
where the term reliability would be applicable.

I can talk a bit about how I'm doing things.  grappa.unix-ag.uni-kl.de
mirrors the Free/Net/OpenBSD repositories by CVSup and makes them
available by rsync, CVS, and CVSweb.  We would offer CVSup too, if
there was a working server daemon for OpenBSD.  Our main reliability
issue has been the prolonged death of the main Micropolis hard
disk, since replaced.  For a couple of days, we suffered like pretty
much everybody else a corrupted OpenBSD repository when the main
server sent out bad data which propagated the mirror chain down to
us.  Our machine crashed once or twice from an instability in
OpenBSD's soft updates code.  We don't run soft updates any more.
Sometimes one of the CVSup updates from our master servers fails
because that machine is down.  The FreeBSD feed even disappeared
for several days from the DNS, and we subsequently switched to a
different one.  You will notice that these are probably not the
kind of reliability issues you had in mind.

Another real world example is my home setup.  This kind of grew
over time according to differences in need and disk space.

My FreeBSD box (kemoauc) fetches the FreeBSD repository by CVSup.
I also have a CVSup server there, and I use local CVSup--i.e.
server and client running on the same machine--to update the doc
and ports trees, because this is faster than CVS.  I use CVS for
the src tree, because I need finer granularity there, since I
sometime update only selected parts of the system.

I also mirror the NetBSD repository on kemoauc.  I don't run any
NetBSD machines, but I like to keep the repository at hand for
reference purposes.  Initially I fetched a copy by rsync, now
occasionally updated by CVSup.

I have an OpenBSD box (ganerc) mirror the OpenBSD repository by
CVSup and use CVS to locally check out the -stable branch of the
ports and src trees.  The repository is also exported by NFS to
another OpenBSD box (ariolc) where I check out the -current ports,
src, and www trees.

Finally, I have set up a CVSweb on kemoauc which draws on the local
FreeBSD and NetBSD repositories and again NFS-mounts the OpenBSD
one from ganerc.  I used to run CVSweb directly on ganerc, but
kemoauc is much faster, and it turned out that CVSweb is bound by
CPU rather than disk/network speed.

Well, this got a bit longer than I intended, but it should give
you an idea of what's involved.

-- 
Christian "naddy" Weisgerber                          naddy_(_at_)_mips_(_dot_)_inka_(_dot_)_de



Visit your host, monkey.org