DeSovietizing OBSD's core install.

[Chuck Yerkes <chuck_(_at_)_snew_(_dot_)_com>]

Quoting Han (han_(_at_)_mijncomputer_(_dot_)_nl):
> OpenBSD is focusses on a secure default install. The default install
> consists off a working set off free software under the BSD licence.
> Not all software has been develloped by the OpenBSD teammembers but
> they do have full authority to the code. And thats why they have to
> consider the license under which the sourcecode is distributed.
> Furthermore they need to have a complete set off software.
[...]


FreeBSD is moving towards (or at least seriously studying)
turning the monolithic code base into smaller packagettes.
A goal, as I heard it from Jordan at a small talk, is to allow
for incremental upgrades and management of smaller sections
of the system as you describe.  It also helps as FreeBSD gets
into smaller devices (I don't want all the cruft that of
ethernet networking on a stand-alone data gathering device,
nor named, nor most of /usr/bin).


I've been thinking about this quite a bit and have toyed with
bits of it over the years.

Personal reference points for me come from installing SunOS 4
a lot and later Solaris (10 minutes ago).  I could choose not
to install the "UUCP package" easily.  Right now I do this by
knowing the directories that have unwanted things and adding
them to SKIPDIR.


The first task is describing all the sub-systems and dependancies.
The hardpart is redoing the make architecture to allow for
learning and installing only the desired bits (or, more
precisely, NOT installing the undesired bits).  This is described
at the bottom.

This would mean that a BIG chunk would be in a package that
I'd call OBSD-BASE.  This would always contain libc and what
you need to get to single user.  At the start, it would contain
EVERYTHING.


Of the current base, obvious things that could break out of that
might be moving lib*.a and .h files into a compiler 'package',
identifying the obvious subsystems such as
- DNS service, DNS service/devel package (.h and libs).
- mail service, mail service/devel
- networking kernel stuff (devices, ifconfig, some others)
- networking userland stuff (r* commands, inetd.conf, etc).
- PPP (depends on networking)
- SLIP (depends on networking)
- secure networking userland and kernel stuff (IPSEC and friends
  that aren't built in -depends on networking).
- filesystem stuff (some NEEDS to be in core, but foreign fs's like
  MSDOS, Amiga, etc are extra).
- TERMINAL stuff (stty, terminal descriptions, things that are
                  useless on a box with no terminals at all).
- Kerberos stuff (which depends on some network stuff, but needs to
                  be there for network stuff to be compiled).

The list goes on (and really isn't THAT huge).

The rest goes into BASE (by default).



I've played with "list every top level package directory" in source.
- Identify which files (including .h and libs) go with each package.
- Identify which packages go together.
- Identify which packages depend on others.

Describe this for a make file.

So if the master make file says compile CORE (and CORE includes
certain libs/, certain .h files, certain userland files, etc)
that must be described.

Then the other parts get defined.  SKIPPACKAGE can do what
SKIPDIR does in a more meta way.  SKIPPACKAGE=sendmail would
skip that package (in this case it would be the same as
SKIPDIR=gnu/usr.sbin/sendmail in /etc/mk.conf).

Basically, there a level of abstraction that turns a PACKAGE
into a series of source directories (phase two) and later into
sub parts of directories for the INSTALL part of make (phase
three).  It can do, or fake, a pkg_add into perhaps a dir other
than /var/db/pkg/ for separation's sake (I'll toss out
/var/db/pkg/obsd/ as the dir.)  Add also a make package phase
for these so that you can easily have a build machine that is
not the production machine and change subsystems.


------------------------------------------------------------
PLUSES:
- Better able to offer patches (binary)  AND upgrades (source).
- Able to remove or not include subsystems
  (for security, managability, specialized systems like my
  Soon-To-Arrive Soekris board with only 64MB of Compact Flash, etc).
- Able to build subsystems and transport in binary to another box.
- Able to account for subsystem integrity better (tripwire/mtree
  can handle a lot of this now).
- First phase is pretty much to 1:1 match as subsystem.
- Workload of breaking it up can be distributed over time 
  and over many people and allow for easy contribution by
  those who don't code - identifying what gets installed
  by rsh as far as .h files, man pages, and binaries is
  not daunting.

MINUSES:
- It means modifying some of the make(1) structure - this is a system
    that *I* don't well understand, personally.
- Difficult to support partial systems
  E.g. "oh, you don't have 'mopchk,' well that's gonna make
  it hard to debug your diskless Ultrix box or DEC terminal server."
  On the other hand, I don't WANT mopd and brethren on my boxes.
  + I offer that it's "as is" just like putting SKIPDIR entries
    into mk.conf - if I choose to.  If someone puts ifconfig into
	SKIPDIR they are in the same realm of pain and unsupportability.
    So we aren't ADDING to the support issue, we're just making
    SKIPDIR a bit more coherent.

This should NOT MEAN:
- More tags in CVS (I don't want named-stable/named-snap).
- Headaches for the developers.
     With a "recording" install(1), all stuff goes, by
  default into a package.  Efforts would take place to
  remove it from the default package (BASE).  These efforts
  are described below and can be done by non-programmers given
  an infrastructure.
------------------------------------------------------------
How would this be approached?
PHASE ONE:
  Change the install so that the files installed/copied from the
    source tree into the live system get noted somehow.
    Default is BASE using /var/db/pkg/OBSD/ (terms may change).

PHASE TWO:
  Identify subsystems in a rough way.  E.g.
    sendmail=gnu/usr.sbin/sendmail/
    games=games
    ppp=usr.sbin/ppp usr.sbin/ppp usr.sbin/pppd usr.sbin/pppoe
    slip=sbin/slattach usr.sbin/slstats usr.sbin/spppcontrol\
        usr.sbin/sliplogin
    uucp=gnu/libexec/uucp libexec/uucpd
    yp=usr.bin/ypcat usr.bin/ypwhich usr.sbin/yppoll usr.sbin/ypset \
        usr.bin/ypmatch usr.sbin/ypbind usr.sbin/ypserv
    rtools=/usr.bin/rsh usr.bin/ruptime usr.bin/rusers usr.bin/rlogin \
        usr.bin/rwall usr.bin/rup usr.bin/rwho
    IntelPCtools=sbin/mount_msdos \
        sbin/wsconsctl/wsconsctl.h usr.sbin/wsconscfg

  AND MORE.

This stuff can be integrated into the TOP level Makefile and expanded
into skipdir (presume they want everything and turn off specifics).
This can be a long discussion.  Where does inetd/inetd.conf go?  I
don't use them.  Should it be separate?



PHASE THREE:
  Start into the makefiles to start breaking out parts of these
  monolithic systems into TARGETS that are needed for use and
  TARGETS that are needed for devel - this ONLY affects the
  INSTALL phase of the build.  

  named installs: /usr/sbin/named, named-xfer, named.boot, etc.
        It also installs MAN pages (not desired on a pico machine)
        It also installs include files and static libraries (ibid)
  This phase requires lots of work.  But it can be, er, phased
  in.  Going from monolithic to detailed means that one package
  can be tackled at a time and added to the list of pkg.install
  files.  If the variable exists, then use it, otherwise treat
  as monolithic and either install it or don't.


------------------------------------------------------------
PARTIAL TASK LIST:
- A "recording" install:
  At some point, the install process should start to "fake"
  packages (eg. create +CONTENTS and do a checksum of appropriate
  files) - perhaps wrap the install(1) program or expand that
  program to leave a trail).
- Decisions on names.  I offer these with changes welcome:
  1 OBSD subsystem packages go into /var/db/pkg/OBSD/
  2 The files installed by each sub-directory are listed in
    Perhaps this is saner to maintain inversely in the Makefile as
       pkg.named="named named-xfer libbind.so"
       pkg.named-devel="foo.h libbind.a"
       pkg.named-man="named.8 dig.1"
    and so forth.
    Should it be fully pathed?  Should it exist only in the
    top level Makefile?  The make folk can answer better.
    


------------------------------------------------------------
Okay, I'm tossing this out for discussion.  Love to hear about
where to get a better grasp of OpenBSD's Make infrastructure
(I could never figure out how to install source MAN pages
rather than the cat'd ones).

1) "Me too" comments can go to AOL, please.
2) Useless flames should be redirected to "nobody_(_at_)_localhost_(_dot_)_"
   If you can't construct a coherent sentence with constructive
   thoughts, this is you.


Discuss and code,


Chuck Yerkes