[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: PLEASE TEST snapshots (long)



Coordinated testing is always a good idea.  Given the number of ports,
about 6000 for all the platforms, there more than one year of full time
work to test everything, and that doesn't include the ports.

Thats why I created the stupid little system at 35.8.65.46, so it could
be the start of something that people might work together on.  Static
web pages to hold info on testing is desperataly bad, but it gets
something up for the moment.  A better database system could be
built later.

So yes, there is interest in such a system from here.  I'm not sure
that a different mailing list would be needed for it, but perhaps.  My
ideas on this have evolved into a system where packages are 
checked into the database, with falvours for the different platforms,
and comment areas for each along with the name of the tester.

6000+ packages is an incredible amount to test, and there will
only be more in the future.

I don't think this idea will encounter resistence from the developers,
but will be met with some skepticism in that people often talk but 
don't do anything.  Thats why I came up with the idiot system I 
did--do, and improve on it, but actually do something.  This is NOT
to disparage what you are saying--I like it.  We should talk after
the current testing phase is done.

--STeve Andre'

On Saturday 06 April 2002 04:23 pm, Rick Robino wrote:
> > On Thu, Apr 04, 2002 at 09:46:33PM +0200, Marc Espie wrote:
> > 3.1-beta is really, really close.
>
> ...
>
> > Please test the damn packages !
> > Please test the damn packages !
>
> ...
>
>     Request for Comments Regarding User-community Test Bed:
>
> Would anyone be interested in an _informal_ but somewhat organized
> coalition of advanced users establishng a test farm for ports and
> general snapshots?
>
> I know that there are alot of people who thoughtfully keep themselves
> out of the main kitchen, but might be able to offer different arch's
> and versions without too much oversight on their part. Despite the
> potential for abuse, an array of test systems loosely open to a wide
> audience could be valuable for the individual who has something to
> contribute and test, but doesn't have the time or money to keep a
> properly varied test setup on hand.
>
> Most people on these particular lists can likely handle the operational
> issues such as quarantine for one box at a time, updating from cvs,
> dealing with out of space issues, reinstalling and banning after a
> bad apple, their own self-defense and good behavior on other people's
> systems, and so on. Many also are likely to have an extra non-i386
> machine looking for a purpose.
>
> If approved, such an effort would need to set up some coordination
> facilities, such as a website, mailing list - agreements on how and
> what to use for regression tests. If some very loose and easy to
> abide by standards could be agreed on, the effort could produce a
> more accurate report from the field on what is working, and other
> benefits - maybe fewer surprise bugs in less popular ports.
>
> This is just an idea. Seems to me that this kind of thing could fly
> if very few regulations or other barriers to experimentation are
> imposed. Just a bare minimum to get a better read, even if on an
> inscrutable personal level for the experimenter. It would be outside
> of and support in a (small?) way higher food-chain projects like
> ports and obsd-only original development.
>
> Sorry for the length - I am not proposing more complication for the
> folks who are already very busy - not even to worry about this
> becoming an orphan they might have to deal with in the future.  Just
> a little bit of herding and it could be destroyed if it fails to
> be useful enough or outlives its purpose. I think anyone of like
> mind would agree that such a burden doesn't deserve to live.
>
> Feedback? It is probably not a good idea to clutter the lists I
> posted to (tech and ports) with the responses. Send them to:
>
> obsdtest_(_at_)_wavedivision_(_dot_)_com
>
> I am willing to contribute some planning, various support services,
> some hosting, a box, and to take responsibility for housekeeping
> like status reports.
>
> Of course, if any of the full-timers (and you know who you are)
> think this is a bad idea, I'll respect that. Just thought I'd
> dip my toe in before I went to alot of effort. Whoever weighs
> in, I will post a much shorter note in a week to update folks
> on what the dialogue results were.
>
> p.s. I'd rather keep this out of misc right now; no offense to
> them, just probably not the pertinent audience.