[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Performance Tracker project update
- To: Erik Cederstrand <erik_(_at_)_cederstrand_(_dot_)_dk>
- Subject: Re: Performance Tracker project update
- From: gnn_(_at_)_freebsd_(_dot_)_org
- Date: Fri, 01 Feb 2008 14:55:01 +0900
- Cc: Kris Kennaway <kris_(_at_)_freebsd_(_dot_)_org>, "freebsd-performance_(_at_)_freebsd_(_dot_)_org" <freebsd-performance_(_at_)_freebsd_(_dot_)_org>
At Wed, 23 Jan 2008 14:49:14 +0100,
Erik Cederstrand wrote:
> Kris Kennaway wrote:
> > This is coming along very nicely indeed!
> > One suggestion I have is that as more metrics are added it becomes
> > important for an "at a glance" overview of changes so we can monitor for
> > performance improvements and regressions among many workloads.
> > One way to do this would be a matrix of each metric with its change
> > compared to recent samples. e.g. you could do a student's T comparison
> > of today's numbers with those from yesterday, or from a week ago, and
> > colour-code those that show a significant deviation from "no change".
> > This might be a bit noisy on short timescales, so you could aggregrate
> > data into larger bins and compare e.g. moving 1-week aggregates.
> > Fluctuations on short timescales won't stand out, but if there is a real
> > change then it will show up less than a week later.
> I agree that there's a need for an overview and some sort of
> notification. I've been collecting historical data to get a baseline for
> the statistics and I'll try to see what I can do over the next weeks.
> > These significant events could also be graphed themselves and/or a
> > history log maintained (or automatically annotated on the individual
> > graphs) so historical changes can also be pinpointed.
> > At some point the ability to annotate the data will become important
> > (e.g. "We understand the cause of this, it was r1.123 of foo.c, which
> > was corrected in r1.124. The developer responsible has been shot.")
> There's a field in the database for this sort of thing. I just think it
> needs some sort of authentication. That'll have to wait a bit.
> > P.S. If I understand correctly, the float test shows a regression? The
> > metric is calculations/second, so higher = better?
> The documentation on Unixbench is scarce, but I would think so.
> BTW if anyone's interested my SVN repo is online at:
> svn://littlebit.dk/website/trunk (Pylons project)
> svn://littlebit.dk/tracker/trunk (sh/Python scripts for runnning the
> server and slaves)
> Be careful with your eyes - this is my first attempt at both shell
> scripting and Python :-)
BTW This is excellent work and, if I get the time, I'll be using some
of this at my day job.
freebsd-performance_(_at_)_freebsd_(_dot_)_org mailing list
To unsubscribe, send any mail to "freebsd-performance-unsubscribe_(_at_)_freebsd_(_dot_)_org"
Visit your host, monkey.org