Re: Bird Counts and Statistical Validity

["Steve Compton" <scompton@sc.rr.com>]

Nathan,

As a 15 year Breeding Survey participant and a
Christmas birder for 22 years, and as the former compiler
of the Charleston count, I have to say I agree with Charlotte. The "errors"
you refer to are largely
obviated by the careful methods under which a complier
must submit the data for a Christmas Count. Remember,
we are looking for population trends, not small differences
in rare or accidental occurances. The BBS certainly has
a more "reliable" methodogy, and there are weaknesses
in certain counts, but I would maintain the Charleston
count data is very reliable for trends of local bird popualations over the
past 40 years. Just test the data, compare the data to any other measure and
you will see.
It is remarkable how predictable the results for a Christmas count can be.
This for me is a strong indication
of statistical reliability.

Nathan, I would recommend you compile a Christmas count for a few years and
test your theories. In any event,
it is fun to read your opinions and to be with you in the field. I look
forward to that opportunity again.

Yours for truth and science,
Steve Compton
scompton@sc.rr.com


----- Original Message -----
From: "Nathan Dias" <diasn@yahoo.com>
To: "Charlotte Goedsche" <goedsche@mindspring.com>; "carolinabirds"
<carolinabirds@duke.edu>
Cc: <amw25@cornell.edu>
Sent: Tuesday, February 18, 2003 3:41 PM
Subject: Bird Counts and Statistical Validity


> Charlotte -
>
> Just because so-called "scientific" studies use
> Christmas count results does not mean they would
> withstand rigorous evaluation by a statistician.
>
> Lookit:
> When considering studies:  in general, samples are
> taken and inferences about the overall population are
> made by comparing samples.  If one cannot trust the
> samples themselves, one cannot place much trust in the
> inferences drawn from their comparison.  The name of
> the game in applied statistics is to minimize the
> uncertainty attached to these inferences.  I shall
> demonstrate how this cannot be done to an acceptable
> degree with data resulting from Christmas Counts and
> GBBC.  My claim was not irresponsible, Charlotte; it
> is founded in statistical bedrock.
>
> Here are some barriers to studies based on GBBC and
> Christmas Count results being statistically valid:
>
> - Too many novice/untrained/kooky observers (who might
> misidentify birds).  Both the spatial distribution and
> the number of observers of varying competence is
> uneven and non-random from one count to the next.
> STRIKE ONE.  Example:  A phone survey/poll asks
> respondents questions with cut-and-dry answers.  The
> folks compiling the results "know what they are up
> against" (they know for sure that the respondent said
> they favored the U.S. going to war with Iraq).  But
> folks using Xmas counts or GBBC results don't know for
> sure if Suzy Dimwit really saw a given Loggerhead
> Shrike or misidentified a Mockingbird.  This is a
> hindrance to having a representative sample.
>
> - Grossly inaccurate numeric data (see my previous
> post on double-counting, under-counting,
> mis-estimation of flocks, etc.).  Everyone knows that
> Christmas count species totals that folks turn in are
> ballpark guesstimations (way too high or too low) in
> many cases.  This numeric data is also inconsistent
> (and non-random) in its inaccuracy.  STRIKE TWO.  In
> and of itself, this doesn't mean a study based on the
> data is not statistically valid, but it does mean the
> uncertainty attached to inferences drawn is so high as
> to make the inferences of much less use.
>
> - Uneven/inconsistent coverage of the same physical
> area(s) from year to year.  STRIKE THREE.  This can
> make comparing data from different years like
> comparing apples to oranges.   NONREPRESENTATIVE
> SAMPLES result from this practice (a cardinal sin if
> one is seeking statistical validity).  Example: many
> christmas counts have sub-areas that are not
> "glamorous."  They receive little or no coverage some
> years, half-hearted coverage in others and good
> coverage by heroic experts in others.  This coverage
> is uneven and nonrandom.  The same argument applies to
> zip codes in the GBBC.  Another uneven coverage
> problem results from the need for boats to cover water
> areas of Christmas Counts.  Another uneven coverage
> problem (on Christmas Counts) happens when land
> (especially plantations) gets sold to
> non-birder-friendly landowners.  Having coverage
> depend on the whims of a cranky landowner is
> definitely inconsistent and nonrandom.
>
> - Due to the relatively low number of observers versus
> total bird population size, the high number of
> non-expert observers, inconsistent sample area
> coverage, inconsistently inaccurate data and many
> other factors: Xmas Counts and GBBC produce
> unacceptably high probabilities of alpha error and
> beta error.  STRIKE FOUR.  -- Alpha error is when one
> sees a statistically significant difference in the
> samples, when no difference actually exists in the
> populations.  Beta error is when one does not see a
> statistically significant difference in the samples,
> when a difference actually does exist in the
> populations.  When probabilities of alpha error and
> beta error are too high, one can throw inferences
> drawn out the window.
>
> I could go on and on with the strikes.  The
> nonrepresentative sample issue alone means NOT
> STATISTICAL VALID and it keeps cropping up due to
> various considerations.  QED.
>
> Breeding Bird Atlas studies are much better (in terms
> of statistical validity) than Xmas counts and GBBC -
> they use highly competent observers, identical routes
> (areas of coverage) over time, identical time spend
> traversing the route from year to year, and more.
>
> Nathan Dias - Charleston, SC.
>
> PS   My father (who has a PhD in Biostatistics) also
> gets red in the face at the thought of folks claiming
> that studies based on Christmas Count and GBBC results
> are statistically valid.  As someone with degrees in
> (pure) math and Computer Science, as well as A grades
> in graduate level courses in Statistics, I agree
> whole-heartedly.
>
> --- Charlotte Goedsche <goedsche@mindspring.com>
> wrote:
> > As a person who birds for fun AND is interested in
> > bird distribution, I'd
> > like to address two issues Nathan Dias brought up in
> > a recent posting on the
> > Great Backyard Bird Count and the Christmas Bird
> > Count, both conducted in
> > North America.
> >
> > His claim that Christmas Bird Counts do not produce
> > "hard scientific
> > (statistically valid) data" is simply irresponsible.
> > Check out the following
> > website for a vast bibliography of scientific
> > studies based on Christmas
> > Bird Counts: http://www.audubon.org/bird/cbc/bb.html
> > . (Since the GBBC is
> > only in its 6th year, it is, of course, too early to
> > expect published
> > scientific results from it.)
> >
> > Checklist programs similar to the GBBC have been
> > used in Canada for many
> > years, by the way. See
> > http://www.oiseauxqc.org/feuillets/cbcp_i_e.html for
> > a history of the Canadian checklist program and some
> > of the publications
> > based on it.
> >
> > While Nathan's claim that "Gut feelings by
> > experienced birders are of
> > about as much use [as the GBBC] (and more likely to
> > spot trends
> > quicker)" may have some validity on the local level,
> > it ignores the
> > geographic and temporal scope of the GBBC, as well
> > as the organizational
> > framework it provides. The GBBC (like other counts)
> > collects masses of
> > personal observations and transforms them into data
> > which can be shared and
> > compared, and will eventually provide valid
> > information on late-winter
> > population dynamics and trends which I expect will
> > lead to scientific
> > publications.
> >
> > So have fun birding, and share the observations you
> > made between 14 and 17
> > February by submitting them to the Great Backyard
> > Bird Count,
> > http://www.birdsource.org/gbbc/ . Checklists will be
> > accepted through 28
> > February, so it's not too late to send in your
> > sightings. And don't forget
> > that at the same site you can also see what species
> > have been reported
> > where, and view maps to compare the distribution of
> > species of interest to
> > you over the years.
> >
> > Good birding!
> >
> > Charlotte
> > Charlotte Goedsche
> > Weaverville NC
> > goedsche@mindspring.com
> >
>
>
> __________________________________________________
> Do you Yahoo!?
> Yahoo! Shopping - Send Flowers for Valentine's Day
> http://shopping.yahoo.com