Monday, March 6, 2017

eBird and the 365 Checklists: Quantity Over Quality?

Perhaps you would enjoy the comfort and familiarity of something well-established and obvious stated to you. Let's try this on for size...eBird is good, free, and always improving. It sorts your lists, informs you of rarities, gives you access to reams of data, and contributes to many scientific studies. I mean, chances are that you got at least one life bird because of eBird, and life birds are practically priceless.

That said, eBird does not have the kind of backing that Google or Oracle does. The people responsible for bringing eBird into the world have not become mega-celebrity billionaires as a result. There is no army of coders behind the face of eBird, gentrifying communities and causing white flight, no legion of brogrammers ruining your favorite local bars. For good or ill, eBird is just not that big.

What that means for us users is that change does not come at a rapid clip, which is fine...the real bugs in eBird or the app tend to get sorted very quickly, while new features are rolled out more slowly. Totally understandable. However, there is one particularly irksome thing in eBird that grinds my gears more than anything else...

This one particular shortfall of eBird is the 365 rule. Now at this point, if you are still reading this, I'm assuming you are an eBird user...if you're not, I apologize for how bored you must be right now, but I am not going to accept the blame for that. Anyhow, let me give an example of what I am talking about....

Joey Birdwatcher photographs a Common Goldeneye. He thinks it is a Barrow's and attaches his photo of it to a checklist. Barrow's Goldeneye is not flagged as rare where Joey Birdwatcher went birding, so the regional reviewers do not see his error.

The Great Ornithologist Felonious Jive, who is not an eBird reviewer, decides one day that he wants to look at pictures of Barrow's Goldeneyes submitted through eBird. He comes across Joey Birdwatcher's photo, which is obviously misidentified. Felonious sees the link to click to "report" the photo, and goes to click on it to report the misidentified bird. What he sees is this:



Now nerds, before you freak out, ML45885311 is not actually a misidentified photo, this is just an example for the sake of a screenshot, so calm down. Anyways, you will see that Felonious has only a single option here, and although he considers the misidentified goldeneye both offensive and inappropriate, he does not believe that eBird wants this photo to be flagged as such just because it features a misidentified bird. He figures they have something totally different in mind, like porn or a mound of dead squirrels. So what is Felonious to do? He is not a reviewer, but as you probably know, you don't have to be a reviewer to flag misidentified bird photos. Sadly, he "only" submitted 364 different checklists last year. Had he submitted 365 different checklists, eBird would have rewarded him with an entirely new power...


Wrong species...there it is. Had Felonious been in the 365 club, he would have seen something like this instead...indeed, he would have the power to flag photos of misidentified birds.

As a veteran birder who can identify the shit out of a great many birds, I'm not stoked about how photo-flagging privileges are awarded. It really seems to be that the quantity of checklists submitted has greater importance, in this instance, than observer quality. But let's face it, the number of eBird checklists submitted in a year has almost no correlation to the person's skill set as an observer. Many of the continent's most notorious stringers actually are out birding a lot and submit a great many eBird checklists. Baffling, but true.

The 365 rule also rewards obsessive birders who create checklists while stopped at red lights (no, I'm not making this up) or on daily binocularless walks to their office, which I guess can be fun for listing purposes but generally have very little scientific value. Are these the best birders to be doing quality control in eBird?

Another point worth bringing up is that the 365 rule penalizes users who have lives outside of birding...believe it or not, not all birders are retired, or wealthy/free enough to be able to bird most of the week, every week. If I still lived life as a Perpetual Weekend six months a year, 365 complete checklists would not seem out of reach, but alas I work full time, year-round, and now I have a baby to care for...let's face it, I'm not going to submit 365 checklists in 2017, and probably not in 2018 either. I will be birding regularly of course, and not to sound conceited (a popular character trait among birders) but I consider my ability to ID birds in photos to be above average. I will not stoop to stoplight birding just to bump up my checklist total, or make checklists for the walk from my driveway to my front door...I have my dignity, and know that these sorts of lists generally cater to the birder's dirtiest obsessive desires more than they do the scientific community.

Of course, it's easy to just complain about something without offering a better idea. An alternative solution? Give eBird reviews the ability to award users this power. Perhaps a reviewer can only give flagging privileges to a certain number of other users, so not everyone and their mother is patrolling eBird and mistakenly flagging photos as the wrong species. Of course not all reviewers have the best judgement either, but I think most would recommend the most skilled users who could flag misidentified photos.



What is the problem indeed?  I wish I could do something about this Orange-breasted Bunting, but alas I cannot. I am unworthy of such immense power...and I don't know who the appropriate reviewers are or how to reach them. But for the sake of finishing this post, I'm sure there are other good (better?) ideas out there for figuring out who should be able to flag misidentified birds...my solution is no Final Solution.

To reiterate, I am grateful for what eBird does already and the vast majority of changes they have implemented over the last few years...but this one is hard to swallow. 

22 comments:

  1. I'm also a "non-power" eBird user, and I look forward to improvements that make eBird more social and interactive. One laborious work-around for this issue (which you may know about already) is to contact one of the reviewers for that area. I did that recently for a crisp photo labeled "Common Redpoll", which was clearly a House Finch. Here's the link to the reviewer list: https://docs.google.com/spreadsheets/d/1i08drC6kGqequ_uRB6vgRdMaClcTljX0pvDzSS0ARic/htmlview?authkey=CL_Hj5AL&pli=1#gid=124519153

    ReplyDelete
    Replies
    1. Ah, thanks for that link, I had lost it and frequently wish I hadn't. I tried several search phrases in eBird's help section to try and relocate it, but it does not appear to be advertised anywhere (correct me if I'm wrong), which I assume is on purpose.

      Delete
    2. To find this document by searching eBird help, first search on "eBird review standards". In that document, search for "Download Review_Tool_and_Filter_Instructions_28Jan2015". In that pdf file, search for "A list of our current reviewers is maintained at this Google Docs spreadsheet: http://bit.ly/1CFRUlX"

      Delete
  2. Found the checklist through the photo and reported it. My reasoning was a link to this blog post. :D

    ReplyDelete
  3. Howdy, Hopefully somebody from eBird will chime in here but as an eBird reviewer myself I'm pretty sure this isn't a rule per se but just a first pass at opening up photo review to non eBird reviewers (I could even quote the email from Marshall which I've got open in another window). There are still kinks in the process that probably need to be worked out. For example currently there is nothing preventing multiple people from putting the same photo into review over and over again (even if it is correct) which would be a bother for the person actually doing review. And there are likely other issues beyond that. I'm sure if there are folks that really want to do photo review they could contact their local ebird reviewers and see how that could happen.

    ReplyDelete
    Replies
    1. It may not be a fixed rule, but it is indefinitely the current one, and some people thought it was a good idea when it was instated...hopefully they will change their minds.

      As far as checking with reviewers to get access to photo review, my understanding from other reviewers is that there is nothing they can do. I would be more than happy to be wrong about that though.

      Delete
    2. Media review has only been available since late 2015 so I suspect that we are still in the shake-down phase. There has been substantial guidance from eBird central on handling media review so I suspect the process will continue to evolve in the near future. I did email Ian Davis at eBird about adding additional reviewers which should be possible. And just today a reviewer noted a (correct) photo that keeps getting put into review even after it;s been validated, so that is an ongoing problem. There may be additional discussion on the eBird forum (there is also a feedback mechanism on the web site although responses are likely to take some time).

      Delete
  4. also, a check of the locale's Audubon grp would flush out the local list police, er, reviewer. *blush* I can sure tell you the name of the Central CA Coast ebird reviewer, and I assure you he is a very gracious, very patient man.

    ReplyDelete
    Replies
    1. I know the shit a lot of reviewers have to put up with...patience is a possibly the greatest virtue if you are a reviewer!

      Delete
  5. 1. In Mexico I learned life birds are not priceless. Frank has snowy owl on his life list now. Bought and paid for.

    2. I had no idea about this power! I can't believe I have this power! I cannot wait to fucking use it, even if the reason I have the power is stupid.

    ReplyDelete
    Replies
    1. Fuck I forgot about the owl transaction! I thought Dan bought it?

      Time to get flagging!

      Delete
    2. Oh you're right. Frank is way too cheap to pay for a bird.

      Delete
  6. Thanks for bringing this to light.
    As an ebird freeloader*, I've come across checklists for my local patch that were either misdated or mis-located (I'm trying to assume they're not intentional lies).
    Then last week I started looking at the Macaulay image search, for geese hybrids in particular. Some alleged GWF x Snow Geese were clearly Canada x ??? s. I was wondering if there was someway I could have reported these. I guess not, without getting reviewer email addresses.


    *I use the data but have submitted a total of two checklists. I know...
    See, I have about 4 years of Rite-in-the-rain notebooks and no smartphone and am very reluctant to spend my freetime doing data entry.

    ReplyDelete
    Replies
    1. Plotting locations correctly will like continue to be a problem for some time I think. Wrong dates are definitely more surprising.

      Classic misidentified hybrid. Check Evan's comment above for a link to the reviewer list.

      Delete
  7. In addition I personally think users should be able to flag species as a possible mis-ID even when there's no photos. You may have noticed down in Mexico how there's "records" of Sinaloa Crows, Bumblebee Hummingbirds, Golden-crowned Warblers and more from downtown Puerto Vallarta. Unfortunately they aren't flagged in their respective counties and can't be flagged by users.

    ReplyDelete
    Replies
    1. Yep, there are some serious filtering work to be done for Mexico. Seems like that work falls to the reviewers...maybe some of these places just need more of them.

      I will say that I do find it annoying when I'm birding in an unfamiliar place and I record an unflagged species, only to get a question about it from a reviewer a month later. Documenting everything is the only way to avoid this problem.

      Delete
  8. I review for Wisconsin where we have over 20 regional reviewers for our average-sized state. Most reviewers are avid birders for the counties they review and have their finger on the bird pulse to catch some of the non-flagged species that may have been ebirded erroneously. Though certainly even with so many reviewers, many errors still get through. I'm not sure all states operate in such a fashion as WI, but I believe having multiple county-specific reviewers helps to improve data quality and free up certain reviewers to tweak and tighten regional filters.

    The points you raise and those raised in the comments are valid and reasonable. Misplots of locations is my personal pet peeve.I completely support you having the ability to flag media. AFterall YOU ARE NUMBER 7! I suspect the email sent to Ian will you resolve this. If not, if you personally know one of Cali reviewers I'm willing to bet they could recommend giving you clearance for flagging media.

    As far as media flagging goes, regional reviewers have been encouraged to take more initiative to periodically look through species photos submitted for their regions. On a global level, I know of at least one U.S. reviewer who has been dedicated to tackling media review of various bird dichotomies/trichotomies often mis-identified (eg. Sharp-shinned/Cooper's Hawk etc.) As you can imagine, it can be a slow, laborious process. Hence why eBird also granted media flagging privileges to certain eBirders that met some set threshold of eBird activity.

    There's also a Facebook for eBird discussion where you raise your concerns if you are so inclined: https://www.facebook.com/groups/288737854555183/


    ReplyDelete
    Replies
    1. Yes, the more reviewers the better...as long as they are good birders and/or reasonable people (this is usually the case, but sadly not always...but that is another topic).

      I think this is not the first email to Ian this post has elicited...currently I am still out in the cold, so to speak...maybe Ian is still busy with better things to do (i.e. crushing birds in Thailand). Not holding my breath though.

      This topic has appeared in that FB group a number of times, but I thought I would give it more thorough treatment here.

      Delete
    2. Hoping you receive some warmth soon...Yep sometimes the skills of certain birders can be over-inflated landing these people in roles that raise eyebrows. Birding has yet to lose the good ole boys club aspect. :)

      Delete