[Realtime-feed-users] Id stability in the GeoJSON feed

[Peter Schmidt]

Hi Adam

Sorry for intervening on this thread, but speaking from personal 
experience, one complication of building a catalog based on feeds is 
that from time to time feeds will be issued for spurious events, hence a 
minimum requirement is a mechanism to track these down and remove them 
from the catalogue.

Not using GeoJSON feeds from USGS but rather tapping into their 
implementation of the FDSN web service I'm not sure how big of a problem 
this is for GeoJSON, but at least for the FDSN-WS we have seen a few of 
these per year although they seem to have become less frequent the last 
year or so (if I remember correctly most of the spurious events have 
been issued by the Pacific Tsunami Warning Seismic System, pt).


regP


On 2016-04-13 20:52, Adam Kalachman wrote:
> Hi Jeremy,
>
> Can you clarify why building a catalog from the feeds is not 
> recommended? Presumably having data pushed to us means we can get it 
> slightly faster, but beyond that, are there other concerns with using 
> the feed in this manner?
>
> Thanks,
> Adam
>
> On Mon, Apr 11, 2016 at 11:00 AM, Fee, Jeremy <jmfee at usgs.gov 
> <mailto:jmfee at usgs.gov>> wrote:
>
>     Hello,
>
>     The ids in the feeds are "eventually consistent" because they run
>     on distributed systems that may have different information at
>     different times.  Generally, we recommend using the feature "id"
>     attribute and when processing a previously unseen id checking the
>     "ids" list to see if the system has already processed the event
>     under a different id.
>
>
>     As an alternative to building a catalog from the feeds (not
>     recommended), and for push access to our information, you can use
>     our java data distribution software to build/update the catalog;
>     and trigger external processing as changes are made.
>
>         https://github.com/usgs/pdl
>
>
>     An example configuration of the "indexer" is available on github:
>
>         https://github.com/jmfee-usgs/pdl-client-examples/tree/indexer
>
>
>
>     Thanks,
>
>     Jeremy
>
>
>     On Fri, Apr 8, 2016 at 5:12 PM, Adam Kalachman
>     <akalachman at google.com <mailto:akalachman at google.com>> wrote:
>
>         Hi,
>
>         I have a couple of questions regarding id stability in the
>         GeoJSON earthquakes feed. The 'ids' field appears to specify
>         the id of each contributing network, but is there any
>         consistency to the ordering of this field? Best I can tell,
>         it's neither chronological nor alphabetical.
>
>         Assuming the order is arbitrary, is there some way that a
>         stable record id, independent of any individual sensor
>         networks, could be added to the GeoJSON schema?
>
>         For example, say the feed reports an earthquake with ids
>         ak000, us001, and nc002, and preferred id us001. Then a client
>         of the feed might've seen ak000 and nc002 previously, and it's
>         the responsibility of the client to track down those
>         earthquake summaries and replace or merge them with the new
>         data. If, however, all three were assigned a single stable id,
>         say, eq0003, then each updated summary could more easily
>         replace its previous versions.
>
>         Thanks,
>         Adam
>
>         -- 
>
>         Adam Kalachman | Software Engineer |akalachman at google.com
>         <mailto:akalachman at google.com> |256-509-5336 <tel:256-509-5336>
>
>         _______________________________________________
>         Realtime-feed-users mailing list
>         Realtime-feed-users at geohazards.usgs.gov
>         <mailto:Realtime-feed-users at geohazards.usgs.gov>
>         https://geohazards.usgs.gov/mailman/listinfo/realtime-feed-users
>
>
>
>
>
> -- 
>
> Adam Kalachman | Software Engineer |akalachman at google.com 
> <mailto:akalachman at google.com> | 256-509-5336
>
>
> _______________________________________________
> Realtime-feed-users mailing list
> Realtime-feed-users at geohazards.usgs.gov
> https://geohazards.usgs.gov/mailman/listinfo/realtime-feed-users


-- 
********************************************************************************
Peter Schmidt                                      Tel: +46-18-4717104
Swedish National Seismological Network (SNSN)   Mobile: +46-73-3190975
Dept. of Earth Sciences:geophysics              e-mail: peter.schmidt at geo.uu.se
Uppsala University
Villavagen 16
SE-75236 Uppsala
********************************************************************************

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://geohazards.usgs.gov/pipermail/realtime-feed-users/attachments/20160413/b86e0aeb/attachment-0001.html>