[Realtime-feed-users] Id stability in the GeoJSON feed

[Fee, Jeremy]

Hi Adam,

The short answer is there are a couple issues with using feeds to build a
catalog:

1) Deleted events are not included in feeds.

2) Events are only included during the feed lifetime.  For example, if you
monitor a 1 day feed you won't see any updates to events more than one day
old.


The longer answer is it depends on what you're trying to do with the data.
Happy to discuss further.


Thanks,

Jeremy


On Wed, Apr 13, 2016 at 3:10 PM, Peter Schmidt <peter.schmidt at geo.uu.se>
wrote:

> Hi Adam
>
> Sorry for intervening on this thread, but speaking from personal
> experience, one complication of building a catalog based on feeds is that
> from time to time feeds will be issued for spurious events, hence a minimum
> requirement is a mechanism to track these down and remove them from the
> catalogue.
>
> Not using GeoJSON feeds from USGS but rather tapping into their
> implementation of the FDSN web service I'm not sure how big of a problem
> this is for GeoJSON, but at least for the FDSN-WS we have seen a few of
> these per year although they seem to have become less frequent the last
> year or so (if I remember correctly most of the spurious events have been
> issued by the Pacific Tsunami Warning Seismic System, pt).
>
>
> regP
>
>
>
> On 2016-04-13 20:52, Adam Kalachman wrote:
>
> Hi Jeremy,
>
> Can you clarify why building a catalog from the feeds is not recommended?
> Presumably having data pushed to us means we can get it slightly faster,
> but beyond that, are there other concerns with using the feed in this
> manner?
>
> Thanks,
> Adam
>
> On Mon, Apr 11, 2016 at 11:00 AM, Fee, Jeremy <jmfee at usgs.gov> wrote:
>
>> Hello,
>>
>> The ids in the feeds are "eventually consistent" because they run on
>> distributed systems that may have different information at different
>> times.  Generally, we recommend using the feature "id" attribute and when
>> processing a previously unseen id checking the "ids" list to see if the
>> system has already processed the event under a different id.
>>
>>
>> As an alternative to building a catalog from the feeds (not recommended),
>> and for push access to our information, you can use our java data
>> distribution software to build/update the catalog; and trigger external
>> processing as changes are made.
>>
>> https://github.com/usgs/pdl
>>
>>
>> An example configuration of the "indexer" is available on github:
>>
>> https://github.com/jmfee-usgs/pdl-client-examples/tree/indexer
>>
>>
>>
>> Thanks,
>>
>> Jeremy
>>
>>
>> On Fri, Apr 8, 2016 at 5:12 PM, Adam Kalachman < <akalachman at google.com>
>> akalachman at google.com> wrote:
>>
>>> Hi,
>>>
>>> I have a couple of questions regarding id stability in the GeoJSON
>>> earthquakes feed. The 'ids' field appears to specify the id of each
>>> contributing network, but is there any consistency to the ordering of this
>>> field? Best I can tell, it's neither chronological nor alphabetical.
>>>
>>> Assuming the order is arbitrary, is there some way that a stable record
>>> id, independent of any individual sensor networks, could be added to the
>>> GeoJSON schema?
>>>
>>> For example, say the feed reports an earthquake with ids ak000, us001,
>>> and nc002, and preferred id us001. Then a client of the feed might've seen
>>> ak000 and nc002 previously, and it's the responsibility of the client to
>>> track down those earthquake summaries and replace or merge them with the
>>> new data. If, however, all three were assigned a single stable id, say,
>>> eq0003, then each updated summary could more easily replace its previous
>>> versions.
>>>
>>> Thanks,
>>> Adam
>>>
>>> --
>>>
>>> Adam Kalachman | Software Engineer |  <akalachman at google.com>
>>> akalachman at google.com | 256-509-5336
>>>
>>> _______________________________________________
>>> Realtime-feed-users mailing list
>>> Realtime-feed-users at geohazards.usgs.gov
>>> https://geohazards.usgs.gov/mailman/listinfo/realtime-feed-users
>>>
>>>
>>
>
>
> --
>
> Adam Kalachman | Software Engineer |  <akalachman at google.com>
> akalachman at google.com | 256-509-5336
>
>
> _______________________________________________
> Realtime-feed-users mailing listRealtime-feed-users at geohazards.usgs.govhttps://geohazards.usgs.gov/mailman/listinfo/realtime-feed-users
>
>
>
> --
> ********************************************************************************
> Peter Schmidt                                      Tel: +46-18-4717104
> Swedish National Seismological Network (SNSN)   Mobile: +46-73-3190975
> Dept. of Earth Sciences:geophysics              e-mail: peter.schmidt at geo.uu.se
> Uppsala University
> Villavagen 16
> SE-75236 Uppsala
> ********************************************************************************
>
>
> _______________________________________________
> Realtime-feed-users mailing list
> Realtime-feed-users at geohazards.usgs.gov
> https://geohazards.usgs.gov/mailman/listinfo/realtime-feed-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://geohazards.usgs.gov/pipermail/realtime-feed-users/attachments/20160415/0760bc25/attachment.html>