[Realtime-feed-users] Id stability in the GeoJSON feed

Tue Apr 26 16:39:19 UTC 2016

Hello,

The most commonly deleted events are automatic location/magnitudes that
turn out not to be a seismic event.  In general, we recommend networks
leave valid events in their catalogs and accurately represent the
quality/uncertainty of the solution.

Event IDs should not be reused once issued.  An event may be undeleted
after further review.

Thanks,

Jeremy

On Fri, Apr 22, 2016 at 3:35 PM, Adam Kalachman <akalachman at google.com>
wrote:

> Hi Jeremy,
>
> Thanks, that's helpful. We have a solution for dealing with updates to old
> events, but I want to know more about deleted events.
>
> Under what circumstances are events deleted? Are they eventually replaced
> by other events with the same id?
>
> Thanks,
> Adam
>
>
> On Fri, Apr 15, 2016 at 2:30 PM, Fee, Jeremy <jmfee at usgs.gov> wrote:
>
>> Hi Adam,
>>
>> The short answer is there are a couple issues with using feeds to build a
>> catalog:
>>
>> 1) Deleted events are not included in feeds.
>>
>> 2) Events are only included during the feed lifetime.  For example, if
>> you monitor a 1 day feed you won't see any updates to events more than one
>> day old.
>>
>>
>> The longer answer is it depends on what you're trying to do with the
>> data.  Happy to discuss further.
>>
>>
>> Thanks,
>>
>> Jeremy
>>
>>
>> On Wed, Apr 13, 2016 at 3:10 PM, Peter Schmidt <peter.schmidt at geo.uu.se>
>> wrote:
>>
>>> Hi Adam
>>>
>>> Sorry for intervening on this thread, but speaking from personal
>>> experience, one complication of building a catalog based on feeds is that
>>> from time to time feeds will be issued for spurious events, hence a minimum
>>> requirement is a mechanism to track these down and remove them from the
>>> catalogue.
>>>
>>> Not using GeoJSON feeds from USGS but rather tapping into their
>>> implementation of the FDSN web service I'm not sure how big of a problem
>>> this is for GeoJSON, but at least for the FDSN-WS we have seen a few of
>>> these per year although they seem to have become less frequent the last
>>> year or so (if I remember correctly most of the spurious events have been
>>> issued by the Pacific Tsunami Warning Seismic System, pt).
>>>
>>>
>>> regP
>>>
>>>
>>>
>>> On 2016-04-13 20:52, Adam Kalachman wrote:
>>>
>>> Hi Jeremy,
>>>
>>> Can you clarify why building a catalog from the feeds is not
>>> recommended? Presumably having data pushed to us means we can get it
>>> slightly faster, but beyond that, are there other concerns with using the
>>> feed in this manner?
>>>
>>> Thanks,
>>> Adam
>>>
>>> On Mon, Apr 11, 2016 at 11:00 AM, Fee, Jeremy <jmfee at usgs.gov> wrote:
>>>
>>>> Hello,
>>>>
>>>> The ids in the feeds are "eventually consistent" because they run on
>>>> distributed systems that may have different information at different
>>>> times.  Generally, we recommend using the feature "id" attribute and when
>>>> processing a previously unseen id checking the "ids" list to see if the
>>>> system has already processed the event under a different id.
>>>>
>>>>
>>>> As an alternative to building a catalog from the feeds (not
>>>> recommended), and for push access to our information, you can use our java
>>>> data distribution software to build/update the catalog; and trigger
>>>> external processing as changes are made.
>>>>
>>>> https://github.com/usgs/pdl
>>>>
>>>>
>>>> An example configuration of the "indexer" is available on github:
>>>>
>>>> https://github.com/jmfee-usgs/pdl-client-examples/tree/indexer
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Jeremy
>>>>
>>>>
>>>> On Fri, Apr 8, 2016 at 5:12 PM, Adam Kalachman <
>>>> <akalachman at google.com>akalachman at google.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I have a couple of questions regarding id stability in the GeoJSON
>>>>> earthquakes feed. The 'ids' field appears to specify the id of each
>>>>> contributing network, but is there any consistency to the ordering of this
>>>>> field? Best I can tell, it's neither chronological nor alphabetical.
>>>>>
>>>>> Assuming the order is arbitrary, is there some way that a stable
>>>>> record id, independent of any individual sensor networks, could be added to
>>>>> the GeoJSON schema?
>>>>>
>>>>> For example, say the feed reports an earthquake with ids ak000, us001,
>>>>> and nc002, and preferred id us001. Then a client of the feed might've seen
>>>>> ak000 and nc002 previously, and it's the responsibility of the client to
>>>>> track down those earthquake summaries and replace or merge them with the
>>>>> new data. If, however, all three were assigned a single stable id, say,
>>>>> eq0003, then each updated summary could more easily replace its previous
>>>>> versions.
>>>>>
>>>>> Thanks,
>>>>> Adam
>>>>>
>>>>> --
>>>>>
>>>>> Adam Kalachman | Software Engineer |  <akalachman at google.com>
>>>>> akalachman at google.com | 256-509-5336
>>>>>
>>>>> _______________________________________________
>>>>> Realtime-feed-users mailing list
>>>>> Realtime-feed-users at geohazards.usgs.gov
>>>>> https://geohazards.usgs.gov/mailman/listinfo/realtime-feed-users
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> Adam Kalachman | Software Engineer |  <akalachman at google.com>
>>> akalachman at google.com | 256-509-5336
>>>
>>>
>>> _______________________________________________
>>> Realtime-feed-users mailing listRealtime-feed-users at geohazards.usgs.govhttps://geohazards.usgs.gov/mailman/listinfo/realtime-feed-users
>>>
>>>
>>>
>>> --
>>> ********************************************************************************
>>> Peter Schmidt                                      Tel: +46-18-4717104
>>> Swedish National Seismological Network (SNSN)   Mobile: +46-73-3190975
>>> Dept. of Earth Sciences:geophysics              e-mail: peter.schmidt at geo.uu.se
>>> Uppsala University
>>> Villavagen 16
>>> SE-75236 Uppsala
>>> ********************************************************************************
>>>
>>>
>>> _______________________________________________
>>> Realtime-feed-users mailing list
>>> Realtime-feed-users at geohazards.usgs.gov
>>> https://geohazards.usgs.gov/mailman/listinfo/realtime-feed-users
>>>
>>>
>>
>
>
> --
>
> Adam Kalachman | Software Engineer | akalachman at google.com | 256-509-5336
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://geohazards.usgs.gov/pipermail/realtime-feed-users/attachments/20160426/04259dbd/attachment.html>