[Realtime-feed-users] Id stability in the GeoJSON feed

Tue Apr 26 16:52:01 UTC 2016

Some of the larger events that have been deleted are described on this page:

http://earthquake.usgs.gov/earthquakes/errata.php

Thanks,

Jeremy

On Tue, Apr 26, 2016 at 10:43 AM, Adam Kalachman <akalachman at google.com>
wrote:

> Interesting. Of these non-seismic events, what is the typical range of
> magnitudes?
>
> Thanks,
> Adam
>
> On Tue, Apr 26, 2016 at 9:39 AM, Fee, Jeremy <jmfee at usgs.gov> wrote:
>
>> Hello,
>>
>> The most commonly deleted events are automatic location/magnitudes that
>> turn out not to be a seismic event.  In general, we recommend networks
>> leave valid events in their catalogs and accurately represent the
>> quality/uncertainty of the solution.
>>
>> Event IDs should not be reused once issued.  An event may be undeleted
>> after further review.
>>
>>
>> Thanks,
>>
>> Jeremy
>>
>>
>> On Fri, Apr 22, 2016 at 3:35 PM, Adam Kalachman <akalachman at google.com>
>> wrote:
>>
>>> Hi Jeremy,
>>>
>>> Thanks, that's helpful. We have a solution for dealing with updates to
>>> old events, but I want to know more about deleted events.
>>>
>>> Under what circumstances are events deleted? Are they eventually
>>> replaced by other events with the same id?
>>>
>>> Thanks,
>>> Adam
>>>
>>>
>>> On Fri, Apr 15, 2016 at 2:30 PM, Fee, Jeremy <jmfee at usgs.gov> wrote:
>>>
>>>> Hi Adam,
>>>>
>>>> The short answer is there are a couple issues with using feeds to build
>>>> a catalog:
>>>>
>>>> 1) Deleted events are not included in feeds.
>>>>
>>>> 2) Events are only included during the feed lifetime.  For example, if
>>>> you monitor a 1 day feed you won't see any updates to events more than one
>>>> day old.
>>>>
>>>>
>>>> The longer answer is it depends on what you're trying to do with the
>>>> data.  Happy to discuss further.
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Jeremy
>>>>
>>>>
>>>> On Wed, Apr 13, 2016 at 3:10 PM, Peter Schmidt <peter.schmidt at geo.uu.se
>>>> > wrote:
>>>>
>>>>> Hi Adam
>>>>>
>>>>> Sorry for intervening on this thread, but speaking from personal
>>>>> experience, one complication of building a catalog based on feeds is that
>>>>> from time to time feeds will be issued for spurious events, hence a minimum
>>>>> requirement is a mechanism to track these down and remove them from the
>>>>> catalogue.
>>>>>
>>>>> Not using GeoJSON feeds from USGS but rather tapping into their
>>>>> implementation of the FDSN web service I'm not sure how big of a problem
>>>>> this is for GeoJSON, but at least for the FDSN-WS we have seen a few of
>>>>> these per year although they seem to have become less frequent the last
>>>>> year or so (if I remember correctly most of the spurious events have been
>>>>> issued by the Pacific Tsunami Warning Seismic System, pt).
>>>>>
>>>>>
>>>>> regP
>>>>>
>>>>>
>>>>>
>>>>> On 2016-04-13 20:52, Adam Kalachman wrote:
>>>>>
>>>>> Hi Jeremy,
>>>>>
>>>>> Can you clarify why building a catalog from the feeds is not
>>>>> recommended? Presumably having data pushed to us means we can get it
>>>>> slightly faster, but beyond that, are there other concerns with using the
>>>>> feed in this manner?
>>>>>
>>>>> Thanks,
>>>>> Adam
>>>>>
>>>>> On Mon, Apr 11, 2016 at 11:00 AM, Fee, Jeremy <jmfee at usgs.gov> wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> The ids in the feeds are "eventually consistent" because they run on
>>>>>> distributed systems that may have different information at different
>>>>>> times.  Generally, we recommend using the feature "id" attribute and when
>>>>>> processing a previously unseen id checking the "ids" list to see if the
>>>>>> system has already processed the event under a different id.
>>>>>>
>>>>>>
>>>>>> As an alternative to building a catalog from the feeds (not
>>>>>> recommended), and for push access to our information, you can use our java
>>>>>> data distribution software to build/update the catalog; and trigger
>>>>>> external processing as changes are made.
>>>>>>
>>>>>> https://github.com/usgs/pdl
>>>>>>
>>>>>>
>>>>>> An example configuration of the "indexer" is available on github:
>>>>>>
>>>>>> https://github.com/jmfee-usgs/pdl-client-examples/tree/indexer
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Jeremy
>>>>>>
>>>>>>
>>>>>> On Fri, Apr 8, 2016 at 5:12 PM, Adam Kalachman <
>>>>>> <akalachman at google.com>akalachman at google.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I have a couple of questions regarding id stability in the GeoJSON
>>>>>>> earthquakes feed. The 'ids' field appears to specify the id of each
>>>>>>> contributing network, but is there any consistency to the ordering of this
>>>>>>> field? Best I can tell, it's neither chronological nor alphabetical.
>>>>>>>
>>>>>>> Assuming the order is arbitrary, is there some way that a stable
>>>>>>> record id, independent of any individual sensor networks, could be added to
>>>>>>> the GeoJSON schema?
>>>>>>>
>>>>>>> For example, say the feed reports an earthquake with ids ak000,
>>>>>>> us001, and nc002, and preferred id us001. Then a client of the feed
>>>>>>> might've seen ak000 and nc002 previously, and it's the responsibility of
>>>>>>> the client to track down those earthquake summaries and replace or merge
>>>>>>> them with the new data. If, however, all three were assigned a single
>>>>>>> stable id, say, eq0003, then each updated summary could more easily replace
>>>>>>> its previous versions.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Adam
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Adam Kalachman | Software Engineer |  <akalachman at google.com>
>>>>>>> akalachman at google.com | 256-509-5336
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Realtime-feed-users mailing list
>>>>>>> Realtime-feed-users at geohazards.usgs.gov
>>>>>>> https://geohazards.usgs.gov/mailman/listinfo/realtime-feed-users
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Adam Kalachman | Software Engineer |  <akalachman at google.com>
>>>>> akalachman at google.com | 256-509-5336
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Realtime-feed-users mailing listRealtime-feed-users at geohazards.usgs.govhttps://geohazards.usgs.gov/mailman/listinfo/realtime-feed-users
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> ********************************************************************************
>>>>> Peter Schmidt                                      Tel: +46-18-4717104
>>>>> Swedish National Seismological Network (SNSN)   Mobile: +46-73-3190975
>>>>> Dept. of Earth Sciences:geophysics              e-mail: peter.schmidt at geo.uu.se
>>>>> Uppsala University
>>>>> Villavagen 16
>>>>> SE-75236 Uppsala
>>>>> ********************************************************************************
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Realtime-feed-users mailing list
>>>>> Realtime-feed-users at geohazards.usgs.gov
>>>>> https://geohazards.usgs.gov/mailman/listinfo/realtime-feed-users
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> Adam Kalachman | Software Engineer | akalachman at google.com |
>>> 256-509-5336
>>>
>>
>>
>
>
> --
>
> Adam Kalachman | Software Engineer | akalachman at google.com | 256-509-5336
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://geohazards.usgs.gov/pipermail/realtime-feed-users/attachments/20160426/be7a1585/attachment-0001.html>