[Realtime-feed-users] Id stability in the GeoJSON feed

Tue Apr 26 17:27:07 UTC 2016

PDL and ENS ( https://sslearthquake.usgs.gov/ens/ ) are the only ways to
receive delete notifications at this time.

Thanks,

Jeremy

On Tue, Apr 26, 2016 at 11:18 AM, Adam Kalachman <akalachman at google.com>
wrote:

> I see. What options are available for learning about these deletions? It
> sounds like they're not in the GeoJSON, but are there ways we can find out
> other than integrating with PDL?
>
> On Tue, Apr 26, 2016 at 9:52 AM, Fee, Jeremy <jmfee at usgs.gov> wrote:
>
>> Some of the larger events that have been deleted are described on this
>> page:
>>
>> http://earthquake.usgs.gov/earthquakes/errata.php
>>
>>
>>
>> Thanks,
>>
>> Jeremy
>>
>>
>> On Tue, Apr 26, 2016 at 10:43 AM, Adam Kalachman <akalachman at google.com>
>> wrote:
>>
>>> Interesting. Of these non-seismic events, what is the typical range of
>>> magnitudes?
>>>
>>> Thanks,
>>> Adam
>>>
>>> On Tue, Apr 26, 2016 at 9:39 AM, Fee, Jeremy <jmfee at usgs.gov> wrote:
>>>
>>>> Hello,
>>>>
>>>> The most commonly deleted events are automatic location/magnitudes that
>>>> turn out not to be a seismic event.  In general, we recommend networks
>>>> leave valid events in their catalogs and accurately represent the
>>>> quality/uncertainty of the solution.
>>>>
>>>> Event IDs should not be reused once issued.  An event may be undeleted
>>>> after further review.
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Jeremy
>>>>
>>>>
>>>> On Fri, Apr 22, 2016 at 3:35 PM, Adam Kalachman <akalachman at google.com>
>>>> wrote:
>>>>
>>>>> Hi Jeremy,
>>>>>
>>>>> Thanks, that's helpful. We have a solution for dealing with updates to
>>>>> old events, but I want to know more about deleted events.
>>>>>
>>>>> Under what circumstances are events deleted? Are they eventually
>>>>> replaced by other events with the same id?
>>>>>
>>>>> Thanks,
>>>>> Adam
>>>>>
>>>>>
>>>>> On Fri, Apr 15, 2016 at 2:30 PM, Fee, Jeremy <jmfee at usgs.gov> wrote:
>>>>>
>>>>>> Hi Adam,
>>>>>>
>>>>>> The short answer is there are a couple issues with using feeds to
>>>>>> build a catalog:
>>>>>>
>>>>>> 1) Deleted events are not included in feeds.
>>>>>>
>>>>>> 2) Events are only included during the feed lifetime.  For example,
>>>>>> if you monitor a 1 day feed you won't see any updates to events more than
>>>>>> one day old.
>>>>>>
>>>>>>
>>>>>> The longer answer is it depends on what you're trying to do with the
>>>>>> data.  Happy to discuss further.
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Jeremy
>>>>>>
>>>>>>
>>>>>> On Wed, Apr 13, 2016 at 3:10 PM, Peter Schmidt <
>>>>>> peter.schmidt at geo.uu.se> wrote:
>>>>>>
>>>>>>> Hi Adam
>>>>>>>
>>>>>>> Sorry for intervening on this thread, but speaking from personal
>>>>>>> experience, one complication of building a catalog based on feeds is that
>>>>>>> from time to time feeds will be issued for spurious events, hence a minimum
>>>>>>> requirement is a mechanism to track these down and remove them from the
>>>>>>> catalogue.
>>>>>>>
>>>>>>> Not using GeoJSON feeds from USGS but rather tapping into their
>>>>>>> implementation of the FDSN web service I'm not sure how big of a problem
>>>>>>> this is for GeoJSON, but at least for the FDSN-WS we have seen a few of
>>>>>>> these per year although they seem to have become less frequent the last
>>>>>>> year or so (if I remember correctly most of the spurious events have been
>>>>>>> issued by the Pacific Tsunami Warning Seismic System, pt).
>>>>>>>
>>>>>>>
>>>>>>> regP
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 2016-04-13 20:52, Adam Kalachman wrote:
>>>>>>>
>>>>>>> Hi Jeremy,
>>>>>>>
>>>>>>> Can you clarify why building a catalog from the feeds is not
>>>>>>> recommended? Presumably having data pushed to us means we can get it
>>>>>>> slightly faster, but beyond that, are there other concerns with using the
>>>>>>> feed in this manner?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Adam
>>>>>>>
>>>>>>> On Mon, Apr 11, 2016 at 11:00 AM, Fee, Jeremy <jmfee at usgs.gov>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> The ids in the feeds are "eventually consistent" because they run
>>>>>>>> on distributed systems that may have different information at different
>>>>>>>> times.  Generally, we recommend using the feature "id" attribute and when
>>>>>>>> processing a previously unseen id checking the "ids" list to see if the
>>>>>>>> system has already processed the event under a different id.
>>>>>>>>
>>>>>>>>
>>>>>>>> As an alternative to building a catalog from the feeds (not
>>>>>>>> recommended), and for push access to our information, you can use our java
>>>>>>>> data distribution software to build/update the catalog; and trigger
>>>>>>>> external processing as changes are made.
>>>>>>>>
>>>>>>>> https://github.com/usgs/pdl
>>>>>>>>
>>>>>>>>
>>>>>>>> An example configuration of the "indexer" is available on github:
>>>>>>>>
>>>>>>>> https://github.com/jmfee-usgs/pdl-client-examples/tree/indexer
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Jeremy
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Apr 8, 2016 at 5:12 PM, Adam Kalachman <
>>>>>>>> <akalachman at google.com>akalachman at google.com> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I have a couple of questions regarding id stability in the GeoJSON
>>>>>>>>> earthquakes feed. The 'ids' field appears to specify the id of each
>>>>>>>>> contributing network, but is there any consistency to the ordering of this
>>>>>>>>> field? Best I can tell, it's neither chronological nor alphabetical.
>>>>>>>>>
>>>>>>>>> Assuming the order is arbitrary, is there some way that a stable
>>>>>>>>> record id, independent of any individual sensor networks, could be added to
>>>>>>>>> the GeoJSON schema?
>>>>>>>>>
>>>>>>>>> For example, say the feed reports an earthquake with ids ak000,
>>>>>>>>> us001, and nc002, and preferred id us001. Then a client of the feed
>>>>>>>>> might've seen ak000 and nc002 previously, and it's the responsibility of
>>>>>>>>> the client to track down those earthquake summaries and replace or merge
>>>>>>>>> them with the new data. If, however, all three were assigned a single
>>>>>>>>> stable id, say, eq0003, then each updated summary could more easily replace
>>>>>>>>> its previous versions.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Adam
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> Adam Kalachman | Software Engineer |  <akalachman at google.com>
>>>>>>>>> akalachman at google.com | 256-509-5336
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Realtime-feed-users mailing list
>>>>>>>>> Realtime-feed-users at geohazards.usgs.gov
>>>>>>>>> https://geohazards.usgs.gov/mailman/listinfo/realtime-feed-users
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Adam Kalachman | Software Engineer |  <akalachman at google.com>
>>>>>>> akalachman at google.com | 256-509-5336
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Realtime-feed-users mailing listRealtime-feed-users at geohazards.usgs.govhttps://geohazards.usgs.gov/mailman/listinfo/realtime-feed-users
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> ********************************************************************************
>>>>>>> Peter Schmidt                                      Tel: +46-18-4717104
>>>>>>> Swedish National Seismological Network (SNSN)   Mobile: +46-73-3190975
>>>>>>> Dept. of Earth Sciences:geophysics              e-mail: peter.schmidt at geo.uu.se
>>>>>>> Uppsala University
>>>>>>> Villavagen 16
>>>>>>> SE-75236 Uppsala
>>>>>>> ********************************************************************************
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Realtime-feed-users mailing list
>>>>>>> Realtime-feed-users at geohazards.usgs.gov
>>>>>>> https://geohazards.usgs.gov/mailman/listinfo/realtime-feed-users
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Adam Kalachman | Software Engineer | akalachman at google.com |
>>>>> 256-509-5336
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> Adam Kalachman | Software Engineer | akalachman at google.com |
>>> 256-509-5336
>>>
>>
>>
>
>
> --
>
> Adam Kalachman | Software Engineer | akalachman at google.com | 256-509-5336
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://geohazards.usgs.gov/pipermail/realtime-feed-users/attachments/20160426/9da604bb/attachment-0001.html>