[Realtime-feed-users] Id stability in the GeoJSON feed

Tue Apr 26 17:18:18 UTC 2016

I see. What options are available for learning about these deletions? It
sounds like they're not in the GeoJSON, but are there ways we can find out
other than integrating with PDL?

On Tue, Apr 26, 2016 at 9:52 AM, Fee, Jeremy <jmfee at usgs.gov> wrote:

> Some of the larger events that have been deleted are described on this
> page:
>
> http://earthquake.usgs.gov/earthquakes/errata.php
>
>
>
> Thanks,
>
> Jeremy
>
>
> On Tue, Apr 26, 2016 at 10:43 AM, Adam Kalachman <akalachman at google.com>
> wrote:
>
>> Interesting. Of these non-seismic events, what is the typical range of
>> magnitudes?
>>
>> Thanks,
>> Adam
>>
>> On Tue, Apr 26, 2016 at 9:39 AM, Fee, Jeremy <jmfee at usgs.gov> wrote:
>>
>>> Hello,
>>>
>>> The most commonly deleted events are automatic location/magnitudes that
>>> turn out not to be a seismic event.  In general, we recommend networks
>>> leave valid events in their catalogs and accurately represent the
>>> quality/uncertainty of the solution.
>>>
>>> Event IDs should not be reused once issued.  An event may be undeleted
>>> after further review.
>>>
>>>
>>> Thanks,
>>>
>>> Jeremy
>>>
>>>
>>> On Fri, Apr 22, 2016 at 3:35 PM, Adam Kalachman <akalachman at google.com>
>>> wrote:
>>>
>>>> Hi Jeremy,
>>>>
>>>> Thanks, that's helpful. We have a solution for dealing with updates to
>>>> old events, but I want to know more about deleted events.
>>>>
>>>> Under what circumstances are events deleted? Are they eventually
>>>> replaced by other events with the same id?
>>>>
>>>> Thanks,
>>>> Adam
>>>>
>>>>
>>>> On Fri, Apr 15, 2016 at 2:30 PM, Fee, Jeremy <jmfee at usgs.gov> wrote:
>>>>
>>>>> Hi Adam,
>>>>>
>>>>> The short answer is there are a couple issues with using feeds to
>>>>> build a catalog:
>>>>>
>>>>> 1) Deleted events are not included in feeds.
>>>>>
>>>>> 2) Events are only included during the feed lifetime.  For example, if
>>>>> you monitor a 1 day feed you won't see any updates to events more than one
>>>>> day old.
>>>>>
>>>>>
>>>>> The longer answer is it depends on what you're trying to do with the
>>>>> data.  Happy to discuss further.
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Jeremy
>>>>>
>>>>>
>>>>> On Wed, Apr 13, 2016 at 3:10 PM, Peter Schmidt <
>>>>> peter.schmidt at geo.uu.se> wrote:
>>>>>
>>>>>> Hi Adam
>>>>>>
>>>>>> Sorry for intervening on this thread, but speaking from personal
>>>>>> experience, one complication of building a catalog based on feeds is that
>>>>>> from time to time feeds will be issued for spurious events, hence a minimum
>>>>>> requirement is a mechanism to track these down and remove them from the
>>>>>> catalogue.
>>>>>>
>>>>>> Not using GeoJSON feeds from USGS but rather tapping into their
>>>>>> implementation of the FDSN web service I'm not sure how big of a problem
>>>>>> this is for GeoJSON, but at least for the FDSN-WS we have seen a few of
>>>>>> these per year although they seem to have become less frequent the last
>>>>>> year or so (if I remember correctly most of the spurious events have been
>>>>>> issued by the Pacific Tsunami Warning Seismic System, pt).
>>>>>>
>>>>>>
>>>>>> regP
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 2016-04-13 20:52, Adam Kalachman wrote:
>>>>>>
>>>>>> Hi Jeremy,
>>>>>>
>>>>>> Can you clarify why building a catalog from the feeds is not
>>>>>> recommended? Presumably having data pushed to us means we can get it
>>>>>> slightly faster, but beyond that, are there other concerns with using the
>>>>>> feed in this manner?
>>>>>>
>>>>>> Thanks,
>>>>>> Adam
>>>>>>
>>>>>> On Mon, Apr 11, 2016 at 11:00 AM, Fee, Jeremy <jmfee at usgs.gov> wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> The ids in the feeds are "eventually consistent" because they run on
>>>>>>> distributed systems that may have different information at different
>>>>>>> times.  Generally, we recommend using the feature "id" attribute and when
>>>>>>> processing a previously unseen id checking the "ids" list to see if the
>>>>>>> system has already processed the event under a different id.
>>>>>>>
>>>>>>>
>>>>>>> As an alternative to building a catalog from the feeds (not
>>>>>>> recommended), and for push access to our information, you can use our java
>>>>>>> data distribution software to build/update the catalog; and trigger
>>>>>>> external processing as changes are made.
>>>>>>>
>>>>>>> https://github.com/usgs/pdl
>>>>>>>
>>>>>>>
>>>>>>> An example configuration of the "indexer" is available on github:
>>>>>>>
>>>>>>> https://github.com/jmfee-usgs/pdl-client-examples/tree/indexer
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Jeremy
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Apr 8, 2016 at 5:12 PM, Adam Kalachman <
>>>>>>> <akalachman at google.com>akalachman at google.com> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I have a couple of questions regarding id stability in the GeoJSON
>>>>>>>> earthquakes feed. The 'ids' field appears to specify the id of each
>>>>>>>> contributing network, but is there any consistency to the ordering of this
>>>>>>>> field? Best I can tell, it's neither chronological nor alphabetical.
>>>>>>>>
>>>>>>>> Assuming the order is arbitrary, is there some way that a stable
>>>>>>>> record id, independent of any individual sensor networks, could be added to
>>>>>>>> the GeoJSON schema?
>>>>>>>>
>>>>>>>> For example, say the feed reports an earthquake with ids ak000,
>>>>>>>> us001, and nc002, and preferred id us001. Then a client of the feed
>>>>>>>> might've seen ak000 and nc002 previously, and it's the responsibility of
>>>>>>>> the client to track down those earthquake summaries and replace or merge
>>>>>>>> them with the new data. If, however, all three were assigned a single
>>>>>>>> stable id, say, eq0003, then each updated summary could more easily replace
>>>>>>>> its previous versions.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Adam
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> Adam Kalachman | Software Engineer |  <akalachman at google.com>
>>>>>>>> akalachman at google.com | 256-509-5336
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Realtime-feed-users mailing list
>>>>>>>> Realtime-feed-users at geohazards.usgs.gov
>>>>>>>> https://geohazards.usgs.gov/mailman/listinfo/realtime-feed-users
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Adam Kalachman | Software Engineer |  <akalachman at google.com>
>>>>>> akalachman at google.com | 256-509-5336
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Realtime-feed-users mailing listRealtime-feed-users at geohazards.usgs.govhttps://geohazards.usgs.gov/mailman/listinfo/realtime-feed-users
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> ********************************************************************************
>>>>>> Peter Schmidt                                      Tel: +46-18-4717104
>>>>>> Swedish National Seismological Network (SNSN)   Mobile: +46-73-3190975
>>>>>> Dept. of Earth Sciences:geophysics              e-mail: peter.schmidt at geo.uu.se
>>>>>> Uppsala University
>>>>>> Villavagen 16
>>>>>> SE-75236 Uppsala
>>>>>> ********************************************************************************
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Realtime-feed-users mailing list
>>>>>> Realtime-feed-users at geohazards.usgs.gov
>>>>>> https://geohazards.usgs.gov/mailman/listinfo/realtime-feed-users
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Adam Kalachman | Software Engineer | akalachman at google.com |
>>>> 256-509-5336
>>>>
>>>
>>>
>>
>>
>> --
>>
>> Adam Kalachman | Software Engineer | akalachman at google.com | 256-509-5336
>>
>
>

-- 

Adam Kalachman | Software Engineer | akalachman at google.com | 256-509-5336
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://geohazards.usgs.gov/pipermail/realtime-feed-users/attachments/20160426/be19dbb5/attachment-0001.html>