[EHPweb] rsync creating problems on ehpmaster and ehpbackup?

[Eric Martinez]

>> 2) rsync of pager stuff to ehp1-4  [4 rsyncs going simultaneously]
>>

Each PAGER rsync is a step in the larger PAGER indexing process. This  
is (in my opinion) well-documented in the wiki (http://ehptools.cr.usgs.gov/trac/wiki/PAGER 
).

>> Recommendations:
>>    C)  Could these rsyncs also be logged to a file in /var/log?

This along with all other status information for each indexing run  
(run every 2 minutes) is logged in an individual file.  Unexpected  
errors are also logged and emails are sent to myself when something  
goes seriously wrong.  Rsync issues were never resolved and despite  
mine, Chris's, and Jeremy's best effort we cannot figure out why these  
rsyncs fail (generally to ehp1 for me).

>>    D)  Could a list of just the files that had changed be made in
>> advance, and only those files get rsync'ed?

Syncing the entire (realtime) content area of PAGER is done  
intentionally.  While the entire directory tree is scanned, only  
changed files are pushed.  Issues with failing rsyncs (see above) can  
cause data to get wildly out of sync without a regular sync of the  
entire directory.  This has happened before and so a full directory  
sync was decided as the way to go.

----

On an unrelated note, I did sync a reasonable about of content today  
(about 1/2 Gig).  This may have caused some alarms to trigger.

Thanks,
	~Eric.