[Shake-dev] ShakeMap installation and user inquiries

Tue Jul 10 00:53:57 UTC 2012

Hi Branden,

Sorry, I missed the 'pending' error, I just saw the one from grind that followed it.

All pending does is make a temporary web page that says that we're working on a particular event. It is a holdover from a time when it took a few minutes to process a ShakeMap. But with the newer ShakeMap software, and faster computers these days, it usually takes less than a minute to run ShakeMap. I think most operators have pending commented out of shake.conf. Even if it fails, it shouldn't cause any problems with later processes. I don't know exactly what the problem is, but if you are keen to investigate it, you can add '-dryrun' to your 'shake' command and it will spit out a list of commands that it would run, but not run them. You can then run them one at a time, and add the '-verbose' flag to any that seem problematic. If that doesn't yield any additional information, you can edit pending and find the lines that contain '$GENEX' and '$TRANSFER' and add '-verbose' to those, too. That will produce even more diagnostic output.

Or, like I said, you can just go into shake.conf and comment out pending (get rid of shakemail, print, and addon, too). I can't guarantee that the problem won't reappear when you try to run transfer, but that will at least tell us something. In any event, if grind, mapping, and genex run, you'll at least have maps you can run. 

Also, now that you've configured the DEM, you can add -qtm back to grind.

Bruce

On Jul 9, 2012, at 4:41 PM, Branden Christensen wrote:

> Bruce:
> 
> 
> Thank you for your prompt response! 
> 
> I downloaded the SRTM30 data, changed the install/macros file and configured shake.conf as:
> 
> default_flags   : retrieve
> default_flags   : pending
> #default_flags  : grind -qtm -xml
> default_flags  : grind -xml
> #default_flags  : mapping -timestamp -itopo -verbose
> default_flags  : mapping -gsm -timestamp -itopo -verbos
> default_flags   : shakemail
> default_flags   : genex -zip -metadata -shape shape
> default_flags   : print
> default_flags   : transfer -www -push
> default_flags   : addon
> default_flags   : plotregr -lab_dev 6
> 
> I am still getting the same error "transfer: Unable to access 9583161 download dir from /opt/shakemap/bin". Perhaps I am pointing to it incorrectly?
> 
> I have added the following lines to the install/macros file and run a "make veryclean" and "make all" after each time (not knowing if this is necessary or not):
> 
> DEMDIR = /opt/shakemap/dem
> DEMDIR = /opt/shakemap/dem/SRTM30/
> DEMDIR = /opt/shakemap/dem/SRTM30/DATA/
> 
> ./shake does not seemed to be pleased with any of these paths. 
> 
> Your thoughts?
> 
> 
> Best, 
> 
> Branden C. Christensen
> Director, OSOP
> branden.christensen at osop.com.pa
> www.osop.com.pa
> Volcan, Panama
> 
> 
> 
> 
> 
> 
> On Fri, Jul 6, 2012 at 5:35 PM, Bruce Worden <cbworden at gmail.com> wrote:
> Hi Branden,
> 
> See my responses embedded below:
> 
> On Jul 5, 2012, at 12:23 PM, Branden Christensen wrote:
> 
>> ...
>> make[2]: Entering directory `/opt/shakemap/src/contour'
>> /usr/bin/cc -O -I . -I /usr/include/gmt -I /usr/include -DGMT45 -c contour.c
>> /usr/bin/cc -O -I . -I /usr/include/gmt -I /usr/include -DGMT45 -c shpopen.c
>> shpopen.c: In function 'SHPOpen':
>> shpopen.c:421: warning: ignoring return value of 'fread', declared with attribute warn_unused_result
>> shpopen.c:431: warning: ignoring return value of 'fread', declared with attribute warn_unused_result
>> shpopen.c:510: warning: ignoring return value of 'fread', declared with attribute warn_unused_result
>> shpopen.c: In function 'SHPReadObject':
>> shpopen.c:1286: warning: ignoring return value of 'fread', declared with attribute warn_unused_result
>> /usr/bin/cc -O -I . -I /usr/include/gmt -I /usr/include -DGMT45 -c dbfopen.c
>> dbfopen.c: In function 'DBFOpen':
>> dbfopen.c:337: warning: ignoring return value of 'fread', declared with attribute warn_unused_result
>> dbfopen.c:357: warning: ignoring return value of 'fread', declared with attribute warn_unused_result
>> dbfopen.c: In function 'DBFClose':
>> dbfopen.c:416: warning: ignoring return value of 'fread', declared with attribute warn_unused_result
>> dbfopen.c: In function 'DBFWriteAttribute':
>> dbfopen.c:947: warning: ignoring return value of 'fread', declared with attribute warn_unused_result
>> dbfopen.c: In function 'DBFWriteTuple':
>> dbfopen.c:1156: warning: ignoring return value of 'fread', declared with attribute warn_unused_result
>> dbfopen.c: In function 'DBFReadTuple':
>> dbfopen.c:1200: warning: ignoring return value of 'fread', declared with attribute warn_unused_result
>> /usr/bin/cc  -L/usr/lib -L/usr/lib contour.o shpopen.o dbfopen.o  -lgmt -lnetcdf -lpsl -lm -o contour
>> /usr/bin/install -m 0755 contour /opt/shakemap/bin
>> /usr/bin/cc -O -I . -I /usr/include/gmt -I /usr/include -c sm_nearneighbor45.c -o sm_nearneighbor.o
>> sm_nearneighbor45.c: In function 'main':
>> sm_nearneighbor45.c:259: warning: passing argument 3 of 'GMT_check_lattice' from incompatible pointer type
>> /usr/include/gmt/gmt_init.h:74: note: expected 'BOOLEAN *' but argument is of type 'GMT_LONG *'
>> sm_nearneighbor45.c:259: warning: passing argument 4 of 'GMT_check_lattice' from incompatible pointer type
>> /usr/include/gmt/gmt_init.h:74: note: expected 'BOOLEAN *' but argument is of type 'GMT_LONG *'
>> sm_nearneighbor45.c:448: warning: passing argument 4 of 'GMT_y_out_of_bounds' from incompatible pointer type
>> /usr/include/gmt/gmt_boundcond.h:54: note: expected 'BOOLEAN *' but argument is of type 'GMT_LONG *'
>> /usr/bin/cc -O -I . -I /usr/include/gmt -I /usr/include -DGMT45 -c ga10.c
>> /usr/bin/cc  -L/usr/lib -L/usr/lib sm_nearneighbor.o ga10.o  -lgmt -lnetcdf -lpsl -lm -o sm_nearneighbor
>> /usr/bin/install -m 0755 sm_nearneighbor /opt/shakemap/bin
>> /usr/bin/cc -O -I . -I /usr/include/gmt -I /usr/include -DGMT45 -c dist_rrupt.c
>> /usr/bin/cc -O -I . -I /usr/include/gmt -I /usr/include -DGMT45 -c vector.c
>> /usr/bin/cc  dist_rrupt.o vector.o -lm -o dist_rrupt
>> /usr/bin/install -m 0755 dist_rrupt /opt/shakemap/bin
>> /usr/bin/cc -O -I . -I /usr/include/gmt -I /usr/include -DGMT45 -c dist_rx.c
>> /usr/bin/cc  dist_rx.o -lm -o dist_rx
>> /usr/bin/install -m 0755 dist_rx /opt/shakemap/bin
>> /usr/bin/cc -O -I . -I /usr/include/gmt -I /usr/include -DGMT45 -c directivity.c
>> /usr/bin/cc -O -I . -I /usr/include/gmt -I /usr/include -DGMT45 -c nrfuncs.c
>> /usr/bin/cc  directivity.o vector.o nrfuncs.o -lm -o directivity
>> /usr/bin/install -m 0755 directivity /opt/shakemap/bin
>> make[2]: Leaving directory `/opt/shakemap/src/contour'
>> ...
>> 
>> Are either of these warnings something to be concerned about? No other errors or failures were reported. 
> 
> 
> These don't look like the kind of errors that will cause problems, but one never knows until one actually tries to run the program.
> 
> 
>> I have chosen to use the following perl modules as most of the historic versions are NOT available at http://backpan.perl.org/:
>> Bundle::LWP v5.835
>> DBD::mysql v4.021
>> HTML::Template v2.91
>> XML::Parser v2.41
>> XML::Write rv0.615
>> enum v1.016
>> Time::CTime v2011.0517
>> Event v1.20
>> Mail::Sender v08.21
>> Config::General  v2.50
>> XML::Simple v2.20 
>> Time::y2038 v20100403
>> 
>> and then there is GMT: v4.5.1-1
> 
> 
> Yes, it's difficult to keep on top of all the module updates. They usually work these days since the language is fairly stable. If you run into module problems, some tweaking may be necessary.
> 
> 
>> 
>> 
>> Questions
>> Just a few simple questions:
>> 
>> 1. In an attempt to confirm that everything is working OK, I tried to execute shake over the example data included with ShakeMap at <shakemap_home>/data called "9583161". 
>> 
>> I entered <shakemap_home>/bin and ran:
>> ./shake -event 9583161 -verbose
>> 
>> the output was:
>> shake: Running: '/opt/shakemap/bin/../bin/retrieve -event 9583161'
>> shake: Running: '/opt/shakemap/bin/../bin/pending -event 9583161'
>> transfer: Unable to access 9583161 download dir from /opt/shakemap/bin
>> shake: Done with 'pending'
>> shake: Running: '/opt/shakemap/bin/../bin/grind -event 9583161 -qtm -xml'
>> cp: cannot stat `/tmp.topo2grd.24763.grd': No such file or directory
>> grdsample: Could not find file [/tmp.topo2grd.24763.grd.1]
>> Could not find /tmp.topo2grd.24763.grd, aborting.
>> grind: Error: Error in topo/sitecorr map creation, exiting.
>> shake: Error in grind: 256
>> 
>> How do I set about fixing the transfer error? I have already reviewed <shakemap_home>/config/transfer.conf for mistakes. 
> 
> This isn't a transfer error, it's an error in grind. Essentially, you have not set up a qtm_file in grind.conf, so grind tries to look for a DEM to convert to Vs30, but not finding it, it issues this rather unhelpful error message. If you run grind without -qtm, it should work. Note that 'mapping' will probably complain that there is no topography, as well. You'll need to configure this in mapping.conf.
> 
> If you are running ShakeMap in a fixed region, it is probably best to generate fixed Vs30 and topography files, and configure them in grind.conf and mapping.conf. If you are running ShakeMaps globally, it is probably best to get the SRTM30 DEM, configure include/macros with the correct path, leave qtm_file unconfigured in grind.conf, leave the topo files unconfigured in mapping.conf, and run mapping with the -gsm option.
> 
> The DEM and some associated tools can be had here:
> 
> ftp://ftpext.usgs.gov/pub/cr/co/golden/lin/ShakeMap%20Portable/SRTM30.tar.gz
> 
> There is also a ShakeMap virtual machine (runs on Virtual Box) on the same site, in case you run into too many installation problems.
> 
> 
>> I then tried to locate tmp.topo2grd.24763.grd and came up empty handed. Perhaps this is created when transfer succeeds. 
>> 
>> 
>> 2. I see that Earthworm's gmew outputs files to .../input/. e.g., 
>> 
>> 998_dig_dat.xml
>> 
>> So I will set the paths so that this information is written to <shakemap_home>/data/input.
>> 
>> Any other important bits? 
>> 
>> 
>> 3. Was the added functionality in gmew enough to be a wholesale replacement to shakemapfeed?
> 
> 
> I don't know anything at all about Earthworm, so someone else will have to help you.
> 
> 
>> 4. Do there exists alternatives to ShakeMap with less overhead (less email, less websites, etc.)? Something similar to the PSD plots calculated by obspy as an alternative to configuring an entire PQLX? ShakeMap appears to be a great distribution system. But I am also interested in a simple tool that just outputs pngs of the ShakeMaps themselves. 
> 
> 
> Not that I know of. All you really need to get the maps are grind and mapping (assuming you have the input data set up). genex will convert the PostScript into JPEGS and make a few other products. transfer can be ignored, but you could also set it up to copy the products you want to a directory of your choice. Everything else is optional. You could pretty easily write a script to run grind and mapping, then convert the maps you're interested in to PNG. 
> 
> Hope this helps,
> Bruce
> 
> 
> 
>> 
>> 
>> 
>> Thank you for your insights and comments. 
>> 
>> 
>> 
>> Kind Regards,
>> 
>> 
>> Branden C. Christensen
>> Director, OSOP
>> branden.christensen at osop.com.pa
>> www.osop.com.pa
>> Volcan, Panama
>> 
>> 
>> 
>> 
>> _______________________________________________
>> Shake-dev mailing list
>> Shake-dev at geohazards.usgs.gov
>> https://geohazards.usgs.gov/mailman/listinfo/shake-dev
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://geohazards.usgs.gov/pipermail/shake-dev/attachments/20120709/47cf61c7/attachment-0001.html>