[EHPweb] EIDS problems

[David Oppenheimer]

We can log in root. Want us to do a fsck?

-------------------------------------------------------
David Oppenheimer                   office:650.329.4792
U.S. Geological Survey              fax:   650.329.4732
345 Middlefield Road.-MS 977    email: oppen at usgs.gov
Menlo Park, CA 94025


-----Original Message-----
From: Jeremy M Fee [mailto:jmfee at usgs.gov] 
Sent: Monday, October 05, 2009 10:05 AM
To: David Oppenheimer; Silverman Stan; Stan Schwarz
Cc: Chris Bidwell; EHPweb
Subject: Re: [EHPweb] EIDS problems

eids2 and eids3 are having strange filesystem errors (saying read only  
although it isn't).  It sounds like the filesystems may have been  
corrupted and need to be repaired (according to google results).

Stan Silverman and Stan Schwarz need to check out /var/log/messages  
(as root) on eids2.wr.usgs.gov and eids3.gps.caltech.edu to find out  
what is going on.

Jeremy


On Oct 5, 2009, at 10:54 AM, David Oppenheimer wrote:

> EIDS2 is spewing the following message to the console.
>
> EXT3 - fs error (device dm0) in start_transaction. Journal aborted.
>
> I can log into the machine as user ppicker from another machine, and  
> it is
> responsive, but I am not sure I can log in via the console.
>
> Let me know if you need me to do something.
>
> -David
>
> -------------------------------------------------------
> David Oppenheimer                   office:650.329.4792
> U.S. Geological Survey              fax:   650.329.4732
> 345 Middlefield Road.-MS 977    email: oppen at usgs.gov
> Menlo Park, CA 94025
>
>
> -----Original Message-----
> From: ehpweb-bounces at geohazards.usgs.gov
> [mailto:ehpweb-bounces at geohazards.usgs.gov] On Behalf Of Jeremy M Fee
> Sent: Monday, October 05, 2009 7:48 AM
> To: Chris Bidwell
> Cc: EHPweb
> Subject: Re: [EHPweb] EIDS problems
>
> Correction, it appears eids1 was not affected by this (whew) and
> messages have not been interrupted.
>
> eids2 and eids3 are still effectively down.
>
>
> Jeremy
>
>
> On Oct 5, 2009, at 7:49 AM, Jeremy M Fee wrote:
>
>> It looks like eids1, eids2, and eids3 are having problems running
>> EIDS and QDDS.  This appears to be down since yesterday.  I can't
>> access /var/log/messages, which likely contains more information.
>>
>>
>> [eids at eids2 QDDS]$ java -jar QDDS.jar
>> Creating new log file: 200910051342
>> Problem in LogIt Constructor: java.io.FileNotFoundException: log/
>> 200910051342 (Read-only file system)
>>
>> [eids at eids2 EIDS]$ java -jar bin/ISTINotifServer.jar
>> Exception opening log file "log/NotificationServer_20091005.log
>> (Read-only file system)":  java.io.FileNotFoundException: log/
>> NotificationServer_20091005.log (Read-only file system)
>>
>> [eids at eids2 EIDS]$ touch test.text
>> touch: cannot touch `test.text': Read-only file system
>>
>>
>> Mount shows / as mounted read-write, and, from what I've read, this
>> is due to filesystem corruption?
>> [eids at eids2 EIDS]$ mount
>> /dev/mapper/VolGroup00-LogVol00 on / type ext3 (rw)
>> none on /proc type proc (rw)
>> none on /sys type sysfs (rw)
>> none on /dev/pts type devpts (rw,gid=5,mode=620)
>> usbfs on /proc/bus/usb type usbfs (rw)
>> /dev/sda1 on /boot type ext3 (rw)
>> none on /dev/shm type tmpfs (rw)
>> none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
>>
>>
>>
>> Jeremy Fee <jmfee at usgs.gov>
>> Geologic Hazards Team
>>
>
> _______________________________________________
> EHPweb mailing list
> EHPweb at geohazards.usgs.gov
> https://geohazards.usgs.gov/mailman/listinfo/ehpweb
>