[EHPweb] gzipped xml files for RSS feeds now being generated

[Jeremy Fee]

Still waiting to hear on the .gz exclusion.  I re-enabled mod_deflate  
in the earthquake virtual host and there is a pretty dramatic drop in  
bandwidth.  I'll keep an eye on this, and if you notice ANY problems  
let me know.


Jeremy Fee
Geologic Hazards Team
(303) 273-8632
jmfee at usgs.gov



On Jul 15, 2008, at 4:26 PM, Jeremy Fee wrote:

> We just spoke with Kannan.  He is adding an exception for .gz files  
> so they will not be decompressed on the fly.  I am also re-enabling  
> compression on our origin servers to decrease our bandwidth usage.   
> This should not interfere with caching, while decreasing overall  
> bandwidth.
>
> He also verified that the reports and billing are based on  
> bandwidth used, compressed or otherwise.  We are still trying to  
> find out why there was no noticeable change in traffic after  
> enabling last mile compression.
>
> I'll send another email when these changes are in place,
>
>
> Jeremy Fee
> Geologic Hazards Team
> (303) 273-8632
> jmfee at usgs.gov
>
>
>
> On Jul 15, 2008, at 2:53 PM, Jeremy Fee wrote:
>
>> Chris and I are planning to call Akamai this afternoon to deal with
>> the on the fly decompression of true gzip files you describe below.
>> I'll ask Kannan about the billing too.
>>
>> AFAIK, the last mile compression service is free.  I'm not sure
>> whether they charge us for compressed or uncompressed volume.
>>
>> All modern browsers support on the fly compression, and since eQuake
>> runs inside firefox it does too.  To take advantage of compression
>> with wget or urllib2(python) you need to manually include the Accept-
>> encoding header and then check received content for compression.
>> Curl offers the --compressed flag which automates the whole process.
>>
>> I'll let you know what we find out on the call,
>>
>>
>> Jeremy Fee
>> Geologic Hazards Team
>> (303) 273-8632
>> jmfee at usgs.gov
>>
>>
>>
>> On Jul 15, 2008, at 1:17 PM, Robert Simpson wrote:
>>
>>> Hi Jeremy,
>>>
>>> Can you clarify some points for me?
>>>
>>> 1) Do we know if Akamai charges us for bandwidth before or after
>>> they perform "Last Mile Compression"?
>>> 2) Do we know if they are charging us extra for the "Last Mile
>>> Compression" service?
>>>
>>> This becomes a big deal since nearly 50% of our bandwidth is
>>> currently used by requests for uncompressed xml files, and, in
>>> fact, 33% of our bandwidth is filled by requests to one free
>>> Firefox plugin named eQuake.
>>>
>>> As a rough estimate 33% of the current $420K that USGS spends on
>>> Akamai each year is $140,000.
>>>
>>> At the moment, if I try to wget a compressed file, Akamai
>>> uncompresses it before it gets sent, so last mile compression seems
>>> more like gratuitous last mile uncompression.  Grrrr.  And they're
>>> probably charging us for that too!
>>>
>>> Bob
>>>
>>> -----------------------------------------------
>>> On Jul 15, 2008, at 11:47 AM, Jeremy Fee wrote:
>>>
>>>> Hi Bob,
>>>>
>>>> We shouldn't need to do this since akamai offers on the fly
>>>> compression.
>>>>
>>>> Jeremy Fee
>>>> Geologic Hazards Team
>>>> (303) 273-8632
>>>> jmfee at usgs.gov
>>>>
>>>>
>>>>
>>>> On Jul 15, 2008, at 12:06 PM, Robert Simpson wrote:
>>>>
>>>>> Hi Guys,
>>>>>
>>>>> I just modified Scott's /home/quake/REQS_MYSQL/rss.pl script so  
>>>>> that
>>>>> it generates compressed eq*xml.gz files (as well as the old
>>>>> uncompressed files) in directory  /eqcenter/catalogs.  These
>>>>> files are
>>>>> used in RSS feeds.  If you see any problems, please let me know.
>>>>>
>>>>> Bob
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> EHPweb mailing list
>>>>> EHPweb at geohazards.usgs.gov
>>>>> https://geohazards.usgs.gov/mailman/listinfo/ehpweb
>>>>
>>>
>> _______________________________________________
>> EHPweb mailing list
>> EHPweb at geohazards.usgs.gov
>> https://geohazards.usgs.gov/mailman/listinfo/ehpweb
>