[EHPweb] gzipped xml files for RSS feeds now being generated

[Jeremy Fee]

We just spoke with Kannan.  He is adding an exception for .gz files  
so they will not be decompressed on the fly.  I am also re-enabling  
compression on our origin servers to decrease our bandwidth usage.   
This should not interfere with caching, while decreasing overall  
bandwidth.

He also verified that the reports and billing are based on bandwidth  
used, compressed or otherwise.  We are still trying to find out why  
there was no noticeable change in traffic after enabling last mile  
compression.

I'll send another email when these changes are in place,


Jeremy Fee
Geologic Hazards Team
(303) 273-8632
jmfee at usgs.gov



On Jul 15, 2008, at 2:53 PM, Jeremy Fee wrote:

> Chris and I are planning to call Akamai this afternoon to deal with
> the on the fly decompression of true gzip files you describe below.
> I'll ask Kannan about the billing too.
>
> AFAIK, the last mile compression service is free.  I'm not sure
> whether they charge us for compressed or uncompressed volume.
>
> All modern browsers support on the fly compression, and since eQuake
> runs inside firefox it does too.  To take advantage of compression
> with wget or urllib2(python) you need to manually include the Accept-
> encoding header and then check received content for compression.
> Curl offers the --compressed flag which automates the whole process.
>
> I'll let you know what we find out on the call,
>
>
> Jeremy Fee
> Geologic Hazards Team
> (303) 273-8632
> jmfee at usgs.gov
>
>
>
> On Jul 15, 2008, at 1:17 PM, Robert Simpson wrote:
>
>> Hi Jeremy,
>>
>> Can you clarify some points for me?
>>
>> 1) Do we know if Akamai charges us for bandwidth before or after
>> they perform "Last Mile Compression"?
>> 2) Do we know if they are charging us extra for the "Last Mile
>> Compression" service?
>>
>> This becomes a big deal since nearly 50% of our bandwidth is
>> currently used by requests for uncompressed xml files, and, in
>> fact, 33% of our bandwidth is filled by requests to one free
>> Firefox plugin named eQuake.
>>
>> As a rough estimate 33% of the current $420K that USGS spends on
>> Akamai each year is $140,000.
>>
>> At the moment, if I try to wget a compressed file, Akamai
>> uncompresses it before it gets sent, so last mile compression seems
>> more like gratuitous last mile uncompression.  Grrrr.  And they're
>> probably charging us for that too!
>>
>> Bob
>>
>> -----------------------------------------------
>> On Jul 15, 2008, at 11:47 AM, Jeremy Fee wrote:
>>
>>> Hi Bob,
>>>
>>> We shouldn't need to do this since akamai offers on the fly
>>> compression.
>>>
>>> Jeremy Fee
>>> Geologic Hazards Team
>>> (303) 273-8632
>>> jmfee at usgs.gov
>>>
>>>
>>>
>>> On Jul 15, 2008, at 12:06 PM, Robert Simpson wrote:
>>>
>>>> Hi Guys,
>>>>
>>>> I just modified Scott's /home/quake/REQS_MYSQL/rss.pl script so  
>>>> that
>>>> it generates compressed eq*xml.gz files (as well as the old
>>>> uncompressed files) in directory  /eqcenter/catalogs.  These
>>>> files are
>>>> used in RSS feeds.  If you see any problems, please let me know.
>>>>
>>>> Bob
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> EHPweb mailing list
>>>> EHPweb at geohazards.usgs.gov
>>>> https://geohazards.usgs.gov/mailman/listinfo/ehpweb
>>>
>>
> _______________________________________________
> EHPweb mailing list
> EHPweb at geohazards.usgs.gov
> https://geohazards.usgs.gov/mailman/listinfo/ehpweb