AWS Log Cost

metadata
  • keywords:
    • software
    • AWS
    • hosting
    • python
    • matplotlib
    • pytz
  • published:
  • updated:
  • Atom Feed

One of the websites that I am administrator of recently moved to AWS for it’s web hosting. The website is entirely static, i.e. it does not use any server-side scripting, and therefore I elected to: use CloudFront as the CDN; and use S3 as the back-end storage. I followed two excellent walkthroughs which made the process very easy:

These two walkthroughs get the user to enable logging - both the logging of direct access to the back-end storage and the logging of web access to the CDN. After a while the log bucket started to fill up with these log files. I decided to periodically download (and remove) the log files (using the AWS SDK for Python) to allow them to be surveyed and statistics to be compiled. Both CloudFront and S3 really are quite verbose and they produce copious amounts of logs - this got me thinking: how much are these logs costing me?

Below you will find two plots which cover the first 2 months of the website on AWS: the first is the number of log files created each day; and the second is the total size of the log files created each day. The legend item “root” is for the logs from S3 and the legend item “cdn” is for the logs from CloudFront. You may download the data as a CSV file if you wish. For context, the website receives about 1,000 hits per day (for the purposes of this discussion a “hit” is a “HTTP GET request that resulted in a HTTP 200 response”).

Download:
  1. 512x250 (25.6 KiB)
  2. 1,024x499 (45.4 KiB)
  3. 2,048x999 (94.0 KiB)
  4. 3,041x1,483 (86.6 KiB)
Download:
  1. 512x251 (22.0 KiB)
  2. 1,024x502 (41.0 KiB)
  3. 2,048x1,003 (84.5 KiB)
  4. 3,028x1,483 (79.7 KiB)

How much does this cost? Using the Amazon S3 pricing tables I was able to estimate what this would cost if I were to retrieve the logs every month, see below.

cdn log number cdn log size root log number root log size
361 /day 0.000297 GiB/day 792 /day 0.001030 GiB/day
10,983 /month 0.009045 GiB/month 24,099 /month 0.031339 GiB/month
$0.0044 /month $0.0002 /month $0.0096 /month $0.0007 /month

This totals about $0.015 per month which, after taxes, is roughly $0.02 per 1000 hits per month. The above estimation only includes charges for GET requests (for downloading the log files), it does not include PUT requests (for AWS creating the log files in the first place) because this forum post implies that they are not charged when logs are created. Finally, I have ignored “Data Transfer OUT” costs (as it is $0.00 /GiB/month for the first GiB) and I have ignored “S3 Inventory” costs (as it is $0.0025 per million objects listed).

It is clear that the dominant cost is the number of GET requests that are made to retrieve the logs and that the actual size of the log files is inconsequential. Still though, approximately $0.02 per month to download the log files is entirely reasonable.