83% Bandwidth Reduction via API Response Change

A typical response to a StatHat stat API call looks like this:

HTTP/1.1 200 OK
Content-Type: application/json
Date: Tue, 02 May 2017 14:53:45 GMT
Content-Length: 25
Connection: keep-alive

{"status":200,"msg":"ok"}

It’s pretty small. 152 bytes. But users sent in 150 billion requests last month, and StatHat responded with that 152 bytes to them all, which is 20 terabytes.

After studying RFC 7231, we are going to change the default success response to:

HTTP/1.1 204 No Content

It is 25 bytes (maybe 26 with a blank line after the header), 127 bytes leaner.

The 204 response is an accurate description of a successful API request:

The 204 (No Content) status code indicates that the server has successfully fulfilled the request and that there is no additional content to send in the response payload body.

While the Date header field is encouraged, it is optional:

An origin server MUST NOT send a Date header field if it does not have a clock capable of providing a reasonable approximation of the current instance in Coordinated Universal Time.

So let’s just pretend we don’t have a good clock.

None of the official StatHat libraries look at the body of the response, they just care about the 2xx success status. We have tried this response in production on a subset of the requests and have not received any reports of it being a problem, so we are rolling it out for all requests.

Note that we will still provide details for multiple stats uploaded in a JSON request and any error cases.

If you have code that parses the original body and would like to continue doing so, include a vb=1 request parameter with your POST or GET request and the servers will respond with the original verbose body output.

This change should remove about 17 terabytes of useless data from the internet pipes each month.


Comments? Send us a tweet.

Permalink:

Previous:
Is Pied Piper using StatHat?
Next:
Send alerts to Keybase team chat channels