Wednesday, December 28, 2016


tl;dr: API Gateway may treat binary as text and mangle the content, such as when our backend server returned Content-Type: text/html; charset=UTF-8 with Content-Encoding: gzip responses.  My workaround for Apache 2.4: SetEnvIf x-amzn-apigateway-api-id .+ no-gzip.  (If the request contains the header with any value, don’t gzip the response.)

Sunday, December 4, 2016

My Path to Programming

Inspired by Dan Luu’s post about the same, in turn based on Tavish’s.

For me, it really started with a sibling rivalry.  I wanted to try everything my older brother did, and secretly, I wanted to be better at it than he was. We had wars about who could write better BASIC programs.  When my dad bought AssemPro for our Amiga, my brother decided to learn it: therefore, so did I.

Monday, September 12, 2016

Update: rumprun and php7

Months ago, I wrote about not being able to get libmemcached to link into a static PHP build.

I don’t take “no” for an answer from code.

I have not only gotten it to link (with a carefully targeted replacement of the linker in the build scripts), I have patched up opcache to both configure correctly while cross-compiling, and to build statically.

I seriously don’t take “no” for an answer from code.

That’s where I stand.  I’ve been focusing on getting some music done, notably this FF8 remix, so rumprun is on the back burner.  Still, the php7-full branch is what I would consider “usable” now.

If you’re reading this in the future, you should check the wiki for the status.

Friday, September 9, 2016

AWS API Gateway: Returning 404 Errors with OAuth 2.0

As discussed before, we’ve been building out some services using AWS API Gateway.

We have an OAuth 2.0 infrastructure that predates API Gateway, and we’ve had a lot of problems with third parties being able to use the APIs behind API Gateway.  Almost any mistake that can be made with the Authorization header set leads to an unhelpful message from Amazon CloudFront (which technically underpins API Gateway): “not a valid key=value pair” pointing to the access token in the Authorization header.

As it turns out, one of these error cases is a response that should generate a 404 Not Found response, because the URL doesn’t exist in API Gateway.

There’s a workaround to fake 404 messages: build fake endpoints into the API definition.

Saturday, July 9, 2016

A hat beside the “PHP7 vs HHVM” ring

I spent all my free time, all week, exploring the difference in performance between PHP 5.6.22, PHP 7.0.8, and HHVM 3.14.1.  (HHVM 3.14.2 came out later in the week, but I didn’t want to redo everything.)

In the end, it turned out kind of useless as a “real world” test.  The server doesn’t have L3 cache, and the client is a virtual machine.  I also didn’t run a large number of trials, nor tune the server stacks for optimum performance. In fact, the tunings I tried had no effect, at best.

tl;dr conclusions:

  • HHVM proxygen is almost completely amazing.  If you have the option, it is probably what you want. It just crashes hard when concurrency exceeds its open file limit.
  • nginx+FastCGI hits a resource limit somewhere, and starts failing requests at higher concurrency, like 160.
  • apache+FastCGI does the same… but the limit is higher, between 256 and 384. The price for this is that it’s only 86% as many requests per second.
  • Providing more FastCGI workers makes the errors hit sooner, but ramp up more slowly.
  • I’m really disappointed in nginx.  I expected more.  A lot more.

Saturday, July 2, 2016


Three years ago, I was porting a Perl CGI-based system to FastCGI, one URL at a time, using mod_rewrite to do the dispatching.  (If the exists where the FCGI dispatcher will look for it, invoke this request via FastCGI.)  A consequence of this is that the core library code needs to run on both paradigms, since both front-ends use it.  That runs straight into problems when the CGI access checking functions blithely print and call exit when the access is denied.

Instead of updating 60+ scripts to read e.g. $session = get_session() or Site::CGI->go_login; I decided to get hackier.

Wednesday, June 29, 2016

rumprun php7 experiment results

I forked rumprun-packages to build its php7 package with more features by default.  The original says it’s an “unmodified PHP 7.0.3” build, but it uses --disable-all to build with a minimal number of extensions by default, and that’s not really great for showing just what could work on rumprun.

I made a “mega” build which contains everything I can possibly include into it, including packaging up libjpeg-turbo and libpng so that this PHP can use gd.

I have been unable to get the memcached extension to work, though.  libmemcached-1.0 is written in C++ and depends on some exception-handling symbols.  PHP links with $(CC), which doesn’t include any C++ support.

I’d probably be more disappointed in this, except that I don’t know if rumprun PHP is all that useful.  Options are the single-threaded CLI HTTP server (not intended for production), or using ye olde php-cgi SAPI… which is also not a multithreaded option.  It expects to run single-threaded child processes under a FastCGI process manager.  (php-fpm simply integrates a process manager, so that you don’t need to find a separate one and integrate it; it’s also multi-process.)

And rumprun doesn’t have processes.  It’s a unikernel.