Client side includes

Here's a future tech to keep an eye on: the "seamless" attribute for iframes. (WhatWG spec) It basically acts as a client-side include for HTML code (and so, really, isn't much like a frame at all). So, just as we've always done server-side, we can now ("now," meaning "lord knows when"... browser support ugh) break a page into fragments and serve each portion from a different URI, which are recombined by the browser. This gives us great control over caching and distribution. We could, for example, use a frame for the navigation that is served from a CDN and refreshed only every 24 hours, and one for an article content that caches for one hour (in case of updates), and one for social media stuff that is never cached. Unlike with regular iframes, all of this can happen within one DOM space, sharing styles, scripts, etc.

Of course, much of the same can be achieved with javascript templating and AJAX fetches. But that's not always appropriate, and it always adds an additional layer of complexity that might be overkill. This is a conceptually simpler approach, and I like having it as an option. Unfortunately, we don't really have it as an option yet. There's no browser support; it's not even mentioned yet on caniuse. So, future-Dave: let's watch this one as it emerges. It looks like a good trick for scalability on sites where the content on a page has a mix of freshness requirements. (And isn't that pretty much every site?)

Defaulting on performance

In creating a highly performant Web environment, I've frequently been hampered by the very conservative default limits that exist in many technologies. In fact, I'd say that at least 4 out of every 5 performance bottlenecks I've encountered have come not from the system being actually overwhelmed, but rather from a self-imposed limit. Systems have numerous in-built constraints on concurrency, memory usage, and many other ways to prevent themselves from utilizing all of their available resources and realizing their true potential. Servers, you need a life coach. I'm here for you.

Of course, these limits were placed there by very wise people with very good reasons. For example, on *NIX operating systems, the designers were considering the needs of a multi-user situation: in a university CS lab where dozens of naive and/or mischievous undergrads are sharing clock cycles of a single CPU, it's important that no one user is allowed to chew up all the resources and bring down the system. But a Web server typically runs just one user, and has much more capacity than it is configured to use by default. When you're chasing down an international superspy in your Bugatti Veyron (who hasn't been there?), you've gotta pop out the electronic speed limiter and go for it. Likewise, when your NCIS slash fiction goes viral (who hasn't been there?), you need to goose the config and let those servers fly.

So, here is an incomplete list of some of the configuration-imposed constraints I've encountered over the years that are typically easy to relieve -- if you do it before the crowds arrive:

• Linux: iptables' ip_conntrack default database size is too small [see racker hacker]
• MySQL: max_connections default is low [see electric toolbox]
• Apache: prefork module is single-threaded [see serverfault]
• MySQL: query cache is disabled by default [it's not always a good idea to turn it on, but when it's good, it can be very good. see mysqlperformanceblog]
• Tomcat: the JDBC default connection pool size is small [see tomcat 6 docs, but also, tomcat 7's new hotness]
• Java: default JVM memory settings don't take advantage of available memory, and default garbage collection can create long "stop-the-world" pauses [this is a deep topic, but here's an intro]
• Linux: default max open files is too low [see stackoverflow]
• MySQL: back_log default is low [see MySQL documentation]
• MySQL: innodb_thread_concurrency default is low [see stackexchange -- though be aware, setting it to "0" (infinite) might be too much]
• Linux: net.core.somaxconn, net.core.netdev_max_backlog are too low [see nginx.org]

Who is your first customer, and why?

More than once recently, I've found myself giving advice in the form of this question from a favorite professor: "Who is your first customer, and why?" This was something he posed to the class repeatedly, in the context of creating a go-to-market strategy for a startup. But these days -- or, heck, fifteen years ago -- what's good for a startup is also good for a person, and any project you might do.

So, it's a valuable question to ask when planning out a project: who is your first [ customer | user | reader | fraggle ], and why? It's natural and healthy to dream big and aim for a million users. But until the case is made to your investor (and yourself) that customer #1 is ready and willing, it's hard to believe that the throngs will follow.

Share the load

For load-testing OvationTix, we've tried a few approaches over the years. The first time around, we used HP LoadRunner, which is an enterprise-level tool with a price to match. It was pretty easy to use, and we got the data we needed, but it was too expensive to become a part of our ongoing development process. Ideally, we'll load-test every release before deploying, and I don't want cost concerns to intimidate us into holding back from deploying good code when it's ready to go.

So we moved to jmeter, running in Amazon EC2 cloud instances, which of course was cheaper. I set up some (admittedly clunky) Windows instances -- a controller and some generators -- and went to work. Again, we got the data we needed, but now the workflow was cumbersome. We had to launch the generators, hope they booted correctly, figure out their IPs, copy those back to the controller, then fire up the scripts, and then we had problems with the test data saturating the connection between the generators and the controller. It was fair, but not great.

For this year, we made it our goal to have a smoothly automated system -- still based around jmeter, which we like. First, we tried BlazeMeter. It's a jmeter PaaS, which is a really cool idea and promises to take care of the infrastructure so we could focus on writing the tests. It's not bad at all, and I think we may use it in the future, but for now, the costs were higher than we wanted, there were too many limitations on usage (the price tiers control things like ramp-up time, max users etc.), and the reporting wasn't as transparent as we wanted.

Finally, we found jmeter-ec2, which is a wrapper around Amazon's API that automates launching linux micro instances, deploying resources to those instances, firing up the test, and aggregating results. It's a lightweight script that runs in a shell and eliminates the need for a dedicated controller -- instead, each generator controls its own virtual users, and the condensed results are sent back to the shell, which makes for much less traffic between the instances (therefore, no saturation). The data collected isn't as deep as with the other approaches, but for our purposes, that's okay. We're mostly interested in simply finding out how many users we can throw at the site before it crashes. Since our plan is to take over the world, our target for concurrent users is currently 7,057,131,972. Wish us luck.

Cache Cachet

One of our goals for TheaterMania is to achieve infinite* scalability. I would like to be able to feel deeply confident that we could handle as much load as could possibly be thrown at us, because we have infinite* scalability. Why the asterisk? Because I'm only really looking to scale reads, and only reads of non-personalized data. There are, of course, ways to scale out writes and personalized reads (e.g. for logged-in users) but the nature of the application is that those are much less essential, and besides, it would be an isolated project so let's do first things first.

So then, infinite* scalability: of course, it's about caching. The approach we've decided on is to render complete HTML pages and store them on a CDN. Any personalization can happen via AJAX calls; as long as those calls fail gracefully, the server handling dynamic content can crash, and the core content of the site is still live, being served by the CDN. For a lot of static content, we use Amazon S3 as a sort of cheapo CDN, but it's not really designed to serve massively parallel requests (I'm not sure what would happen if we tried), and it won't request content updates automatically from an origin server. Fortunately, true CDNs abound, and our plan is to leverage one. Next step is to comparison-shop CloudFront, CloudFlare, and ??? (Akamai?). I'm hoping that since our needs are relatively modest -- we don't need ultra-low latency or global edge servers -- we can find one that fits our budget.

Our challenge then will be to make sure we really understand the cache-manipulation API. As Gautam told me, "when you cache complete pages, you have to be sure you have a very reliable cache-busting mechanism." Wise.

301 means 301

Ever since I took control of my own DNS, I've been doing a lot of redirecting, bouncing people around to temporary sites, or adding special subdomains. (My host, dnsmadeeasy.com has a feature called "HTTP redirection records" that lets me serve the redirect straight from DNS, which is convenient.)

One mistake I've made a few times, though, is using a 301 (Moved Permanently) when I should use a 302 ("Found" a/k/a Moved Temporarily). The problem with this is that because 301's are permanent, browsers are allowed to cache them. Which means that once you establish a 301, it can be very hard to undo it, if it's cached by users' browsers. 302s, meanwhile, are loose and flexible; the browser will re-request the original URI on every request, and if the redirect has been removed, or changed, the browser will detect that.

So use 301s with care. Start with a 302, and make sure it works -- and make sure you really really want this to be permanent -- before locking it down as a 301.

Dave's Simple Rules

1. Engage the user. Respect the user. Create an environment for collaborative discovery.
2. Complexity ≠ Sophistication. Seek elegance.
3. Find a question. Find the answer. Share the answer. Find another question.
4. Be excellent today.

May 15, 2012