A Day at the Maker Faire

The Maker Faire has always been one of those things that have sounded cool and we've meant to go to, but hadn't got around to. This last weekend we went up and a had a very tiring, but interesting day out. We were a little hampered by the fact that we came in through the back (Caltrain) door, and didn't get a map. It turns out we missed two whole buildings, and didn't find the good food zone until after we had eaten some fairly lame sandwiches. Still, a good time was had by all.

The best part is talking with the individuals playing around with random hacks in their garage: "how does it work?" and "what is this?" always led to interesting conversations. There's really nothing so fun as listening to someone who is excited by what they are doing explaining it to you. Since we came in the back door, the first zone we entered was the musical hack zone, which immediately sold the boy child, who engaged in some in-depth discussions of types of pickups and digital effects on electric guitars. Our favourite here was the guy with the Wii controller in his head of his guitar and the flat-panel display in the body, wired up so that particular chords have particular imagery, flinging the guitar head down gave a wah-wah effect with a coordinated visual effect. Hitting a button gave a distorted sound, and a distorted image to match. Credit where credit is due: Ben Lewry, and he'll make one for you too, apparently.

Most interesting reward for lowest expectations: A small tent, looking for all the world like a something you'd expect to find a Tarot reader in, tucked away among the 3-D cameras and glasses. Even the label wasn't particularly enticing: "holomagistics" or something of that sort. Inside the small, darkened tent was a small oscilloscope screen showing simple looped figures. Nothing too exciting, except the figures figures are 3-D. No fancy glasses. No need to stand in exactly the right spot. Amazing. If I understood correctly, this is done by reprogramming the scanning so that vertical scan lines alternate between different (computed) perspective views, and your brain performs the object fusion to give the sense of depth. Someone, somewhere, get this guy to think beyond "coordinated music and image displays" and invest.

The kinetic sculptures were engaging and beautiful (I love the displays in science museums of old brass-and-crystal machines, my favourite by far is the Fourier analysis machine in the London science museum, all gleaming spheres and dials), but my favourite art was Bulatov's metal sculptures. Cuteness points to the people with the candy-fab machine, slowly churning our sintered sugar creations."Please don't eat the sculptures!" is not a sign you get to read every day.

We did appreciate the Exploratorium's little puzzles too, though. My favourite: hollow aluminum tube, about 4 or 5 feet long, and a magnet. Drop the magnet through the tube, and it takes a suspiciously long time to fall through. This is the Exploratorium, mind, so we set aside our initial suspicions of some kind of slight-of-hand.

Somewhere late in the afternoon, we decided that we were well and truly tired out, so we sought out the Tesla coils, had a nice chat with the fellows there about their plans to build 12-story tall Tesla coils in the Nevada desert, and watched the lightning show. Even the 12 foot scaled down versions, running well below capacity, were impressive enough, and a good way to sign off the day.

Invidious comparisons

What with World Cup Qualifying, the UEFA Champions League, the CONCACAF Champions League, the Premiership, MLS, and the UEFA Cup, I've been spending a lot of hours watching football the last few weeks.

While it is true that it is hard to beat the top 4 Premiership teams for skill, as a competition the Premier League has become downright boring. The fact that in any given season the question of which team will come in 5th provides the most drama and the current spark of interest in games like Man Utd vs Aston Villa is based mainly on the fact that Chelsea and Man United have been showing cracks in their invulnerability is telling. The UEFA Champions League has essentially become the Premier League with the role of whipping boy played by various French teams. Oh, what a shock! Liverpool against Chelsea. There is a charm in watching Chelsea dink their little triangles up and down the field and in watching Ronaldo perform his little steppy-steppy moves, but honestly, is there really much surprise in watching Man Utd win on a late goal? Again? Yawn.

It used to be that the Euro matches provided the same pleasure that the World Cup did: watching teams you'd barely heard of and learning about new fantastic players you'd never seen before. Given that the same dozen top teams swap around the same top players, you don't get that so much, except in some of the early rounds.

On the other hand, the World Cup qualifiers have been rather fun, especially given the plethora of new European teams to enjoy: Kazakhstan has a young and lively team that's rather fun to watch, if somewhat overmatched -- who knew? We'll miss you in the final.

Other teams we'll miss: Gosh, Man City versus Hamburg. Two teams charging up and down the field going at it hammer and tongs. There was more pure entertainment in the first half-hour than in half a season at Stamford Bridge. Holy smokes!

The biggest entertainment surprise for me has been the CONCACAF Champions League semifinals. Sure, I'd heard of Cruz Azul and Atlante, but never really seen them play. And Santos Laguna? The Puerto Rico Islanders? Not on my radar. I confess to having watched the tournament only intermittently, largely to track the team-formerly-known-as-Earthquakes. The semi-finals were fantastic fun, however (at least for a neutral). I once heard the perfect soccer match described as having at least one change in the lead, at least one great goal, and at least one moment of controversy. Both the semi-finals obliged, although it was hard to beat Atlante game for sheer drama: an extra-time (winning) penalty and 3-red-card fight 3 minutes after that. Sad as I was the Dynamo didn't make it this far, I'm looking forward to Cruz Azul vs Atlante in the final, which is something I hardly expected.

Speaking of the Dynamo, they came to town a couple weeks ago to play the team-currently-known-as-Earthquakes. It was just as well that Ching was off on World Cup duty because if he had been the one finishing chances instead of Mullen, things may have turned out much worse for the home side. (I've whined about the MLS continuing with a full program during qualification week before: I see they now partly accomodate the national team by giving their wannabe Man Utd of the US the week off. Funny how LA always gets the special treatment.) It was about a half-hour into it, about the time Mullen skied his third "this is a goal, oh wait, no it isn't" chance, that I said to my son "I don't think this is going to stay scoreless" and he said "I think once there is one goal, there will be a boatload." Truer words were never spoken: 15 minutes later it was 3-2. It is odd and scary for a Yallop team to have such a fragile looking defense. It is going to be a long season if they don't tighten that up.

Season Two of the New-New-New-Quakes

Earthquakes 0 New England 1

Drat. On paper a midfield with Huckerby, Corrales, Convey, and Alvarez looks pretty hot, and there were brief flashes of brilliant football on Saturday. Mostly, however, we saw the classic weakness of the Quakes (both the new-new-new-Quakes and the new-new-Quakes under Yallop's tutelage previously), which is to say: nice approach work, lousy finishing. Or no finishing at all. Once again we revive the impassioned plea of Quake fans: "SHOOOOT!"

We also saw Huckerby spending far too much of the match completely disconnected from play out on his own on the wing. Has someone told him he's not allowed to stray 5 yards from the touchline or something? Or is the loss of the one guy on the field capable of hitting him at full sprint with a 40-yard cross-field pass (Ronnie O'Brien), put a huge hole in the team?

On the whole, the Quakes played the better football, but neither team looks particularly good, and after the goal the Quakes looked quite ragged.

And, astonishingly enough, this is the first time in many years of season tickets, that I have got seriously rained on at a Quakes match. Cold rain down my neck just about summed up this opener.

Next up: The relabeled-highjacked-still-playing-in-a-college-stadium-I-notice-new-new-Quakes.

More Security Theatre

Surely, with the California economy in freefall, the budget process hopelessly broken, and a water crisis about to cause major damage to both agriculture and fishing, Joel Anderson (R, 77th) has more important things to worry about than the fact that shock! you can see both ground-level and satellite pictures of buildings on the Internet.

But, no, one of the brilliant minds that held California's budget hostage for months and was a cheerleader for the misbegotten anti-vehicle-registration-tax has decided that his next mission is to censor Google Earth, and so is putting forward AB-255 (see http://www.cnn.com/2009/TECH/03/11/google.earth.censor.california/).

The idea is to make Google blur images of "important" buildings: government buildings, schools, churches, and medical facilities. Supposedly this is to fight terrorism. How a blurry image of a hospital on Google Earth will prevent terrorists from parking a truck bomb in front of the main entrance remains unclear. How this will prevent terrorists from going to the hospital's own web site which probably has useful things like maps to help people find their way and a street-view picture of their main entrance anyway also remains unclear.

Why churches? Can't God protect His own? Anderson states that churches and synagogues have been attacked . True. I don't see any evidence that tossing a firebomb through a window was aided and abetted in any way by the presence of a picture of said building on the Internet. Indeed, a map indicating the location was probably more useful. Shall we censor that next? And then the white page listing giving the address?

Why not shopping malls? Theatres? Sporting venues? Don't people congregate there too?

It is ludicrous to suppose that this bill would prevent any terrorist attack, or that it would seriously hamper in any way the efforts of those intent on such attacks.

No, this is not about counter-terrorism at all. It is about the appearance of counter-terrorism. And what it is really about is taking a first step into censoring content on the Internet that some government busy-body finds objectionable. Anderson says that this would only be doing what other governments around the world have done. What do you want to bet that a year from now, the argument is that since we already censor pictures of government buildings, censoring information about government officials is just the next logical step in the "War on Terror"?

Fun with Statistics

I was working with my daughter on her AP Stats homework the other night. The section is all about using the normal model (the old Bell curve of infamy) based on information about proportions. Since normal distributions pop up fairly often in the real world, and proportions are a common way of expressing information about distributions, this is all good stuff with wide applicability.

It pays to be careful however. Not every distribution is a normal distribution (ages in Gaza, for example, is strongly skewed to the left) and not every sample is a reliable one. One thing I like about the presentation in this section is that all the problems require a check of a set of assumptions that should be met for the normal model to be applicable. Even better, some of the problems violate one or the other of the assumptions. Here are the conditions:

1. The randomization condition: The sample needs to be an unbiased representative selection from the population.
   
2. The 10% condition: The sample represents less that 10% of the population.
3. Success/failure conditions: The sample needs to be enough that the number of successes and failures is more than 10. So if the sample size is n and the proportion is p, then both np >= 10 and n(1-p) >= 10.

Undoubtedly the last couple of assumptions will get mapped to some stronger mathematical basis involving confidence levels and the like later on (Statistics is annoying different from the rest of mathematics, being built from the top down instead of the bottom up) but this is a nice set of rules of thumb to apply to many statistical claims floating about out there.

So, given a proportion p and a sample size n that meet these conditions, what can we do? The fundamental operations all take place by computing areas under the normal curve to give probabilities. For example, given that 13% of the population is left-handed, an auditorium with 120 seats, of which 15 have left-handed desks, what is the probability that there will not be a left-handed desk available for some poor lefty?

If we know the number of standard deviations away from the mean this (15 lefties) represents, there are standard tables and fancy calculators that give the probability that a value is less (or greater, by subtracting from 1) this value. The standard deviation is easy enough to compute: σ=√(p (1-p)/n) ≅ 0.0307

Now we want to know P(#lefties > 15) based on this model. The z-score (number of standard deviations) is computed simply as well: (p-p̂ )/σ ≅ 0.1629
Looking this up, we get the cumulative probability that a sample will be less than 15 of about 0.8580 so the probability that it will be greater is 1-0.8580. So about 14.2% of the time, some poor lefty will have to do without a left-handed desk.

All this assumes that we meet the conditions in the first place, however. So, let's look. The success/failure condition is the easiest: np = 120*0.15 =18 and n(1-p) = 120*0.85 = 102, both of which are greater than 10. Does the sample represent less than 10% of the population? Well, that depends on what you take the "population" to be. You have to be a little careful to avoid getting fooled. Is the population "all possible students everywhere"? Then the sample surely does represent less than 10% of the population. If this auditorium is for the exclusive use of the fine folks of the East Krumwich School for the Sinister Arts, maybe a better characterization of the population is "all possible students of EKSSA" and it could be that 120 is not less than 10% of the population. But in all likelihood, the 10% condition will be met, or the auditorium was a very bad investment. What about randomization? Again, it all depends. If this auditorium is at a highly selective college, then you have to ask whether the selection the college applied in accepting students biases the sample in an important way with respect to left-handedness.

The randomization condition is where the assumptions of normality can really head south. It is very easy to come up with samples that are biased in some way or other. In the recent election certain national polls were biased against younger voters because the sampling was conducted exclusively on land-lines, and a larger percentage of younger voters use cell-phones and have no land-line. Similarly, a survey of the number of people who make confidential information available over the web by looking at the number of FaceBook users who post confidential information is useless: people who aren't going to make confidential information available over the web are less likely to use FaceBook in the first place.

The take-home message here is, whenever you see some statement of the form "X% of Ys are Z", your very first question needs to be "how could the selection of Ys be biased?"

It is perhaps worth pointing out that there is also a fundamental assumption in applying the normal model in this way that you know what the proportion actually is. Since the usual way to come up with p is by performing some kind of statistical sampling, there is an uncomfortable circularity possible. Maybe the proportion of lefties in the world at large is 13%, but the proportion of lefties at an art school is going to be higher, because the population is different. (Or, alternatively, you can say that the selection of a sample of students in an art school auditorium is not an unbiased sampling of the population of students as a whole.)

Another interesting aspect of the normal model is that everything hinges on the variance. A distribution with greater variance is more flattened out, with larger tails; one with less variance is squished in towards the middle, with less in the tails. That is, loss of variance means reversion to the mean, and reversion to the mean means fewer extremes, and that means that the expected difference between two selections is less. At some level this is obvious (true by definition), but the implications can be interesting and are not always appreciated. Stephen Jay Gould devoted a whole book to this idea (Full House, ISBN 978-0609801406) with examples from such diverse arenas as baseball and the Cambrian explosion. The capsule summary is that in competitive arenas there is a long term tendency to reduce variance, which means that the difference between the worst and the best gets smaller, which means that you no longer see the 22-0 blow-outs in English league play that you did in the 1880s, and you can make the case that it was easier for Babe Ruth to rack up a lot of home runs than for Hank Aaron, because Babe Ruth got to play against relatively worse pitchers more of the time. The long term trend is towards mediocrity. (Although please note: the absolute mean may in fact be higher, and probably is, as there is a convergence on the techniques that get the most benefit to the limits of what is possible. So "mediocre" in the sense of "closer to the mean" not in the sense of "bad".)

I think you can make the case that this also appears to apply to the world of politics: as time goes by, the relative excellence of the candidates tends to get closer and we have fewer really bad candidates as well as fewer really outstanding candidates, and the result is closer elections. Where are the statesmen of the caliber of the founding fathers (set aside the bias from the rosy glow of the passage of time)? More unlikely to appear. On the plus side, grossly inadequate candidates are less likely as well. The only way to escape from the trap of mediocrity and small differences in excellence is to change the game and jump to a completely new distribution: take steroids, start leveraging new social media, get a fancy high-tech swimsuit.

Help! Atheists Ate My Planet

One of the interesting things about having an eclectic book habit is that sometimes you read two very different books one after another and find they share some unexpected common thread. The two books in question are China Road (ISBN 978-0-8129-7524-6), in which Rob Gifford makes one last road trip through China, debating with himself whether China is about to take over the world or collapse into chaos, and Mythic Ireland (ISBN 0-500-27872-5), in which Michael Dames takes a scholarly look at the levels of Irish mythmaking.

What on earth can these two books have to do with one another?

The blog title gives it away: both lament the state of the environment and blame atheism for the carnage. This is a bit rich coming from a couple of apparent Christians, whose own holy book contains the exhortations to run roughshod over the planet. Christianity sets up an intrinsic animosity between people and the planet, but only the End-Timer extremists are up-front about it. Still, the fact that the most insistent anti-environmentalists, in the United States at least, are also the most dogmatic Christians can hardly be taken as an accident. The Christian/anti-environmental link is old ground, and well plowed elsewhere.

Dames makes the interesting observation that Ireland has present and active in its culture several distinct overlays arriving in successive waves of migration: Mesolithic (hunter-gatherer) and Neolithic (farmer) goddess worship, Bronze and Iron Age Indo-European sky gods, and Christianity. The Irish tendency to repurpose ancient deities as saints (e.g. Brigit, goddess of the flame) is well known: the bulk of the book is an extended and detailed observation that this repurposing has been going on for a very long time, and that some of the very old (7000-2000 BC) traditions still peek through. All well and good, but then Dames tries to make the claim that "Scientific Rationalism" is just another mythic overlay.

The recent appreciation by Popper and others that 'scientific discovery is akin to explanatory story-telling, to myth-making, and to the poetic imagination' places science in the broad mythic field where, in any case, it was born and nurtured.

Hmm...Right there on page 10, you start wondering just how much nonsense you're going to have to put up with. Well, quite a bit, it turns out, although mainly in small enough doses to keep you going. (Dames has a penchant for listing out every possible ancilliary meaning of a word root to prove mythic points.) Where he really jumps the shark, however, is in trying to make the case that it is the atheistic view of scientific rationalism that is degrading the environment. Dames undermines his own case (in typically overblown BS-laden terms):

Since in all mythologies (including the Christian) the individual is a cosmos-in-miniature, a person divided in two implies an equivalent an equivalent split in the macrocosm. Thus the spirit or soul which previously had been united with the body of the world was driven out, leaving behind a soulless mass of gross matter. Animals, plants, rocks, rivers, clouds, stars, sun and moon, were all now denied their share of the divine vitality. No longer were they regarded as living manifestations of the gods, but as a barrier blocking the view of the abstract and immaterial Godhead. This newly despicable world was repeatedly put to the torch at Lough Derg, and is incinerated again at Hiroshima, and in human-induced global warming. The pilgrimage to Ferry House is an Irish version of the Platonic European walk, where the world, as soulless Thing, is treated with contempt.

This may read like an indictment of Christian ethics and a pining for Neolithic morality, but no, this is not Dames' point at all. He pins dualism on the Greeks and the "mythology" of scientific rationalism. "In time, Christianity incorporated this teaching," he acknowledges (as he must since it is undeniable), but fails to clearly see that separating the divine from nature is a Christian idea, not a Greek one. The distiction between mind and body, spirit and flesh, is a part of the older traditions too. It is Christianity that brought contempt for the non-human parts of creation, not Plato, and not the scientific rationalism. Let's just set aside completely the non-sequitur of equating science with atheism, atheism with amorality, and amorality with immorality.

In the other corner, Rob Gifford observes the staggering environmental cost of China's development, and again, tries to pin it all on those darned atheists. It is a mess of faulty correlations and (again, sigh I do so get tired of this) the equation of atheism with amorality and immorality, as if they were all the same thing. He finds the blossoming of Christianity and other faiths in China a hopeful sign, but if we want to go for faulty correlations, how about this one: the acceleration of environmental degradation in China correlates with the positive uptake of Christianity. If Chinese are lost for ethical underpinnings, is it because they were raised to praise atheism, or because the Chinese government has made a systematic program of tearing down ethical frameworks one after another over the past fifty years, so that the only sane response is skepticism to any ethical pronouncement from authorities? Gifford runs through the program in detail, and then concludes that Chinese no longer believe anything and therefore (hoo, that's quite a therefore!) their atheism provides their justification for doing anything -- lousy health and safety rules, lousy environmental protection, lousy consumer protections, you name it.

None of it follows, of course.

Why do you have to believe in sky fairies to want to have a clean place for your grandchildren to live? Why does the fundamentally rational position that you can't outstrip your natural resources indefinitely need a supernatural justification for action?

Short answer: you don't.

Programming in XML Aware Languages

There is a lively debate going on over at xml-dev about whether XML applications should (or can) be programmed entirely in XML-aware languages such as XQuery and XSLT. As is usual in such matters, "performance" is being called on to fight a proxy war for religious ideals. All the usual positions are being trotted out "higher level language can't do lower level things "efficiently" (although in this case, amusingly enough, there are partizans on both sides claiming their favourite is "higher"), and the old "it depends on what your application is doing" chestnut, to say nothing of "if it's Turing complete, it can do anything" crossing swords with "just because it is theoretically possible doesn't make it efficient". All so very tiresome and predictable. We have been here before, boys and girls, many many many times, all the way back to the days of FORTRAN versus assembly (and indeed soft programming versus hard programming before that).

In the interests of full disclosure I should confess that I work for a purveyor of an XQuery implementation, so you might suppose I have a dog in this hunt. I don't really. I have seen the dark side of both extremes. I have seen the "forget the XML: just get me my objects, quick" meme lead to much wasted effort in trying to get a data-binding layer to stop being the bottleneck of the application and the focus of disproportionate amounts of development effort or expense in evaluating, integrating, and cursing third-party tools. I have seen this several times, in fact, and each time it made me crazy with frustration to spend so much effort on something so utterly pointless to the end goal. One project went so far as to pickle XML into binary object structures for storage, with lots of special code to handle cross-platform incompatibilities in that binary structure, which was then rehydrated as pure XML for shipment to the business logic layer of the application. That said, I have also seen the dark side of the "XML is so cool, let's put it everywhere" meme, the god-awful DOM code to deal with what was, at the end of the day, just a handful of numeric parameter settings. Stupidity burns on both sides of this debate. (Sturgeon's law, I think.)

What I find immensely frustrating about such language wars is that the notion that "performance" is a meaningful and useful criteria to use to judge the quality of a programming language goes largely without question. As Pauli famously remarked: That's not even wrong. Performance is a characteristic of a specific implementation of a specific algorithm in the context of specific data. A bad algorithm is a bad algorithm, in whatever language it is expressed. A good algorithm is good in any language. What often gets missed is: A good algorithm applied in the wrong context is also a bad algorithm. (Folks: friends don't let friends waste a good radix sort on ordering three numbers.)

The only tenuous connection "performance" has to "language" is if the language makes it easier to produce a better implementation of an algorithm for the data at hand. Often, "language" here means "readily available, high-quality libraries".

I once saw a very concrete example of this. There was some code written in language X to do some specialized HTML parsing. It was then ported to language Y, which was, in the language ways de jour, supposedly many times slower than language X. Turns out, however, that the ported code ran several times faster. Puzzlement from the local language X partizans ensued. The reason was not too hard to fathom, however: it was just so much easier to write a better algorithm in language Y given the libraries it had on hand that we did. We could have done so in language X, certainly, but no one ever did, because it was too much effort.

Which is to say, it is human efficiency that is at stake most directly here, not computer cycles. Here it all runs into psychology and a great deal of baby-duck imprinting frequently gets in the way: I have always constructed my algorithms in such-and-such a fashion, therefore such-and-such a fashion is the better way to construct algorithms. (The prevalence of such a attitude is example #463 of why software engineering isn't actually an engineering profession. Here is another.)

Programming may be the last bastion for the Sapir-Whorf hypothesis, but with the wonderful realization that we aren't stuck with one native tongue. We get to pick which language suits the task at hand, which gets us back full circle to the debate: some applications are well-suited to being done entirely in a language with XML concerns as a primary organizing principle, and some aren't. Pick the right one for the right task. Or, as my son would say Don't Be Dumb.