MathLing

Why Sprint Sucks

2010-11-27T16:11:00.000-08:00

Once upon a time, the girl child, in performing an act of kindness and mercy for a baby crab, drenched herself, and alas, her phone, in a briny estuary. The phone made a few pathetic meeping noises from her pocket, and then spoke no more.

Various attempts were made to revive the poor beast, but it was done for, so a new phone was in order. While the nice lad at the Sprint store did heroics to pull the contact list off it, he pointed out that we might want to add in the equipment protection plan so that if something like this were to happen again, a replacement phone could be had without splashing out the primo price. Supposedly as long as it wasn't abuse, it would cover "everything": phones being dropped, bits falling off, whatever. Falling in an estuary? Sure thing. Given the girl child's track record on wearing out phones after about a year, it seemed like seven bucks a month well spent.

Time passed, and several phones, and another phone started losing its mind and hanging. Covered? No, that's software, not covered, but hey, it's OK because you're due for an upgrade.

Shortly thereafter the upgraded phone fell in some soup, and died. Covered? No, water damage not covered. No? When we *got* this plan precisely because of water damage. Nope. So nice young man was either a lying snake or an ignorant worm. Great. Fine, clone the info back to the phone that keeps losing its mind, and we'll wait out the year until upgrade time. Live and learn.

Then the trackball fell off. Covered? No. That's OK, she learned the keyboard shortcuts, waiting out the six months until upgrade.

Then the power charger socket broke so it couldn't be charged.

She talked to the nice Sprint people on line. Two of them, in fact, both of whom assured her that since I was a Sprint Premier customer and she was on my plan, she could use one of my upgrades, no problem, and they'd even put a note in the account so that she got the mail-in rebate as well. Just go to the store and they'll handle it. How helpful. She printed these conversations out.

What do you think happened when we went to the store?

Is the charge socket breaking covered? No. Of course not. Apparently, rather than "everything" being covered by the monthly tax, nothing is covered. Talk about your bait and switch, AKA lie, lie, lie.

Sprint is, of course, the master of bait and switch. They advertise their wonderful 4G network, and charge me a monthly premium because my phone is a "4G phone", but where oh where is this alleged 4G network? Anywhere I have ever been? Nope. Glad to see Visalia and Fresno are covered, though. And Sprint advertises that the phone can act as a networking hotspot, but do they tell you that it can only do that if you pony up an additional thirty a month to enable it? Naaah. That would be actual honest dealing, something Sprint is constitutionally incapable of.

OK, so can we do the upgrade thing the online guys spoke of? Let's just check....oh, sorry, no, because my line isn't due for an upgrade for another year it can't happen. So what about those on-line helpers? Apparently they were also either lying snakes or ignorant worms. But what we could do is add another line and then you'd get the discount on the phone.

OK, so if I pay to activate a line, and the extra ten bucks a month for at least the next two years (or pay $150 to cancel that line) as well as paying for a line that I don't even have a phone connected to, then, you'll actually deign to replace a broken phone that we have poured many many months of seven bucks a month of replacement plan money into, is that it? I'll get $150 rebate on a phone for the low-low cost of $240, or the even lower low cost of $160?

But you can't, heaven forfend, apply an upgrade to an account of over a decade's standing 35 days (yes, 35 lousy days) early? Or do what two of your online account reps independently said we could do? No, apparently not.

We declined that particular kind offer. We put the girl child's number onto my old phone. We cancelled the rip-off equipment non-protection plan. In 35 days, she'll get her telephonic life back.

Since we didn't actually buy a phone, we are going to even get the customer service "how do you rate us" call. So here it is, Sprint: on a scale of 1 to 10, your customer service today rates a 0. Thanks a lot. For nothing.

Italy vs Serbia

2010-10-13T08:35:00.000-07:00

It looks like yet another international football match has been marred by fan violence: the qualifier between Italy and Serbia was abandoned after seven minutes and a kick-off delayed for half-an-hour. The scenes from Genoa are not pretty, with Serbian yahoos making their nationalistic salutes, doing their best to tear up stadium barriers, and tossing incendiaries onto the pitch. It is particularly loathsome in that the pricing for the game was set to encourage families to bring their children, and many children were indeed in attendance. The disregard for human life is appalling.

The likely outcome for this is Serbia being banned from the competition at least and, so reports say, Italy awarded a 3-0 victory. The EU mavens may well be rethinking the implications of insisting on freedom-of-travel for Serbians as a condition of joining, and maybe rethinking getting Serbia to join at all. This is likely a large part of the point of the "fans" actions. They will go back home with their heads high, nursing their unfounded sense of Serbia being unfairly put upon by the rest of the world. Good riddance.

I'm all in favour of punishing the Serbian FA for the actions of their so-called fans. Still, where it the Italian responsibility here? Why should Italy be rewarded for poor crowd control? What struck me is that the scenes in the video clips I saw did not seem substantially different from what we see from the ultras in any important match involving Italian teams. The Italian FA and police have apparently long accepted fans bringing flares and fireworks into stadiums, and waving and tossing them, and have decided to solve the problem by not solving it at all: by wrapping fans in perspex cages and hoping they don't end up with a Heysel or Ibrox or Villa Park on their hands. That looks to me like an appalling disregard for human life also. Do we really have to wait for the next disaster before they act? Really? Shame on them, too.

Glottochronology

2010-01-04T07:54:00.000-08:00

A couple of papers on glottochronology recently came across my desk: Measures of lexical distance between languages and Lexical evolution rates by automated stability measure by Filippo Petroni and Maurizio Serva.

Rather than rely on human judgements of which terms are cognates, they compute normalized Levenshtein distances between pairs of words from the Swadesh list and then use another automated procedure to compute the correlations to compute phylogenetic trees. The resulting trees are comparable to those produced in the traditional way, and the papers include some analysis of the stability of the technique and such. It is interesting in that since it is a purely mechanical operation, one can grind through a lot of languages given nothing more than some basic vocabulary lists. The word lists and resulting trees are on-line for the interested. (Note: the time axis in the trees runs, annoyingly, from 0 to 5000 plus some unspecified amount of trailing years, so mapping the branch points to approximate dates requires some amount of mental arithmetic.)

There is something deeply appealing to me about applying mathematical approaches to linguistics in this way, but some note of caution is warranted as well. The drawing of phylogenetic trees (as the name should indicate) is borrowed from the use of matching techniques on DNA sequences to construct (true) phylogenetic tress. The application in genetics is more straight-forward, however, in that snakes don't invade dog-land and replace half their genes. In addition, while DNA sequences capture the essence of inherited traits in living creatures, there is a lot more to a language than 200 core words. Plus, while all living things appear to apply exactly the same rules in interpreting DNA sequences, languages have a lot of different takes on how they use phonemes to generate meaning: it isn't clear that Levenshtein difference is a great measure to use in comparing languages that use vowel variations for inflections (think: strong verbs) against those that use endings (think: weak verbs) when applied to some default form. In sum: there are a lot more features to language than words, we can't be sure that the comparison rules are consistent, and language contact and borrowing plays a role that it doesn't (outside the microbial world) in genetics.

This is not to say this is a useless enterprise, not by any means, and certainly the practitioners in this field are well aware of these issues. Indeed, the mathematical approach provides some hope of including additional language features and untangling some of the mess. The details of the comparisons can give clues to some of the impacts of contact.

Speaking of contact, Santa left me the John McWhorter's "Our Magnificent Bastard Tongue", in which McWhorter applies his work on creoles to the history of English and rails against the focus on words, pointing out that the grammar of English (and Proto-Germanic) shows some very interesting things about the history of English as a language formed in language contact, and not just by borrowing a lot of words. In particular, he points to a couple of odd-ball features of English grammar (the meaningless do, and the use of progressive for present tense) as coming from Welsh. He argues that the supposed Celtic genocide never happened (and cites some recent genetic data to back him up). Later on, the Danelaw led, not just to the adoption of some words, but by the streamlining of the grammar. Fascinating stuff. The argument looks weakest in trying to explain why written English changes at such odds with the supposed timing of the changes on the ground.

Where things turn truly speculative is when he points to some slender evidence for Semitic (Phoenician?) influence on Proto-Germanic: the loss of inflectional endings, the consonant shift, and the introduction of a lot of words that lack cognates in the rest of Indo-European. Tantalizing, for sure, but surely speculative.

Bringing us back to the top, having a fuller set of features to compare languages should be able to show us these things. Does Proto-Germanic show affinity for Phoenician? How much?

Drat! A Reason to Boot to Vista

2009-12-24T08:32:00.000-08:00

Some months back I acquired a nice practically brand-new quad-core box. The fellow I got in from was kind enough to install it as a dual-boot with Fedora and Vista.

What a pleasure it has been to to have to lug the laptop home and use it to tunnel in to work to get meaningful work done from home without worrying about whether things will fail in mysterious ways, and being able to configure my own imap server if I want to. Like all Vista machines, that laptop functions like an old VW Bug with funky quirks that don't apply to any other such machine ("reverse doesn't work, and you have to be gentle nudging it into second or it will stall"). In my case, the cursed laptop refuses to install anything except (mirable dictu, touch wood) Firefox updates.

In any event, after lo! these many months, I have never once felt anything close to the need to actually boot to Vista on my new box. Until yesterday, dang.

This is all my son's fault. He was playing with a fractal drawing tool I had installed, and wanted more. Since he mostly uses a Windows box, he went looking for something that would run on Windows. And he found a truly astonishing bit of donation-ware: Incendia. If you are remotely interested in computer art, or fractals, go check it out and toss the author some money, because he deserves it. Or check out some of the samples. Here's one my son did.

Bad news for me: this doesn't run on Linux, and really, nothing remotely close to it runs on Linux. So if I want to play too, I have to boot to Vista. Of course, it is Vista, and in this case I have to log in as administrator to run the program or suffer outrageous flicker.

When Irish Eyes Are Not Smiling

2009-11-22T11:13:00.001-08:00

France 2 Ireland 1 FIFA 0

Someone once told me that the perfect game was one that had a close score, a lot of chances, at least one switch in the lead, at least one moment of controversy, and a come-from-behind triumph.

Well, by that measure, the World Cup playoff between France and Ireland must be some kind of gem. I guess we forgot about the part "and the outcome not decided by cheating or an officiating mistake."

There has been much ink spilled, and all manner of ugly truths put about on this issue, from which many wild conclusions have followed.

Ugly truth (courtesy Roy Keane): France is going to the World Cup, get over it.

Ah Roy, politic as ever. And still burning with anger over his treatment by the FAI, apparently.

Ugly truths (Roy Keane, again): Look at how well you're developing your youth squads, Ireland: why are you even in the playoffs? Look at all the chances you didn't finish: why is the game so close that one bad call makes you lose the game? Look at how you defended the set-piece: why are you letting Henry goal-side, why are you letting the ball bounce in the box, where's my goal-keeper?

Roy is obviously a graduate of the blame-the-victim school of grief counseling. That said, he has a point. Aidy Boothroid said something similar when Attwell (may his name live in infamy forever) handed Reading a goal for a ball that never saw the inside of the posts: you should take your own chances and not put yourself in a position where one bad call makes the difference.

Ugly truth: Why is Ireland even playing France? Why did France get special seeding by FIFA changing the rules midstream? Maybe France should have been playing Portugal. (Not that the seeding helped Russia.)

FIFA deserves any black eye it gets for its continual bending of the rules to the benefit of the favoured few top teams. This seems a common failing of soccer federations: UEFA does it, the English FA does it, the MLS is a hopeless case in this regard. It all stems from a shocking misapprehension of a key attaction of the sport: the real chance that underdogs can win on any given day.

Ugly truth: What excuse do the officials have for being out of position on a set piece?

Well, they weren't out of position, actually. The linesman was down by the corner, and the ref was back outside of the area so he could get a good view of the shocking amount of pushing and pulling going on.

Ugly truth: "Everyone cheats, it is up to the refs to spot it"

The first half may be an ugly truth, although it must count as a poke in the eye to all the honest players out there, but the second half is an unwarranted conclusion.

And the most obvious ugly truth of all, courtesy Henry, trying to save his reputation: Yes, I handled the ball; I think the fair thing would be for a replay.

Yeah, right. Like that was ever going to happen. If you were so gung ho on fair play, where were you at the time?

Some have concluded from this swamp that soccer needs video replays to eliminate error.

Folks, soccer needs video replay about as much as it needs TV timeouts. The best proposal I have heard has a fifth official watching video replays who can "raise his flag" to the ref, if he thinks the officials on the field have missed anything, leaving it up to the center ref to decide whether to accept that call or not. Such a system might be quite workable, and may stop FIFA from suffering the ultimate humiliation of seeing a World Cup Final decided by cheating. My feeling is, it will neither satisfy the purists who want the game to flow without endless stoppages and revisionist decision-making, nor those who want to ensure there are no "incorrect" calls, ever. Some wrong decisions seem so trivial that it would be pointless to object in the flow, until they turn out to end up in a goal. Example: half-way line throw-in goes to the wrong team, who quickly launch the ball thirty yards upfield, forward gets behind the defender, bam! goal. (Real example, real game, by the way.) Now what? Rewind the last 15 seconds of play for a throw-in decision? Or review every throw-in decision in real time?

This game does not need to follow the NFL down the path to perdition where there is no such thing as cheating, only calculated application of specific infractions for which there is, somewhere in the 300 page rule book, a specified penalty. Do we really want soccer to become a game where you need to be advised that "if a push happens in the last twenty yards during the last 3 minutes of the game, and there are no timeouts left, the aggrieved team has an option of calling a bonus timeout, unless it is Tuesday, or a flying monkey is spotted in the stands" (or whatever the latest gerrymandered NFL rule is, I can't keep 'em straight). I say not.

What this game needs is a dose of sportsmanship. Stop calling cheating "simulation" for starters. Put some real teeth into the "ungentlemanly conduct" law -- if you're going to toss yellows about for foul language (another thing "everyone does"), how about for cheating? Any attempt to con the officials, should be dealt with very harshly. If you're going to add video-based fines and bans for fouls, how about for cheating? If Henry knew he would miss the first three games of the World Cup (which is about all France can expect to play, based on their recent form), he may not of fessed up in the heat of the moment, but at last France would suffer a realistic penalty for their actions. Or maybe: proactive red and yellow cards: you should have got a yellow in your last match but you conned the ref, fine, have one for the start of your next game.

Pull disclosure: My husband is half-Irish. Much dark muttering into Guinness followed last week's debacle.

SIGIR Day 1

2009-07-22T04:28:00.000-07:00

Opening session: Susan Dumais' Salton Award Lecture

Dumais walked through a personal history of her work. She spent some time talking about the problem of re-finding: her data shows that 50-80% of web visits are re-visits and 30-50% of queries are re-queries, which speaks to a tremendous opportunity to use that knowledge to help users more effectively. In experiments with refinding on the desktop she discovered that while queries to a search box tend to be short with few operators, if you give people a rich faceted interface, they will use it. Being able to augment the results with new criteria and resort based on those criteria is key. She also pointed out that some attributes are highly resource dependent. For example, which "date" should you use? For a meeting: the time the meeting is scheduled for; for an email: when it was sent; for a photo: when it was created; for a document: when it was last changed, etc. She then spoke about some work on personalization, where prior clicks along only got to predictions that were right 45% of the time, whereas adding in prior clicks plus dwell time plus other session information got to 75%.

On web search at 15: "There is something amazing in that you can have 20 billion pages of diverse uncurated stuff and find anything!"

Although web search is amazingly effective, many tasks are still hard, and much remains to be done. "Search in the future will look nothing like today's simple search engine interfaces. If in 10 years we are still using a rectangular box and a list of results, I should be fired." [Mar 2007]

She spoke of thinking beyond traditional IR boundaries and identified several challenges:

information dynamics: information changes a lot and people revisit pages a lot. Queries have different temporal dynamics and pages have different rates of change. Including this information in the user interfaces would be very helpful. What if changes in search results from the last time you searched were highlighted in some way? What if changes in a page you re-vsit were highlighted in some way. She gave by way of example a browser plug-in she has been experimenting with, which showed her that there was a free continental breakfast announcement added to the SIGIR page. (Which I missed, not having this fancy plug-in, drat!)
data as a critical resource: research on user dynamics depends on data such as large scale web query logs, which are hard to get, and harder to share. We can also look at operational systems as an experimental resource using A/B testing (as Amazon does, for example).

In the Q&A someone raised the poverty of the web search box, which hasn't changed visually in 25 years. She remarked: "We are not nailing search anywhere. You're right. We are training ourselves to ask questions that those resources can answer." She then went on to remark that a lot has changed behind the scenes, but transparency matters and matters a lot more for re-finding: if you give people the tools to tell you what they know about what they are looking for they will be more successful.

Classification and Clustering

"Context-Aware Query Classification"
Derek Hao Hu (Hong Kong University of Science and Technology)
presenting for a long list of authors from the university and Microsoft

The basic idea was to apply a fairly simple statistical model to use the previous query and previously clicked links from that query to predict the category for the current one, using a query classification based on some web dictionary or taxonomy such as Wikipedia. The idea is to figure out if by "bass" you are talking about the fish or the instrument, for example.

"Refined Experts: Improving Classification in Large Taxonomies"
Paul Bennett (Microsoft) and Nam Nguyen (Cornell). Paul Bennett presenting.

Interesting data points:
If you have a hierarchy of classes, using hierarchical training is (slightly) better than flat training. What this means is that if you have taxonomy that says class A contains subclasses A1, A2, and A3, the training set for A1 is things in A, with positive examples being things in A1. The point is, you don't put things from B in as negative examples of A1 to do hierarchical
training.

Significantly better is something he calls Refinement and Refined Experts.

The basic idea for Refinement is to use cross-validation to predict which test documents belong in the class, and then use both the predicted and the actual to do the training lower down.

The basic idea for Refined Experts is to add "metafeatures" that are predictions from the child classes as part of the input to feature vectors for the parent class. This means augmenting the training data for class A with metadata for A1, A2, etc.. So you train from the bottom up, and use cross-validation from the leaves to add to the training set for larger classes.

So there is a bottom up training pass followed by a top-down training pass.

The intuition behind this is early cut-off of miscategorization from the top, and that it may be easier to distinguish some things at the leaves than in the context of larger classes at the top.

The Refined Experts algorithm led to a 20-30% improvement in F1.

"Dynamicity vs. Effectiveness: Studying Online Clustering for Scatter/Gather"
Weimao Ke et al (University of North Carolina) Weimao Ke presenting

The central observation here, albeit based on a very small study, is that you can pre-compute clusters statically and then just do tree cutting to select the appropriate clusters in at run time and get results (both in terms of precision/recall and user experience) that are just as good as you do from computing the clusters dynamically from the search results and from selected clusters of search results. Given that clustering algorithms run to the O(kn) to O(n^2) (n=#documents, k=#classes), being able to run this offline could be a win. The main down-side, of course, is that this doesn't play well with a highly dynamic document set.

Web 2.0

"A Statistical Comparison of Tag and Query Logs"
Mark J. Carman (University of Lugano), Mark Baillie (University of Strathclyde), Robert Gwadera, Fabio Crestani (University of Lugano) Mark Carman presenting

Given you are interested in personalization, you need personalized judgements of relevance. You could use query logs for this, perhaps, but what if they aren't available? Perhaps you can use tag data as a proxy for that. The question is: is this really valid?

This work reported on an experiment to compare the vocabulary distribution from queries, tags, and content to see. It compared AOL query log data against tag data from Delicious and categories from Open Directory Project (DMoz).

The conclusions:
For the tags and queries for a URL:

the vocabularies are similar
term distributions are correlated but not identical
the similarity doesn't seem to depend on category or popularity
content vocabulary more similar to queries than to tags
query/tag distributions more similar to each other than to content

In the Q&A it was pointed out that given that the documents are selected based on the query terms, the closeness of query terms to document terms is unsurprising.

"Simultaneously Modeling Semantics and Structure of Threaded Discussions: A Sparse Coding Approach and Its Applications"
Chen Lin (Fudan University), Jiang-Ming Yang , Rui Cai , Xin-Jing Wang (Microsoft Research, Asia), Wei Wang (Fudan University), Lei Zhang (Microsoft Research Asia)
Chen Lin presenting

This talk was all about figuring interesting things about about threaded email discussions. The central assumption was that you had no information about who replied to whom in the thread.

A mathematical model involving post/term and topic/term matrices and applying a minimization algorithm against a function that combines structural (reply-to) and semantic (topic) terms to infer topic/post relationships. The assumptions are that a thread may have several topics, each post is approximated as a linear combination of previous posts, a post is related to only a few topics, and each post is related to only a few other posts.

This model of the semantics and structure of a thread was then tested against other techniques on slashdot and Apple discussion forums for various purposes. The first was to construct reply relationships, based on the assumption that the reply is similar to the post it is replying to. The technique did beter than the various baselines (reply to nearest, reply to root, basic document (term) similarity, etc.). It also did well for junk identification (against term frequency, SVM, and others), and for identifying the experts (by finding hubs in the graphs of replies).

The interesting thing is that the structural terms dominated, mathematically, which says that if you actually know what the reply-to structure is (which you do for both slashdot and the Apple forums, for example -- that provided the basis for evaluating the quality), you've won half the battle in solving some of these problems.

"Enhancing Cluster Labeling Using Wikipedia"
David Carmel, Haggai Roitman, Naama Zwerdling (IBM Haifa Research Lab)
David Carmel presenting

The problem here is to come up with a label for a cluster of documents. The standard technique is to identify the "important terms" associated with the cluster using various statistical techniques. The problem is that such terms may be statistically useful, but they may not actually help humans figure out what the cluster is about. Further, a good label may not even appear in the actual content.

They did a small study to look at whether an ODP label could be identified as an important term from the content and in only 15% of the categories did that term appear in the top 5. Example: "Electronics" category gets terms electron, coil, voltage, tesla, etc. -- good terms, but none good as a label.

Their solution: use Wikipedia:
Use the cluster's content to search Wikipedia, and use metadata from Wikipedia to produce labels. The overall process is simple enough: cluster, get important terms using conventional techniques, take those as a query to Wikipedia (boosting the weights of the terms that were more important), extract metadata (title, category) from top N relevant documents, and then choose the best. For judging the best categories, they found that you needed to avoid getting too many relevant documents from Wikipedia, and a ranking system based on propagating scores (based on the linkage of documents and labels and terms) worked the better than one based on average mutual information. The technique seems fairly robust in the face of noise: even at 40% noise the results were reasonable.

For final thoughts:
Wikipedia is great for the topics it covers, but it doesn't cover everything.

Similar techniques could be used for other mining tasks: word sense disambiguation, semantic relatedness of fragments, coreference resolution, multi-lingual alignment or retrieval, query expansion, entity ranking, clustering and categorization, etc. etc.

Efficiency

"Compressing Term Positions in Web Indexes"
Hao Yan, Shuai Ding, Torsten Suel (Polytechnic Institute of NYU)
Thorsten Suel presenting

The paper looked at various techniques for compressing positions in an inverted index by exploiting clustering effects in relatively short documents (web pages). It then looked at some issues with query processing with position data.

The key observation is that positions in text are clustered; they are not evenly distributed. This observation can be exploited for specialized compression schemas (cf IPC Moffat/Stuiver 1996), LLRUN, others). The problem is that these don't work so well with short documents.

A second observation is that when storing positions we already know the term frequencies and document size. Various experiments were done with various adaptive encoding techniques, such as variants of Rice coding using information about frequency and document size to divide document into buckets and determine basis of coding, with and without adaptation from bucket to bucket, using regression to modify the encodings, using Huffman coding with different tables, chosen based on various features.

Different terms behaved differently, but the compression gains from all these techniques were disappointingly modest. Maybe a completely different approach is needed: storing approximate positions, or something else. Maybe we need to arrange to avoid decompressing most position data: if only use positions on top N (for position boost), there is little difference in the results, for example.

One comment: given the large size of the collections these days, storing a few extra parameters to improve modeling is not a big deal.

"Brute Force and Indexed Approaches to Pairwise Document Similarity Comparisons with MapReduce"
Jimmy Lin (University of Maryland)

The paper looked at computing pair-wise document similarity using MapReduce and various algorithms, with various parameter settings. The data set was some Medline documents. The take-home message was that in a MapReduce environent, the cost of keeping track of intermediate data and shipping that around the network was great enough that the brute-force approach of just computing the dot products on all pairs of term vectors actually took less time, and since it was an exact result rather than an estimate (which is what the other approaches were) it gave better answers also.

On the other hand, given that the study platform was Hadoop, which was evolving as the study was going on (so that the same test on different days would give very different times), the absolute performance measures here are probably not meaningful.

"Efficiency Trade-Offs in Two-Tier Web Search Systems"
Ricardo Baeza-Yates , Vanessa Murdock (Yahoo!), Claudia Hauff (University of Twente)
Ricardo Baeza-Yates presenting

The problem context for this paper is a search system which has a small local server (cheap, fast) and a larger 2nd tier server (expensive, slow). Queries that cannot be answered by the local server must be referred to the 2nd tier server.

Approach 1: Search the local server and if that does not give you good results then refer the search to the tier 2 server
Problem: you end up waiting longer than if you sent to tier 2 in the first place.

Approach 2: Search both in parallel and then merge or merge only if you need to.
Problem: Increases load on tier 2 and so is more costly.

Solution: Predict which queries are local, and send to tier 1 only or to both in parallel, depending on prediction. The paper looked at a mathematical model for determining when this was worth doing and a sample system was OK.

Result assessor:
If you failed to predict that you needed tier 2, and you did: you have to wait while the query is sent to tier 2; the query serialized (line in approach 1)
If you predicted you needed tier 2 and you didn't, you added extra load to tier 2 but you don't have to wait for it to come back.

Predictor:
Train based on top 20 p20(20) using SVM features from literature for
pre-retrieval prediction
Post-retrieval assessment: if have enough results, then OK

You can improve this a lot by caching false negatives.

A question came up about what the caching of whole results does to this model. It isn't clear.

A Day at the Maker Faire

2009-06-04T10:03:00.000-07:00

The Maker Faire has always been one of those things that have sounded cool and we've meant to go to, but hadn't got around to. This last weekend we went up and a had a very tiring, but interesting day out. We were a little hampered by the fact that we came in through the back (Caltrain) door, and didn't get a map. It turns out we missed two whole buildings, and didn't find the good food zone until after we had eaten some fairly lame sandwiches. Still, a good time was had by all.

The best part is talking with the individuals playing around with random hacks in their garage: "how does it work?" and "what is this?" always led to interesting conversations. There's really nothing so fun as listening to someone who is excited by what they are doing explaining it to you. Since we came in the back door, the first zone we entered was the musical hack zone, which immediately sold the boy child, who engaged in some in-depth discussions of types of pickups and digital effects on electric guitars. Our favourite here was the guy with the Wii controller in his head of his guitar and the flat-panel display in the body, wired up so that particular chords have particular imagery, flinging the guitar head down gave a wah-wah effect with a coordinated visual effect. Hitting a button gave a distorted sound, and a distorted image to match. Credit where credit is due: Ben Lewry, and he'll make one for you too, apparently.

Most interesting reward for lowest expectations: A small tent, looking for all the world like a something you'd expect to find a Tarot reader in, tucked away among the 3-D cameras and glasses. Even the label wasn't particularly enticing: "holomagistics" or something of that sort. Inside the small, darkened tent was a small oscilloscope screen showing simple looped figures. Nothing too exciting, except the figures figures are 3-D. No fancy glasses. No need to stand in exactly the right spot. Amazing. If I understood correctly, this is done by reprogramming the scanning so that vertical scan lines alternate between different (computed) perspective views, and your brain performs the object fusion to give the sense of depth. Someone, somewhere, get this guy to think beyond "coordinated music and image displays" and invest.

The kinetic sculptures were engaging and beautiful (I love the displays in science museums of old brass-and-crystal machines, my favourite by far is the Fourier analysis machine in the London science museum, all gleaming spheres and dials), but my favourite art was Bulatov's metal sculptures. Cuteness points to the people with the candy-fab machine, slowly churning our sintered sugar creations."Please don't eat the sculptures!" is not a sign you get to read every day.

We did appreciate the Exploratorium's little puzzles too, though. My favourite: hollow aluminum tube, about 4 or 5 feet long, and a magnet. Drop the magnet through the tube, and it takes a suspiciously long time to fall through. This is the Exploratorium, mind, so we set aside our initial suspicions of some kind of slight-of-hand.

Somewhere late in the afternoon, we decided that we were well and truly tired out, so we sought out the Tesla coils, had a nice chat with the fellows there about their plans to build 12-story tall Tesla coils in the Nevada desert, and watched the lightning show. Even the 12 foot scaled down versions, running well below capacity, were impressive enough, and a good way to sign off the day.

Invidious comparisons

2009-04-10T07:51:00.001-07:00

What with World Cup Qualifying, the UEFA Champions League, the CONCACAF Champions League, the Premiership, MLS, and the UEFA Cup, I've been spending a lot of hours watching football the last few weeks.

While it is true that it is hard to beat the top 4 Premiership teams for skill, as a competition the Premier League has become downright boring. The fact that in any given season the question of which team will come in 5th provides the most drama and the current spark of interest in games like Man Utd vs Aston Villa is based mainly on the fact that Chelsea and Man United have been showing cracks in their invulnerability is telling. The UEFA Champions League has essentially become the Premier League with the role of whipping boy played by various French teams. Oh, what a shock! Liverpool against Chelsea. There is a charm in watching Chelsea dink their little triangles up and down the field and in watching Ronaldo perform his little steppy-steppy moves, but honestly, is there really much surprise in watching Man Utd win on a late goal? Again? Yawn.

It used to be that the Euro matches provided the same pleasure that the World Cup did: watching teams you'd barely heard of and learning about new fantastic players you'd never seen before. Given that the same dozen top teams swap around the same top players, you don't get that so much, except in some of the early rounds.

On the other hand, the World Cup qualifiers have been rather fun, especially given the plethora of new European teams to enjoy: Kazakhstan has a young and lively team that's rather fun to watch, if somewhat overmatched -- who knew? We'll miss you in the final.

Other teams we'll miss: Gosh, Man City versus Hamburg. Two teams charging up and down the field going at it hammer and tongs. There was more pure entertainment in the first half-hour than in half a season at Stamford Bridge. Holy smokes!

The biggest entertainment surprise for me has been the CONCACAF Champions League semifinals. Sure, I'd heard of Cruz Azul and Atlante, but never really seen them play. And Santos Laguna? The Puerto Rico Islanders? Not on my radar. I confess to having watched the tournament only intermittently, largely to track the team-formerly-known-as-Earthquakes. The semi-finals were fantastic fun, however (at least for a neutral). I once heard the perfect soccer match described as having at least one change in the lead, at least one great goal, and at least one moment of controversy. Both the semi-finals obliged, although it was hard to beat Atlante game for sheer drama: an extra-time (winning) penalty and 3-red-card fight 3 minutes after that. Sad as I was the Dynamo didn't make it this far, I'm looking forward to Cruz Azul vs Atlante in the final, which is something I hardly expected.

Speaking of the Dynamo, they came to town a couple weeks ago to play the team-currently-known-as-Earthquakes. It was just as well that Ching was off on World Cup duty because if he had been the one finishing chances instead of Mullen, things may have turned out much worse for the home side. (I've whined about the MLS continuing with a full program during qualification week before: I see they now partly accomodate the national team by giving their wannabe Man Utd of the US the week off. Funny how LA always gets the special treatment.) It was about a half-hour into it, about the time Mullen skied his third "this is a goal, oh wait, no it isn't" chance, that I said to my son "I don't think this is going to stay scoreless" and he said "I think once there is one goal, there will be a boatload." Truer words were never spoken: 15 minutes later it was 3-2. It is odd and scary for a Yallop team to have such a fragile looking defense. It is going to be a long season if they don't tighten that up.

Season Two of the New-New-New-Quakes

2009-03-25T07:52:00.000-07:00

Earthquakes 0 New England 1

Drat. On paper a midfield with Huckerby, Corrales, Convey, and Alvarez looks pretty hot, and there were brief flashes of brilliant football on Saturday. Mostly, however, we saw the classic weakness of the Quakes (both the new-new-new-Quakes and the new-new-Quakes under Yallop's tutelage previously), which is to say: nice approach work, lousy finishing. Or no finishing at all. Once again we revive the impassioned plea of Quake fans: "SHOOOOT!"

We also saw Huckerby spending far too much of the match completely disconnected from play out on his own on the wing. Has someone told him he's not allowed to stray 5 yards from the touchline or something? Or is the loss of the one guy on the field capable of hitting him at full sprint with a 40-yard cross-field pass (Ronnie O'Brien), put a huge hole in the team?

On the whole, the Quakes played the better football, but neither team looks particularly good, and after the goal the Quakes looked quite ragged.

And, astonishingly enough, this is the first time in many years of season tickets, that I have got seriously rained on at a Quakes match. Cold rain down my neck just about summed up this opener.

Next up: The relabeled-highjacked-still-playing-in-a-college-stadium-I-notice-new-new-Quakes.

More Security Theatre

2009-03-12T08:53:00.000-07:00

Surely, with the California economy in freefall, the budget process hopelessly broken, and a water crisis about to cause major damage to both agriculture and fishing, Joel Anderson (R, 77th) has more important things to worry about than the fact that shock! you can see both ground-level and satellite pictures of buildings on the Internet.

But, no, one of the brilliant minds that held California's budget hostage for months and was a cheerleader for the misbegotten anti-vehicle-registration-tax has decided that his next mission is to censor Google Earth, and so is putting forward AB-255 (see http://www.cnn.com/2009/TECH/03/11/google.earth.censor.california/).

The idea is to make Google blur images of "important" buildings: government buildings, schools, churches, and medical facilities. Supposedly this is to fight terrorism. How a blurry image of a hospital on Google Earth will prevent terrorists from parking a truck bomb in front of the main entrance remains unclear. How this will prevent terrorists from going to the hospital's own web site which probably has useful things like maps to help people find their way and a street-view picture of their main entrance anyway also remains unclear.

Why churches? Can't God protect His own? Anderson states that churches and synagogues have been attacked . True. I don't see any evidence that tossing a firebomb through a window was aided and abetted in any way by the presence of a picture of said building on the Internet. Indeed, a map indicating the location was probably more useful. Shall we censor that next? And then the white page listing giving the address?

Why not shopping malls? Theatres? Sporting venues? Don't people congregate there too?

It is ludicrous to suppose that this bill would prevent any terrorist attack, or that it would seriously hamper in any way the efforts of those intent on such attacks.

No, this is not about counter-terrorism at all. It is about the appearance of counter-terrorism. And what it is really about is taking a first step into censoring content on the Internet that some government busy-body finds objectionable. Anderson says that this would only be doing what other governments around the world have done. What do you want to bet that a year from now, the argument is that since we already censor pictures of government buildings, censoring information about government officials is just the next logical step in the "War on Terror"?

Fun with Statistics

2009-01-07T21:51:00.000-08:00

I was working with my daughter on her AP Stats homework the other night. The section is all about using the normal model (the old Bell curve of infamy) based on information about proportions. Since normal distributions pop up fairly often in the real world, and proportions are a common way of expressing information about distributions, this is all good stuff with wide applicability.

It pays to be careful however. Not every distribution is a normal distribution (ages in Gaza, for example, is strongly skewed to the left) and not every sample is a reliable one. One thing I like about the presentation in this section is that all the problems require a check of a set of assumptions that should be met for the normal model to be applicable. Even better, some of the problems violate one or the other of the assumptions. Here are the conditions:

The randomization condition: The sample needs to be an unbiased representative selection from the population.
The 10% condition: The sample represents less that 10% of the population.
Success/failure conditions: The sample needs to be enough that the number of successes and failures is more than 10. So if the sample size is n and the proportion is p, then both np >= 10 and n(1-p) >= 10.

Undoubtedly the last couple of assumptions will get mapped to some stronger mathematical basis involving confidence levels and the like later on (Statistics is annoying different from the rest of mathematics, being built from the top down instead of the bottom up) but this is a nice set of rules of thumb to apply to many statistical claims floating about out there.

So, given a proportion p and a sample size n that meet these conditions, what can we do? The fundamental operations all take place by computing areas under the normal curve to give probabilities. For example, given that 13% of the population is left-handed, an auditorium with 120 seats, of which 15 have left-handed desks, what is the probability that there will not be a left-handed desk available for some poor lefty?

If we know the number of standard deviations away from the mean this (15 lefties) represents, there are standard tables and fancy calculators that give the probability that a value is less (or greater, by subtracting from 1) this value. The standard deviation is easy enough to compute: σ=√(p (1-p)/n) ≅ 0.0307

Now we want to know P(#lefties > 15) based on this model. The z-score (number of standard deviations) is computed simply as well: (p-p̂ )/σ ≅ 0.1629
Looking this up, we get the cumulative probability that a sample will be less than 15 of about 0.8580 so the probability that it will be greater is 1-0.8580. So about 14.2% of the time, some poor lefty will have to do without a left-handed desk.

All this assumes that we meet the conditions in the first place, however. So, let's look. The success/failure condition is the easiest: np = 120*0.15 =18 and n(1-p) = 120*0.85 = 102, both of which are greater than 10. Does the sample represent less than 10% of the population? Well, that depends on what you take the "population" to be. You have to be a little careful to avoid getting fooled. Is the population "all possible students everywhere"? Then the sample surely does represent less than 10% of the population. If this auditorium is for the exclusive use of the fine folks of the East Krumwich School for the Sinister Arts, maybe a better characterization of the population is "all possible students of EKSSA" and it could be that 120 is not less than 10% of the population. But in all likelihood, the 10% condition will be met, or the auditorium was a very bad investment. What about randomization? Again, it all depends. If this auditorium is at a highly selective college, then you have to ask whether the selection the college applied in accepting students biases the sample in an important way with respect to left-handedness.

The randomization condition is where the assumptions of normality can really head south. It is very easy to come up with samples that are biased in some way or other. In the recent election certain national polls were biased against younger voters because the sampling was conducted exclusively on land-lines, and a larger percentage of younger voters use cell-phones and have no land-line. Similarly, a survey of the number of people who make confidential information available over the web by looking at the number of FaceBook users who post confidential information is useless: people who aren't going to make confidential information available over the web are less likely to use FaceBook in the first place.

The take-home message here is, whenever you see some statement of the form "X% of Ys are Z", your very first question needs to be "how could the selection of Ys be biased?"

It is perhaps worth pointing out that there is also a fundamental assumption in applying the normal model in this way that you know what the proportion actually is. Since the usual way to come up with p is by performing some kind of statistical sampling, there is an uncomfortable circularity possible. Maybe the proportion of lefties in the world at large is 13%, but the proportion of lefties at an art school is going to be higher, because the population is different. (Or, alternatively, you can say that the selection of a sample of students in an art school auditorium is not an unbiased sampling of the population of students as a whole.)

Another interesting aspect of the normal model is that everything hinges on the variance. A distribution with greater variance is more flattened out, with larger tails; one with less variance is squished in towards the middle, with less in the tails. That is, loss of variance means reversion to the mean, and reversion to the mean means fewer extremes, and that means that the expected difference between two selections is less. At some level this is obvious (true by definition), but the implications can be interesting and are not always appreciated. Stephen Jay Gould devoted a whole book to this idea (Full House, ISBN 978-0609801406) with examples from such diverse arenas as baseball and the Cambrian explosion. The capsule summary is that in competitive arenas there is a long term tendency to reduce variance, which means that the difference between the worst and the best gets smaller, which means that you no longer see the 22-0 blow-outs in English league play that you did in the 1880s, and you can make the case that it was easier for Babe Ruth to rack up a lot of home runs than for Hank Aaron, because Babe Ruth got to play against relatively worse pitchers more of the time. The long term trend is towards mediocrity. (Although please note: the absolute mean may in fact be higher, and probably is, as there is a convergence on the techniques that get the most benefit to the limits of what is possible. So "mediocre" in the sense of "closer to the mean" not in the sense of "bad".)

I think you can make the case that this also appears to apply to the world of politics: as time goes by, the relative excellence of the candidates tends to get closer and we have fewer really bad candidates as well as fewer really outstanding candidates, and the result is closer elections. Where are the statesmen of the caliber of the founding fathers (set aside the bias from the rosy glow of the passage of time)? More unlikely to appear. On the plus side, grossly inadequate candidates are less likely as well. The only way to escape from the trap of mediocrity and small differences in excellence is to change the game and jump to a completely new distribution: take steroids, start leveraging new social media, get a fancy high-tech swimsuit.

Help! Atheists Ate My Planet

2008-12-23T07:55:00.000-08:00

One of the interesting things about having an eclectic book habit is that sometimes you read two very different books one after another and find they share some unexpected common thread. The two books in question are China Road (ISBN 978-0-8129-7524-6), in which Rob Gifford makes one last road trip through China, debating with himself whether China is about to take over the world or collapse into chaos, and Mythic Ireland (ISBN 0-500-27872-5), in which Michael Dames takes a scholarly look at the levels of Irish mythmaking.

What on earth can these two books have to do with one another?

The blog title gives it away: both lament the state of the environment and blame atheism for the carnage. This is a bit rich coming from a couple of apparent Christians, whose own holy book contains the exhortations to run roughshod over the planet. Christianity sets up an intrinsic animosity between people and the planet, but only the End-Timer extremists are up-front about it. Still, the fact that the most insistent anti-environmentalists, in the United States at least, are also the most dogmatic Christians can hardly be taken as an accident. The Christian/anti-environmental link is old ground, and well plowed elsewhere.

Dames makes the interesting observation that Ireland has present and active in its culture several distinct overlays arriving in successive waves of migration: Mesolithic (hunter-gatherer) and Neolithic (farmer) goddess worship, Bronze and Iron Age Indo-European sky gods, and Christianity. The Irish tendency to repurpose ancient deities as saints (e.g. Brigit, goddess of the flame) is well known: the bulk of the book is an extended and detailed observation that this repurposing has been going on for a very long time, and that some of the very old (7000-2000 BC) traditions still peek through. All well and good, but then Dames tries to make the claim that "Scientific Rationalism" is just another mythic overlay.

The recent appreciation by Popper and others that 'scientific discovery is akin to explanatory story-telling, to myth-making, and to the poetic imagination' places science in the broad mythic field where, in any case, it was born and nurtured.

Hmm...Right there on page 10, you start wondering just how much nonsense you're going to have to put up with. Well, quite a bit, it turns out, although mainly in small enough doses to keep you going. (Dames has a penchant for listing out every possible ancilliary meaning of a word root to prove mythic points.) Where he really jumps the shark, however, is in trying to make the case that it is the atheistic view of scientific rationalism that is degrading the environment. Dames undermines his own case (in typically overblown BS-laden terms):

Since in all mythologies (including the Christian) the individual is a cosmos-in-miniature, a person divided in two implies an equivalent an equivalent split in the macrocosm. Thus the spirit or soul which previously had been united with the body of the world was driven out, leaving behind a soulless mass of gross matter. Animals, plants, rocks, rivers, clouds, stars, sun and moon, were all now denied their share of the divine vitality. No longer were they regarded as living manifestations of the gods, but as a barrier blocking the view of the abstract and immaterial Godhead. This newly despicable world was repeatedly put to the torch at Lough Derg, and is incinerated again at Hiroshima, and in human-induced global warming. The pilgrimage to Ferry House is an Irish version of the Platonic European walk, where the world, as soulless Thing, is treated with contempt.

This may read like an indictment of Christian ethics and a pining for Neolithic morality, but no, this is not Dames' point at all. He pins dualism on the Greeks and the "mythology" of scientific rationalism. "In time, Christianity incorporated this teaching," he acknowledges (as he must since it is undeniable), but fails to clearly see that separating the divine from nature is a Christian idea, not a Greek one. The distiction between mind and body, spirit and flesh, is a part of the older traditions too. It is Christianity that brought contempt for the non-human parts of creation, not Plato, and not the scientific rationalism. Let's just set aside completely the non-sequitur of equating science with atheism, atheism with amorality, and amorality with immorality.

In the other corner, Rob Gifford observes the staggering environmental cost of China's development, and again, tries to pin it all on those darned atheists. It is a mess of faulty correlations and (again, sigh I do so get tired of this) the equation of atheism with amorality and immorality, as if they were all the same thing. He finds the blossoming of Christianity and other faiths in China a hopeful sign, but if we want to go for faulty correlations, how about this one: the acceleration of environmental degradation in China correlates with the positive uptake of Christianity. If Chinese are lost for ethical underpinnings, is it because they were raised to praise atheism, or because the Chinese government has made a systematic program of tearing down ethical frameworks one after another over the past fifty years, so that the only sane response is skepticism to any ethical pronouncement from authorities? Gifford runs through the program in detail, and then concludes that Chinese no longer believe anything and therefore (hoo, that's quite a therefore!) their atheism provides their justification for doing anything -- lousy health and safety rules, lousy environmental protection, lousy consumer protections, you name it.

None of it follows, of course.

Why do you have to believe in sky fairies to want to have a clean place for your grandchildren to live? Why does the fundamentally rational position that you can't outstrip your natural resources indefinitely need a supernatural justification for action?

Short answer: you don't.

Programming in XML Aware Languages

2008-12-03T08:19:00.000-08:00

There is a lively debate going on over at xml-dev about whether XML applications should (or can) be programmed entirely in XML-aware languages such as XQuery and XSLT. As is usual in such matters, "performance" is being called on to fight a proxy war for religious ideals. All the usual positions are being trotted out "higher level language can't do lower level things "efficiently" (although in this case, amusingly enough, there are partizans on both sides claiming their favourite is "higher"), and the old "it depends on what your application is doing" chestnut, to say nothing of "if it's Turing complete, it can do anything" crossing swords with "just because it is theoretically possible doesn't make it efficient". All so very tiresome and predictable. We have been here before, boys and girls, many many many times, all the way back to the days of FORTRAN versus assembly (and indeed soft programming versus hard programming before that).

In the interests of full disclosure I should confess that I work for a purveyor of an XQuery implementation, so you might suppose I have a dog in this hunt. I don't really. I have seen the dark side of both extremes. I have seen the "forget the XML: just get me my objects, quick" meme lead to much wasted effort in trying to get a data-binding layer to stop being the bottleneck of the application and the focus of disproportionate amounts of development effort or expense in evaluating, integrating, and cursing third-party tools. I have seen this several times, in fact, and each time it made me crazy with frustration to spend so much effort on something so utterly pointless to the end goal. One project went so far as to pickle XML into binary object structures for storage, with lots of special code to handle cross-platform incompatibilities in that binary structure, which was then rehydrated as pure XML for shipment to the business logic layer of the application. That said, I have also seen the dark side of the "XML is so cool, let's put it everywhere" meme, the god-awful DOM code to deal with what was, at the end of the day, just a handful of numeric parameter settings. Stupidity burns on both sides of this debate. (Sturgeon's law, I think.)

What I find immensely frustrating about such language wars is that the notion that "performance" is a meaningful and useful criteria to use to judge the quality of a programming language goes largely without question. As Pauli famously remarked: That's not even wrong. Performance is a characteristic of a specific implementation of a specific algorithm in the context of specific data. A bad algorithm is a bad algorithm, in whatever language it is expressed. A good algorithm is good in any language. What often gets missed is: A good algorithm applied in the wrong context is also a bad algorithm. (Folks: friends don't let friends waste a good radix sort on ordering three numbers.)

The only tenuous connection "performance" has to "language" is if the language makes it easier to produce a better implementation of an algorithm for the data at hand. Often, "language" here means "readily available, high-quality libraries".

I once saw a very concrete example of this. There was some code written in language X to do some specialized HTML parsing. It was then ported to language Y, which was, in the language ways de jour, supposedly many times slower than language X. Turns out, however, that the ported code ran several times faster. Puzzlement from the local language X partizans ensued. The reason was not too hard to fathom, however: it was just so much easier to write a better algorithm in language Y given the libraries it had on hand that we did. We could have done so in language X, certainly, but no one ever did, because it was too much effort.

Which is to say, it is human efficiency that is at stake most directly here, not computer cycles. Here it all runs into psychology and a great deal of baby-duck imprinting frequently gets in the way: I have always constructed my algorithms in such-and-such a fashion, therefore such-and-such a fashion is the better way to construct algorithms. (The prevalence of such a attitude is example #463 of why software engineering isn't actually an engineering profession. Here is another.)

Programming may be the last bastion for the Sapir-Whorf hypothesis, but with the wonderful realization that we aren't stuck with one native tongue. We get to pick which language suits the task at hand, which gets us back full circle to the debate: some applications are well-suited to being done entirely in a language with XML concerns as a primary organizing principle, and some aren't. Pick the right one for the right task. Or, as my son would say Don't Be Dumb.

The Great Vegetarian Experiment

2008-11-25T08:42:00.000-08:00

For one of my daughter's classes she has a project to work on one of the "five pillars of health" (is it just me, or does this sound vaguely cultist?) for six weeks. She chose to go without meat for that period, and we decided to be supportive and work on our own "pillars of health" at the same time. We were all worried about whether the "the only good bean is hummus" boy child would make it though, but he signed on, and stuck with PBJs instead of ham sandwiches in his lunch.

So how did it go?

Success: Finally a Pad Thai recipe worth making

I have tried before to make Pad Thai, and the results have always been disappointing. But this vegetarian one from epicurious is really good. Tip: use 1 to 2 T of tamarind paste from a jar instead of mucking about with blocks and pods. But don't skip out on the tamarind entirely: it adds an essential kind of tanginess.

Failure: Life without chicken stock

From the start it was agreed that this was not to be a vegan experiment: dairy and eggs were on the table. Early on we realized that life without chicken stock is too awful to contemplate. We usually have several jars of home-made stock about the place, as well as fish stock in the freezer, and some home-made beef stock intermittently. Yes, you can make vegetable stock (a good corn stock being a great base for chile and chowder), but for most soups, chicken stock turned out to have some essential quality we couldn't do without.

A good cookbook goes a long way

Mollie Katzen's Moosewood Cookbook has lots of excellent vegetarian recipes in it. The vegetarian moussaka and gado gado were big hits. We also leaned on The Greek Vegetarian (spanakopita, some nice salads) and various Indian cookbooks we had lying around the house
(Madhur Jaffrey is my favourite: we have several of her books).

Simple things

Some of the easiest things turned out the best: fondue, cheese/fruit plates, a simple pesto/tomato/olive pasta sauce, veggie burgers, quiche, corn/black bean burritos. Higher on the effort scale, but not that complex: spinach/mushroom/onion calzones, butternut squash raviolis with sage butter, savory crepes (the mushroom with crème fraîche was best).

Biggest surprises

We usually do all our shopping and most of our cooking on Saturday, with the rest of the week being mostly a matter of simple side dishes and reheating tupperware. The biggest surprise (and biggest impediment to doing this on an ongoing basis) turned out to be that a lot of vegetarian recipes just don't lend themselves well to being pre-prepared in this way. Towards the end of the week there would either be a rather disappointing result or a lot of mad scrambling and late dinners. I did not expect this in the least. I expect that more experience and expertise would go a long way to helping here. We've had a lot of years of honing down our collection of (mostly non-vegetarian) recipes to a set that work well with this kind of process; we just haven't got there with the purely vegetarian results. On the other hand, a hunk of beef holds up uncooked better in the fridge (or freezer) for a week than many vegetables.

We don't use a lot of bacon/prosciutto/ham in our cooking, but boy, that little bit makes all the difference in a lot of recipes. Bean soups, pasta sauces, spinach salad, and quiche really lose something without it. Given how little (relatively speaking) we use, I was really surprised that this was the only meat I really missed.

True confessions: Sneaking out for a steak

On the days my daughter was away at dinner, and on our dinner-out Fridays, the rest of us frequently went for the carnivore's fare. But dang! I really like those chicken rajas at La Fiesta.

Final thoughts

Are we about to abandon our omnivorous ways and go vegetarian? Nope. On the other hand, we did discover some great new recipes, and are happy to fold them into our rotation. We have also discovered a willingness (even from the vegetable-hating teen boy) to try more things. All in all, I'd call it a success.

We Were Not Asked to Amend the Dictionary

2008-11-12T07:57:00.001-08:00

I try to understand the Proposition 8 supporters, I really do. I have read hundreds of blog postings and comments and letters to the editors, and nope, I still don't get it. The recent claims that somehow folks who are pointing out that Prop 8 was all about bigotry and hate is somehow hateful and bigoted still makes me choke on my tea.

Here is my summary of the arguments in favour of this horrid example of the tyranny of the majority:

"It isn't about civil rights; it is about the definition of the word 'marriage'."
The folks who make this claim often follow this with statements about how they support civil unions and giving gays all the same rights and privileges of marriage, but they just don't want the word 'marriage' applied to it.
Response: We were voting on an amendment to the Constitution of the State of California, not an amendment to Webster's Dictionary. The constitution does not concern itself definitions of words that have no impact on the law. Words such as 'marriage' and 'spouse' and 'parent' are scattered about in many places in the legal code. If you really believe that legally redefining the word 'marriage' has no impact on actual rights of actual people, let's try a little thought experiment, shall we? Let's imagine that the vote was to define 'marriage' to only apply to a pair of Zoroastrians. If it is just a word, then why are you so exercised about it?

"Gay rights aren't about civil rights."
For some reason, some blacks get really hot under the collar about any other group talking about wanting equal rights before the law as "civil rights". It isn't a rational reaction, so there's not really a rational counter-argument. There is nothing in the term "civil rights" that says "black rights" and nothing that says it only pertains to sad treatment of black people in this country. There is also the sense, in some circles, that somehow gays "choose" their "lifestyle" so their quest for equality before the law somehow doesn't count. I have never met a gay person who felt it was a choice. In fact, I never met anyone who felt that who they fell in love with was a "choice". You love who you love.

"Radical judges overturned the will of the people."
Our constitutional system was designed precisely to protect unpopular minorities from the majority. It is the job of judges to ensure the law and rights are applied equally to all. If a simple majority can vote to take away rights from one unpopular minority, why not another? Would it be OK to pass a majority vote to forbid atheists from voting? No. Equal protection under the law means equal protection under the law.

"We're just trying to protect traditional marriage."
The folks who make this claim seem to have a very limited view of tradition. Whose tradition did you have in mind? There seems to be quite a tradition of multiple wives, historically and cross-culturally. There is also a fine old tradition of marrying girls barely out of puberty. No, not that one? Saying "the Christian" tradition, doesn't help there. King David? Jacob, Leah, and Rachel? But even if you accept the premise that a marriage between two adults, one of each sex, as traditional: just exactly how does outlawing other people's marriages "protect" such an institution? If marriage is truly such a foundational social and economic building block of our society, we ought to be encouraging gay marriages.

"Churches would be forced to marry homosexuals."
What a stupid idea. Are Catholic churches forced to marry Moslems? No. Why on earth would any rational being suppose that suddenly the state would force rules on the religious rituals performed by churches? Even if someone were to suggest such a thing (and why would they?), it would be such an obvious violation of the US Constitution that it would be shot down instantly (by those 'activist judges').

"Homosexuality is against God's will."
There are, of course, a lot of hidden assumptions and unwarranted conclusions buried in here. Which God did you have in mind? Zeus doesn't seem to have a lot of problems with it. How do you know it is against God's will? Well, OK, Leviticus says it is an abomination and unclean for a man to lie with a man (so...lesbian marriage is OK, right?). But here's the thing: Leviticus says a lot of things about what is unclean: shellfish, bugs (except great locusts), a woman who has just had a baby, skin diseases of certain sorts. (Oh, and here's a laugh: Jacob's marriage -- right out, they're sisters.) It says many things about what is forbidden: eating fruit from a tree planted less than three years ago, eating blood-rare steak, holding a seance. So where is the proposition banning those? Where are the pickets against Red Lobster and the Outback? It also says you shall not oppress your neighbour, but instead love him. So... that part of the Bible, not so important for you? And just because something is written in the Bible, how do you know that makes it God's will? Because the Bible says so? That doesn't even count as an argument, never mind a convincing one. But even if I set aside those objections, the fundamental problem remains: the California Constitution is not a religious tract, California is not a theocracy, and your religious doctrines have no business being enshrined in our law. We are not all Christians here. Making a religious argument for a legal question is a category error. You don't want to marry gays in your church? OK. Done. That has nothing to do with what the law should say.

"It's Adam and Eve, not Adam and Steve."
This is bumper sticker, not an argument. If I hadn't seen it repeated so often, I'd dismiss it out of hand, it is that dumb. But, let's go through it. (1) California is not a theocracy, much less a Christian theocracy. (2) Even if it were, the fact that the first people were a man and a woman has no bearing on the question of marriage. One could just as well say "It's Adam, not Eve" and use that to somehow claim that we should exterminate all women in the world. (3) The Bible isn't exactly clear on the Adam and Eve thing, anyhow: how long were those Land of Nod folks around, anyhow? (4) The Bible is wrong. Sorry, I'm not going to sugarcoat this in politically correct "let's be especially nice to dumb ideas just because they are religious." So: incorrect on the facts, irrelevant to the question, and a category error.

"It's icky. It makes me uncomfortable."
Response: Grow up. Eating meat is icky to some people. Childbirth is icky. Making compost is icky. Your personal squeamishness is not a sound basis for public policy.

"What's the point? They can't have children."
This one just makes me angry. By this reasoning, all post-menopausal marriages should be outlawed, all post-vasectomy marriages should be outlawed, and probably all marriages involving contraception should be outlawed. And just because you can't see the point, what gives you the right to judge whether that couple has a reason to get married? There are plenty of reasons to want to get married, and children is just one of them. Reproductive plans of the married couple are none of your business and none of the state's business. But the fact is, gay couples can and do have children via adoption and artificial insemination. Forbidding those children's parents to be married just leaves them vulnerable. Yes: it harms children.

"It's harmful for children."
Well, given that the effect of the ban is harmful to the children of gay couples, I'd expect to see some really compelling evidence that the mere presence of gay marriages harms children by the folks making this claim. But there is none. Canada has had gay marriage for some years now, and there is no evidence anywhere that the children of Canada are suffering thereby. Tied up in this is a confusion between pedophilia and homosexuality, which have nothing to do with one another. You want to protect children from sexual abuse? Look in the mirror, Catholic church.

"Where will it all end? Next thing you know we'll have multiple wives and people marrying their pets!"
Whew. Finally finally something resembling an actual rational argument, although not, in fact, a very strong one. Slippery slope arguments always assume that there can be no rational reason to draw a boundary at all, once you decide against the current boundary. But in this case, it isn't so. There are rational reasons for disallowing polygamy: in practice it leads to coercive marriages involving young girls, maltreatment of young boys (aka rivals), and using additional marriages as a means to publish and control existing wives. There are rational reasons for disallowing "marriage" to pets: a non-sentient being cannot enter into a legal agreement willingly. Where these reasons differ from the reasons put forward for outlawing gay marriage is that they are civil arguments, not religious ones, and therefore actually have a valid place in the consideration of public policy.

All in all, Prop 8 was a scurrilous attack on not just civil rights but human rights, our constitutional principles, and families, and the arguments in favour of it are all absurd. I'm ashamed of my state, and I'm angry. There's a part of me that wants to go picket the Mormons and insist that if the Roman Catholic Church wants to inject its peculiar brand of bigotry into our Constitution then maybe its time to start paying some taxes in our state too. Fighting hate with hate is not a way forward, but the folks who are now saying that "the people have spoken" and "get over it" need to understand that when you vote to remove rights that people already have, the anger isn't going to go away. This is not just another defeat at the polls. I have some confidence that the California Supreme Court will do the right thing and recognize that a majority vote to remove fundamental human rights is an illegitimate act under our Constitution. We'll see.

Recent Reading

2008-11-11T08:06:00.000-08:00

Charles de Lint, Moonlight and Vines
ISBN: 978-0765309174

I admire Charles de Lint's writing immensely. As a prose stylist, you could do worse. Far worse. He has a wonderful eye for detail in drawing his settings and people, and a deft touch in drawing together the natural, the urban, and the supernatural. His characters are interesting, the kind of people you wished you were sitting down in a coffee shop with. Still, there was something dissatisfying about this collection of interlinked short stories. It is like listening to some of Glenn Gould's records: after the zillionth etude your mind rebels and craves either something really different or some larger overarching structure. Something with the size of a symphony shouldn't should like the same little thing over and over. This book strikes me the same way. Each story was fine by itself, and many were very fine indeed, but the interlinked characters leads you into wanting something more akin to the structure of a novel, and it just isn't there. De Lint attempts to draw the book together by making the last story circle back to the first, but it is very unconvincing. The last story isn't, fundamentally, a story. It reads like a 2am confessional from the author. The characters in it bear little relation to the ones introduced in the first story. Still, worth reading. Just read it in pieces.

Barbara Robinette Moss, Change Me Into Zeus's Daughter, A Memoir
ISBN: 978-0743202190

This is eight tenths of a fantastic book. Moss recalls her appallingly poor childhood as one of seven children of an alcoholic and violent father. What makes it only a partial success is that Moss writes towards an ending where she gets her terrible teeth fixed and emerges swam-like into her new beautiful reality. But the suffering because of her teeth isn't the central thrust of the narrative, we don't get any of the payoff of the emergence, and honestly, the most interesting thread of this story is her mother. Here is a woman who has records with her own voice recording on, who has a love of learning and reading and poetry which she somehow manages to pass on to her children under the most appalling circumstances, who holds the kids together as best she can, and sticks with an terrible husband through thin and thinner, but who finally, finally breaks free. And we get that story only in glimpes. The moment when Moss' father, now remarried in a more stable (emotionally and financially) environment refuses to help with her educational fees is set up as some kind of key emotional moment, but it looks whiny. What did you expect? The moment that we didn't get a good view in was when Moss' mother finally leaves the bum. One of the most heart-rending moments for me, in part because it is told with such flat affect, was when the children destroy the mother's records by tossing them around the yard like frisbees when they are to be left behind in yet another forced-out-again move. What must she have been feeling at that moment? How does hope die? How, after all that, do you finally decide enough is enough? Why? All unanswered, alas. Still, a good and moving read.