Why Sprint Sucks

Once upon a time, the girl child, in performing an act of kindness and mercy for a baby crab, drenched herself, and alas, her phone, in a briny estuary. The phone made a few pathetic meeping noises from her pocket, and then spoke no more.

Various attempts were made to revive the poor beast, but it was done for, so a new phone was in order. While the nice lad at the Sprint store did heroics to pull the contact list off it, he pointed out that we might want to add in the equipment protection plan so that if something like this were to happen again, a replacement phone could be had without splashing out the primo price. Supposedly as long as it wasn't abuse, it would cover "everything": phones being dropped, bits falling off, whatever. Falling in an estuary? Sure thing. Given the girl child's track record on wearing out phones after about a year, it seemed like seven bucks a month well spent.

Time passed, and several phones, and another phone started losing its mind and hanging. Covered? No, that's software, not covered, but hey, it's OK because you're due for an upgrade.

Shortly thereafter the upgraded phone fell in some soup, and died. Covered? No, water damage not covered. No? When we *got* this plan precisely because of water damage. Nope. So nice young man was either a lying snake or an ignorant worm. Great. Fine, clone the info back to the phone that keeps losing its mind, and we'll wait out the year until upgrade time. Live and learn.

Then the trackball fell off. Covered? No. That's OK, she learned the keyboard shortcuts, waiting out the six months until upgrade.

Then the power charger socket broke so it couldn't be charged.

She talked to the nice Sprint people on line. Two of them, in fact, both of whom assured her that since I was a Sprint Premier customer and she was on my plan, she could use one of my upgrades, no problem, and they'd even put a note in the account so that she got the mail-in rebate as well. Just go to the store and they'll handle it. How helpful. She printed these conversations out.

What do you think happened when we went to the store?

Is the charge socket breaking covered? No. Of course not. Apparently, rather than "everything" being covered by the monthly tax, nothing is covered. Talk about your bait and switch, AKA lie, lie, lie.

Sprint is, of course, the master of bait and switch. They advertise their wonderful 4G network, and charge me a monthly premium because my phone is a "4G phone", but where oh where is this alleged 4G network? Anywhere I have ever been? Nope. Glad to see Visalia and Fresno are covered, though. And Sprint advertises that the phone can act as a networking hotspot, but do they tell you that it can only do that if you pony up an additional thirty a month to enable it? Naaah. That would be actual honest dealing, something Sprint is constitutionally incapable of.

OK, so can we do the upgrade thing the online guys spoke of? Let's just check....oh, sorry, no, because my line isn't due for an upgrade for another year it can't happen. So what about those on-line helpers? Apparently they were also either lying snakes or ignorant worms. But what we could do is add another line and then you'd get the discount on the phone.

OK, so if I pay to activate a line, and the extra ten bucks a month for at least the next two years (or pay $150 to cancel that line) as well as paying for a line that I don't even have a phone connected to, then, you'll actually deign to replace a broken phone that we have poured many many months of seven bucks a month of replacement plan money into, is that it? I'll get $150 rebate on a phone for the low-low cost of $240, or the even lower low cost of $160?

But you can't, heaven forfend, apply an upgrade to an account of over a decade's standing 35 days (yes, 35 lousy days) early? Or do what two of your online account reps independently said we could do? No, apparently not.

We declined that particular kind offer. We put the girl child's number onto my old phone. We cancelled the rip-off equipment non-protection plan. In 35 days, she'll get her telephonic life back.

Since we didn't actually buy a phone, we are going to even get the customer service "how do you rate us" call. So here it is, Sprint: on a scale of 1 to 10, your customer service today rates a 0. Thanks a lot. For nothing.

Italy vs Serbia

It looks like yet another international football match has been marred by fan violence: the qualifier between Italy and Serbia was abandoned after seven minutes and a kick-off delayed for half-an-hour.  The scenes from Genoa are not pretty, with Serbian yahoos making their nationalistic salutes, doing their best to tear up stadium barriers, and tossing incendiaries onto the pitch.  It is particularly loathsome in that the pricing for the game was set to encourage families to bring their children, and many children were indeed in attendance.  The disregard for human life is appalling.

The likely outcome for this is Serbia being banned from the competition at least and, so reports say, Italy awarded a 3-0 victory.  The EU mavens may well be rethinking the implications of insisting on freedom-of-travel for Serbians as a condition of joining, and maybe rethinking getting Serbia to join at all.  This is likely a large part of the point of the "fans" actions.  They will go back home with their heads high, nursing their unfounded sense of Serbia being unfairly put upon by the rest of the world.  Good riddance. 

I'm all in favour of punishing the Serbian FA for the actions of their so-called fans.  Still, where it the Italian responsibility here?  Why should Italy be rewarded for poor crowd control?  What struck me is that the scenes in the video clips I saw did not seem substantially different from what we see from the ultras in any important match involving Italian teams.  The Italian FA and police have apparently long accepted fans bringing flares and fireworks into stadiums, and waving and tossing them, and have decided to solve the problem by not solving it at all: by wrapping fans in perspex cages and hoping they don't end up with a Heysel or Ibrox or Villa Park on their hands.  That looks to me like an appalling disregard for human life also.  Do we really have to wait for the next disaster before they act?  Really?  Shame on them, too.

Glottochronology

A couple of papers on glottochronology recently came across my desk: Measures of lexical distance between languages and Lexical evolution rates by automated stability measure by Filippo Petroni and Maurizio Serva.

Rather than rely on human judgements of which terms are cognates, they compute normalized Levenshtein distances between pairs of words from the Swadesh list and then use another automated procedure to compute the correlations to compute phylogenetic trees. The resulting trees are comparable to those produced in the traditional way, and the papers include some analysis of the stability of the technique and such. It is interesting in that since it is a purely mechanical operation, one can grind through a lot of languages given nothing more than some basic vocabulary lists. The word lists and resulting trees are on-line for the interested. (Note: the time axis in the trees runs, annoyingly, from 0 to 5000 plus some unspecified amount of trailing years, so mapping the branch points to approximate dates requires some amount of mental arithmetic.)

There is something deeply appealing to me about applying mathematical approaches to linguistics in this way, but some note of caution is warranted as well. The drawing of phylogenetic trees (as the name should indicate) is borrowed from the use of matching techniques on DNA sequences to construct (true) phylogenetic tress. The application in genetics is more straight-forward, however, in that snakes don't invade dog-land and replace half their genes. In addition, while DNA sequences capture the essence of inherited traits in living creatures, there is a lot more to a language than 200 core words. Plus, while all living things appear to apply exactly the same rules in interpreting DNA sequences, languages have a lot of different takes on how they use phonemes to generate meaning: it isn't clear that Levenshtein difference is a great measure to use in comparing languages that use vowel variations for inflections (think: strong verbs) against those that use endings (think: weak verbs) when applied to some default form.  In sum: there are a lot more features to language than words, we can't be sure that the comparison rules are consistent, and language contact and borrowing plays a role that it doesn't (outside the microbial world) in genetics.

This is not to say this is a useless enterprise, not by any means, and certainly the practitioners in this field are well aware of these issues. Indeed, the mathematical approach provides some hope of including additional language features and untangling some of the mess. The details of the comparisons can give clues to some of the impacts of contact.

Speaking of contact, Santa left me the John McWhorter's "Our Magnificent Bastard Tongue", in which McWhorter applies his work on creoles to the history of English and rails against the focus on words, pointing out that the grammar of English (and Proto-Germanic) shows some very interesting things about the history of English as a language formed in language contact, and not just by borrowing a lot of words. In particular, he points to a couple of odd-ball features of English grammar (the meaningless do, and the use of progressive for present tense) as coming from Welsh. He argues that the supposed Celtic genocide never happened (and cites some recent genetic data to back him up). Later on, the Danelaw led, not just to the adoption of some words, but by the streamlining of the grammar. Fascinating stuff. The argument looks weakest in trying to explain why written English changes at such odds with the supposed timing of the changes on the ground.

Where things turn truly speculative is when he points to some slender evidence for Semitic (Phoenician?) influence on Proto-Germanic: the loss of inflectional endings, the consonant shift, and the introduction of a lot of words that lack cognates in the rest of Indo-European. Tantalizing, for sure, but surely speculative.

Bringing us back to the top, having a fuller set of features to compare languages should be able to show us these things. Does Proto-Germanic show affinity for Phoenician? How much?

Drat! A Reason to Boot to Vista

Some months back I acquired a nice practically brand-new quad-core box. The fellow I got in from was kind enough to install it as a dual-boot with Fedora and Vista.

What a pleasure it has been to to have to lug the laptop home and use it to tunnel in to work to get meaningful work done from home without worrying about whether things will fail in mysterious ways, and being able to configure my own imap server if I want to. Like all Vista machines, that laptop functions like an old VW Bug with funky quirks that don't apply to any other such machine ("reverse doesn't work, and you have to be gentle nudging it into second or it will stall"). In my case, the cursed laptop refuses to install anything except (mirable dictu, touch wood) Firefox updates.

In any event, after lo! these many months, I have never once felt anything close to the need to actually boot to Vista on my new box. Until yesterday, dang.

This is all my son's fault. He was playing with a fractal drawing tool I had installed, and wanted more. Since he mostly uses a Windows box, he went looking for something that would run on Windows. And he found a truly astonishing bit of donation-ware: Incendia. If you are remotely interested in computer art, or fractals, go check it out and toss the author some money, because he deserves it. Or check out some of the samples. Here's one my son did.

Bad news for me: this doesn't run on Linux, and really, nothing remotely close to it runs on Linux. So if I want to play too, I have to boot to Vista. Of course, it is Vista, and in this case I have to log in as administrator to run the program or suffer outrageous flicker.

When Irish Eyes Are Not Smiling

France 2 Ireland 1 FIFA 0

Someone once told me that the perfect game was one that had a close score, a lot of chances, at least one switch in the lead, at least one moment of controversy, and a come-from-behind triumph.

Well, by that measure, the World Cup playoff between France and Ireland must be some kind of gem. I guess we forgot about the part "and the outcome not decided by cheating or an officiating mistake."

There has been much ink spilled, and all manner of ugly truths put about on this issue, from which many wild conclusions have followed.

Ugly truth (courtesy Roy Keane): France is going to the World Cup, get over it.

Ah Roy, politic as ever. And still burning with anger over his treatment by the FAI, apparently.

Ugly truths (Roy Keane, again): Look at how well you're developing your youth squads, Ireland: why are you even in the playoffs? Look at all the chances you didn't finish: why is the game so close that one bad call makes you lose the game? Look at how you defended the set-piece: why are you letting Henry goal-side, why are you letting the ball bounce in the box, where's my goal-keeper?

Roy is obviously a graduate of the blame-the-victim school of grief counseling. That said, he has a point. Aidy Boothroid said something similar when Attwell (may his name live in infamy forever) handed Reading a goal for a ball that never saw the inside of the posts: you should take your own chances and not put yourself in a position where one bad call makes the difference.

Ugly truth: Why is Ireland even playing France? Why did France get special seeding by FIFA changing the rules midstream? Maybe France should have been playing Portugal. (Not that the seeding helped Russia.)

FIFA deserves any black eye it gets for its continual bending of the rules to the benefit of the favoured few top teams. This seems a common failing of soccer federations: UEFA does it, the English FA does it, the MLS is a hopeless case in this regard. It all stems from a shocking misapprehension of a key attaction of the sport: the real chance that underdogs can win on any given day.

Ugly truth: What excuse do the officials have for being out of position on a set piece?

Well, they weren't out of position, actually. The linesman was down by the corner, and the ref was back outside of the area so he could get a good view of the shocking amount of pushing and pulling going on.

Ugly truth: "Everyone cheats, it is up to the refs to spot it"

The first half may be an ugly truth, although it must count as a poke in the eye to all the honest players out there, but the second half is an unwarranted conclusion.

And the most obvious ugly truth of all, courtesy Henry, trying to save his reputation: Yes, I handled the ball; I think the fair thing would be for a replay.

Yeah, right. Like that was ever going to happen. If you were so gung ho on fair play, where were you at the time?

Some have concluded from this swamp that soccer needs video replays to eliminate error.

Folks, soccer needs video replay about as much as it needs TV timeouts. The best proposal I have heard has a fifth official watching video replays who can "raise his flag" to the ref, if he thinks the officials on the field have missed anything, leaving it up to the center ref to decide whether to accept that call or not. Such a system might be quite workable, and may stop FIFA from suffering the ultimate humiliation of seeing a World Cup Final decided by cheating. My feeling is, it will neither satisfy the purists who want the game to flow without endless stoppages and revisionist decision-making, nor those who want to ensure there are no "incorrect" calls, ever. Some wrong decisions seem so trivial that it would be pointless to object in the flow, until they turn out to end up in a goal. Example: half-way line throw-in goes to the wrong team, who quickly launch the ball thirty yards upfield, forward gets behind the defender, bam! goal. (Real example, real game, by the way.) Now what? Rewind the last 15 seconds of play for a throw-in decision? Or review every throw-in decision in real time?

This game does not need to follow the NFL down the path to perdition where there is no such thing as cheating, only calculated application of specific infractions for which there is, somewhere in the 300 page rule book, a specified penalty. Do we really want soccer to become a game where you need to be advised that "if a push happens in the last twenty yards during the last 3 minutes of the game, and there are no timeouts left, the aggrieved team has an option of calling a bonus timeout, unless it is Tuesday, or a flying monkey is spotted in the stands" (or whatever the latest gerrymandered NFL rule is, I can't keep 'em straight). I say not.

What this game needs is a dose of sportsmanship. Stop calling cheating "simulation" for starters. Put some real teeth into the "ungentlemanly conduct" law -- if you're going to toss yellows about for foul language (another thing "everyone does"), how about for cheating? Any attempt to con the officials, should be dealt with very harshly. If you're going to add video-based fines and bans for fouls, how about for cheating? If Henry knew he would miss the first three games of the World Cup (which is about all France can expect to play, based on their recent form), he may not of fessed up in the heat of the moment, but at last France would suffer a realistic penalty for their actions. Or maybe: proactive red and yellow cards: you should have got a yellow in your last match but you conned the ref, fine, have one for the start of your next game.

Pull disclosure: My husband is half-Irish. Much dark muttering into Guinness followed last week's debacle.

SIGIR Day 1

Opening session: Susan Dumais' Salton Award Lecture

Dumais walked through a personal history of her work. She spent some time talking about the problem of re-finding: her data shows that 50-80% of web visits are re-visits and 30-50% of queries are re-queries, which speaks to a tremendous opportunity to use that knowledge to help users more effectively. In experiments with refinding on the desktop she discovered that while queries to a search box tend to be short with few operators, if you give people a rich faceted interface, they will use it. Being able to augment the results with new criteria and resort based on those criteria is key. She also pointed out that some attributes are highly resource dependent. For example, which "date" should you use? For a meeting: the time the meeting is scheduled for; for an email: when it was sent; for a photo: when it was created; for a document: when it was last changed, etc. She then spoke about some work on personalization, where prior clicks along only got to predictions that were right 45% of the time, whereas adding in prior clicks plus dwell time plus other session information got to 75%.

On web search at 15: "There is something amazing in that you can have 20 billion pages of diverse uncurated stuff and find anything!"

Although web search is amazingly effective, many tasks are still hard, and much remains to be done. "Search in the future will look nothing like today's simple search engine interfaces. If in 10 years we are still using a rectangular box and a list of results, I should be fired." [Mar 2007]

She spoke of thinking beyond traditional IR boundaries and identified several challenges:

• information dynamics: information changes a lot and people revisit pages a lot. Queries have different temporal dynamics and pages have different rates of change. Including this information in the user interfaces would be very helpful. What if changes in search results from the last time you searched were highlighted in some way? What if changes in a page you re-vsit were highlighted in some way. She gave by way of example a browser plug-in she has been experimenting with, which showed her that there was a free continental breakfast announcement added to the SIGIR page. (Which I missed, not having this fancy plug-in, drat!)
• data as a critical resource: research on user dynamics depends on data such as large scale web query logs, which are hard to get, and harder to share. We can also look at operational systems as an experimental resource using A/B testing (as Amazon does, for example).

In the Q&A someone raised the poverty of the web search box, which hasn't changed visually in 25 years. She remarked: "We are not nailing search anywhere. You're right. We are training ourselves to ask questions that those resources can answer." She then went on to remark that a lot has changed behind the scenes, but transparency matters and matters a lot more for re-finding: if you give people the tools to tell you what they know about what they are looking for they will be more successful.

Classification and Clustering

"Context-Aware Query Classification"
Derek Hao Hu (Hong Kong University of Science and Technology)
presenting for a long list of authors from the university and Microsoft

The basic idea was to apply a fairly simple statistical model to use the previous query and previously clicked links from that query to predict the category for the current one, using a query classification based on some web dictionary or taxonomy such as Wikipedia. The idea is to figure out if by "bass" you are talking about the fish or the instrument, for example.

"Refined Experts: Improving Classification in Large Taxonomies"
Paul Bennett (Microsoft) and Nam Nguyen (Cornell). Paul Bennett presenting.

Interesting data points:
If you have a hierarchy of classes, using hierarchical training is (slightly) better than flat training. What this means is that if you have taxonomy that says class A contains subclasses A1, A2, and A3, the training set for A1 is things in A, with positive examples being things in A1. The point is, you don't put things from B in as negative examples of A1 to do hierarchical
training.

Significantly better is something he calls Refinement and Refined Experts.

The basic idea for Refinement is to use cross-validation to predict which test documents belong in the class, and then use both the predicted and the actual to do the training lower down.

The basic idea for Refined Experts is to add "metafeatures" that are predictions from the child classes as part of the input to feature vectors for the parent class. This means augmenting the training data for class A with metadata for A1, A2, etc.. So you train from the bottom up, and use cross-validation from the leaves to add to the training set for larger classes.

So there is a bottom up training pass followed by a top-down training pass.

The intuition behind this is early cut-off of miscategorization from the top, and that it may be easier to distinguish some things at the leaves than in the context of larger classes at the top.

The Refined Experts algorithm led to a 20-30% improvement in F1.

"Dynamicity vs. Effectiveness: Studying Online Clustering for Scatter/Gather"
Weimao Ke et al (University of North Carolina) Weimao Ke presenting

The central observation here, albeit based on a very small study, is that you can pre-compute clusters statically and then just do tree cutting to select the appropriate clusters in at run time and get results (both in terms of precision/recall and user experience) that are just as good as you do from computing the clusters dynamically from the search results and from selected clusters of search results. Given that clustering algorithms run to the O(kn) to O(n^2) (n=#documents, k=#classes), being able to run this offline could be a win. The main down-side, of course, is that this doesn't play well with a highly dynamic document set.

Web 2.0

"A Statistical Comparison of Tag and Query Logs"
Mark J. Carman (University of Lugano), Mark Baillie (University of Strathclyde), Robert Gwadera, Fabio Crestani (University of Lugano) Mark Carman presenting

Given you are interested in personalization, you need personalized judgements of relevance. You could use query logs for this, perhaps, but what if they aren't available? Perhaps you can use tag data as a proxy for that. The question is: is this really valid?

This work reported on an experiment to compare the vocabulary distribution from queries, tags, and content to see. It compared AOL query log data against tag data from Delicious and categories from Open Directory Project (DMoz).

The conclusions:
For the tags and queries for a URL:

• the vocabularies are similar
• term distributions are correlated but not identical
• the similarity doesn't seem to depend on category or popularity
• content vocabulary more similar to queries than to tags
• query/tag distributions more similar to each other than to content
  

In the Q&A it was pointed out that given that the documents are selected based on the query terms, the closeness of query terms to document terms is unsurprising.

"Simultaneously Modeling Semantics and Structure of Threaded Discussions: A Sparse Coding Approach and Its Applications"
Chen Lin (Fudan University), Jiang-Ming Yang , Rui Cai , Xin-Jing Wang (Microsoft Research, Asia), Wei Wang (Fudan University), Lei Zhang (Microsoft Research Asia)
Chen Lin presenting

This talk was all about figuring interesting things about about threaded email discussions. The central assumption was that you had no information about who replied to whom in the thread.

A mathematical model involving post/term and topic/term matrices and applying a minimization algorithm against a function that combines structural (reply-to) and semantic (topic) terms to infer topic/post relationships. The assumptions are that a thread may have several topics, each post is approximated as a linear combination of previous posts, a post is related to only a few topics, and each post is related to only a few other posts.

This model of the semantics and structure of a thread was then tested against other techniques on slashdot and Apple discussion forums for various purposes. The first was to construct reply relationships, based on the assumption that the reply is similar to the post it is replying to. The technique did beter than the various baselines (reply to nearest, reply to root, basic document (term) similarity, etc.). It also did well for junk identification (against term frequency, SVM, and others), and for identifying the experts (by finding hubs in the graphs of replies).

The interesting thing is that the structural terms dominated, mathematically, which says that if you actually know what the reply-to structure is (which you do for both slashdot and the Apple forums, for example -- that provided the basis for evaluating the quality), you've won half the battle in solving some of these problems.

"Enhancing Cluster Labeling Using Wikipedia"
David Carmel, Haggai Roitman, Naama Zwerdling (IBM Haifa Research Lab)
David Carmel presenting

The problem here is to come up with a label for a cluster of documents. The standard technique is to identify the "important terms" associated with the cluster using various statistical techniques. The problem is that such terms may be statistically useful, but they may not actually help humans figure out what the cluster is about. Further, a good label may not even appear in the actual content.

They did a small study to look at whether an ODP label could be identified as an important term from the content and in only 15% of the categories did that term appear in the top 5. Example: "Electronics" category gets terms electron, coil, voltage, tesla, etc. -- good terms, but none good as a label.

Their solution: use Wikipedia:
Use the cluster's content to search Wikipedia, and use metadata from Wikipedia to produce labels. The overall process is simple enough: cluster, get important terms using conventional techniques, take those as a query to Wikipedia (boosting the weights of the terms that were more important), extract metadata (title, category) from top N relevant documents, and then choose the best. For judging the best categories, they found that you needed to avoid getting too many relevant documents from Wikipedia, and a ranking system based on propagating scores (based on the linkage of documents and labels and terms) worked the better than one based on average mutual information. The technique seems fairly robust in the face of noise: even at 40% noise the results were reasonable.

For final thoughts:
Wikipedia is great for the topics it covers, but it doesn't cover everything.

Similar techniques could be used for other mining tasks: word sense disambiguation, semantic relatedness of fragments, coreference resolution, multi-lingual alignment or retrieval, query expansion, entity ranking, clustering and categorization, etc. etc.

Efficiency

"Compressing Term Positions in Web Indexes"
Hao Yan, Shuai Ding, Torsten Suel (Polytechnic Institute of NYU)
Thorsten Suel presenting

The paper looked at various techniques for compressing positions in an inverted index by exploiting clustering effects in relatively short documents (web pages). It then looked at some issues with query processing with position data.

The key observation is that positions in text are clustered; they are not evenly distributed. This observation can be exploited for specialized compression schemas (cf IPC Moffat/Stuiver 1996), LLRUN, others). The problem is that these don't work so well with short documents.

A second observation is that when storing positions we already know the term frequencies and document size. Various experiments were done with various adaptive encoding techniques, such as variants of Rice coding using information about frequency and document size to divide document into buckets and determine basis of coding, with and without adaptation from bucket to bucket, using regression to modify the encodings, using Huffman coding with different tables, chosen based on various features.

Different terms behaved differently, but the compression gains from all these techniques were disappointingly modest. Maybe a completely different approach is needed: storing approximate positions, or something else. Maybe we need to arrange to avoid decompressing most position data: if only use positions on top N (for position boost), there is little difference in the results, for example.

One comment: given the large size of the collections these days, storing a few extra parameters to improve modeling is not a big deal.


"Brute Force and Indexed Approaches to Pairwise Document Similarity Comparisons with MapReduce"
Jimmy Lin (University of Maryland)

The paper looked at computing pair-wise document similarity using MapReduce and various algorithms, with various parameter settings. The data set was some Medline documents. The take-home message was that in a MapReduce environent, the cost of keeping track of intermediate data and shipping that around the network was great enough that the brute-force approach of just computing the dot products on all pairs of term vectors actually took less time, and since it was an exact result rather than an estimate (which is what the other approaches were) it gave better answers also.

On the other hand, given that the study platform was Hadoop, which was evolving as the study was going on (so that the same test on different days would give very different times), the absolute performance measures here are probably not meaningful.

"Efficiency Trade-Offs in Two-Tier Web Search Systems"
Ricardo Baeza-Yates , Vanessa Murdock (Yahoo!), Claudia Hauff (University of Twente)
Ricardo Baeza-Yates presenting

The problem context for this paper is a search system which has a small local server (cheap, fast) and a larger 2nd tier server (expensive, slow). Queries that cannot be answered by the local server must be referred to the 2nd tier server.

Approach 1: Search the local server and if that does not give you good results then refer the search to the tier 2 server
Problem: you end up waiting longer than if you sent to tier 2 in the first place.

Approach 2: Search both in parallel and then merge or merge only if you need to.
Problem: Increases load on tier 2 and so is more costly.

Solution: Predict which queries are local, and send to tier 1 only or to both in parallel, depending on prediction. The paper looked at a mathematical model for determining when this was worth doing and a sample system was OK.

Result assessor:
If you failed to predict that you needed tier 2, and you did: you have to wait while the query is sent to tier 2; the query serialized (line in approach 1)
If you predicted you needed tier 2 and you didn't, you added extra load to tier 2 but you don't have to wait for it to come back.

Predictor:
Train based on top 20 p20(20) using SVM features from literature for
pre-retrieval prediction
Post-retrieval assessment: if have enough results, then OK

You can improve this a lot by caching false negatives.

A question came up about what the caching of whole results does to this model. It isn't clear.

A Day at the Maker Faire

The Maker Faire has always been one of those things that have sounded cool and we've meant to go to, but hadn't got around to. This last weekend we went up and a had a very tiring, but interesting day out. We were a little hampered by the fact that we came in through the back (Caltrain) door, and didn't get a map. It turns out we missed two whole buildings, and didn't find the good food zone until after we had eaten some fairly lame sandwiches. Still, a good time was had by all.

The best part is talking with the individuals playing around with random hacks in their garage: "how does it work?" and "what is this?" always led to interesting conversations. There's really nothing so fun as listening to someone who is excited by what they are doing explaining it to you. Since we came in the back door, the first zone we entered was the musical hack zone, which immediately sold the boy child, who engaged in some in-depth discussions of types of pickups and digital effects on electric guitars. Our favourite here was the guy with the Wii controller in his head of his guitar and the flat-panel display in the body, wired up so that particular chords have particular imagery, flinging the guitar head down gave a wah-wah effect with a coordinated visual effect. Hitting a button gave a distorted sound, and a distorted image to match. Credit where credit is due: Ben Lewry, and he'll make one for you too, apparently.

Most interesting reward for lowest expectations: A small tent, looking for all the world like a something you'd expect to find a Tarot reader in, tucked away among the 3-D cameras and glasses. Even the label wasn't particularly enticing: "holomagistics" or something of that sort. Inside the small, darkened tent was a small oscilloscope screen showing simple looped figures. Nothing too exciting, except the figures figures are 3-D. No fancy glasses. No need to stand in exactly the right spot. Amazing. If I understood correctly, this is done by reprogramming the scanning so that vertical scan lines alternate between different (computed) perspective views, and your brain performs the object fusion to give the sense of depth. Someone, somewhere, get this guy to think beyond "coordinated music and image displays" and invest.

The kinetic sculptures were engaging and beautiful (I love the displays in science museums of old brass-and-crystal machines, my favourite by far is the Fourier analysis machine in the London science museum, all gleaming spheres and dials), but my favourite art was Bulatov's metal sculptures. Cuteness points to the people with the candy-fab machine, slowly churning our sintered sugar creations."Please don't eat the sculptures!" is not a sign you get to read every day.

We did appreciate the Exploratorium's little puzzles too, though. My favourite: hollow aluminum tube, about 4 or 5 feet long, and a magnet. Drop the magnet through the tube, and it takes a suspiciously long time to fall through. This is the Exploratorium, mind, so we set aside our initial suspicions of some kind of slight-of-hand.

Somewhere late in the afternoon, we decided that we were well and truly tired out, so we sought out the Tesla coils, had a nice chat with the fellows there about their plans to build 12-story tall Tesla coils in the Nevada desert, and watched the lightning show. Even the 12 foot scaled down versions, running well below capacity, were impressive enough, and a good way to sign off the day.