A simple stance on the static/dynamic typing debate

Given that my only other post so far steps on the toes of a major company, I thought it might be appropriate to make my second post a bit less contentious. Well, that was until Arthur Whitney (creator of the K programming language, among others) came to talk about language design to our research group over at the Stanford Graphics Lab.

I found the talk and ensuing discussion very interesting, in large part because Whitney’s taste in language design is so far from my own. Of course, its hard to argue with results, and so I’ve decided to spend some time trying to understand my own biases about programming langauges.

Static and Dynamic Typing

Most programming-language geeks seem to fall strictly on one side of the static/dynamic divide or the other (those who talk about “optional typing” as a middle road are almost always closet dynamic-typers). Despite the long-standing flame-war passionate debate between the opposing camps, there is one important position that I haven’t seen explained.

Looking at Method Dispatch

Here is a simple question that can be asked for most langauges: given a method call:

obj.m( a )

what factors contribute to the decision of what code gets run?

Note that I’m using the obj.m( a ) syntax purely as a representative. There might be languages where the same thing is expressed as “obj m a“, “m(obj, a)” or any number of other variations. The syntactic details, and the object-oriented styling just make it easier to contrast some of the typical punching bags in the static/dynamic debate.

My own survey of languages leads to the following list of factors that might affect dispatch:

  1. The method name: m
  2. The dynamic run-time value of the object obj
  3. The statically computed type of obj
  4. The dynamic run-time value of the argument: a
  5. The statically computed type of a
  6. The lexical context of the entire expression

Comparing Some Languages

A typical dynamic class-based language like Smalltalk or Python will use only (1) and (2) above to control dispatch. For the purposes of dispatch the object (or its class) is treated like a dictionary mapping keys to values. In this case the key is the method name, typically encoded as a string “m“.

Dynamic languages with multimethod dispatch also take (4) into account. This is built-in functionality in languages like the Common Lisp Object System (CLOS) and Dylan, and Guido himself demonstrated a way of adding them to Python as a decorator.

Many functional programming languages would use only (1) and (6) to resolve a call. In many of these languages functions like m are just ordinary values, and thus function names get looked up in the current lexical environment like any other name. Since a name typically only binds to a single value, there is no need to consult any other information to resolve a call. If the programmer wants to execute different paths depending on dynamic information (2, 4) then they will use pattern matching – an orthogonal mechanism to ordinary dispatch.

Haskell’s type classes allow dispatch in that language to take (3) and (5) into account. A function name in Haskell might belong to a type class, and thus the compiler must search a suitable type-class instance which provides the concrete functions for particular types of operands. The search for type class instances is driven by the lexical environment (6).

A typical static class-based language like C++, Java, C# or Scala will use (1), (2), (3), (5) and (6). Our mainstream static object-oriented languages don’t currently support multimethods, often for historical reasons (c.f. Martin Odersky’s comments (bottom of the page) on why Scala eschews multimethod dispatch). A static object-oriented language with multimethods could take all of the available information (1-6) into account. If you can’t already guess, that is exactly the kind of language I would advocate building.

Disambiguation

It is completely reasonable to ask why any of this matters. Widely-used languages have covered very different parts of this space, with the only constant being that dispatch depends in some way on the name of the operation being invoked (which it would seem pretty difficult to ignore).

Where your choices about how to handle dispatch really start to matter is when you have multiple entities with the same name. I intentionally used the word “entities” to be all-encompassing, but I’ll give a quick example.

Suppose you have a base class Base and a derived class Derived. What happens if both Base and Derived try to declare a method m, for different purposes? What if Base and Derived were developed by different people, or different organizations? What if Derived was originally coded using version 1.0 of Base which didn’t have an m method, and then the programmer tries to upgraded to version 1.1 which adds the new method to Base?

As another example, what if you have two protocols or interfaces IFoo and IBar and some objects need to support both protocols? Are the two protocols allowed to have operations with the same name m, but different semantics?

In a functional-programming setting the question becomes: what if two separately-developed modules or typeclasses try to define operations with the same name – can user code import both?

Let’s look at how we can cope with these situations, depending on what kinds of dispatch our language supports:

Strategy: Statically-Typed Object-Oriented Languages

If our dispatch logic takes the static types of the operands into account, then there isn’t really a problem. In the expression obj.m(a) we can look at the static type of obj. If it is IFoo, then we can call IFoo::m; and if it is IBar, we can call IBar::m. If there is not enough information in the current lexical context to know which operation the user meant, then an error can be signaled because of the ambiguity.

This is important – the compiler can use all the information at its disposal in trying to resolve a potential ambiguity. In particular, this means they can use all the information that is manifest in the code the user is writing – including the types of the user’s variables.  Only when there is no information available that unambiguously indicates the programmer’s intent will the compiler throw up its hands and request that the user be explicit. All of these languages give the programmer some way to explicitly signal their intent.

In contrast, if our language cannot take any of this contextual information (static types, lexical context) into account, then there is no way that can ever determine the programmer’s intent. All such references would be ambiguous, and we need to figure out some way to circumnavigate the problem.

Strategy: “That just doesn’t happen in practice!”

The easiest solution here is to just bury your head in the sand. This might involve making your language resolve all these name clashes by fiat – every class is just a key-to-value mapping, and there can only be one value with a particular key. Alternatively, you might try to protect the user from creating name clashes in the first place – don’t allow a class to inherit multiple methods with the same name, or to re-declare an inherited methods.

In all honesty, these are reasonable approaches for any “little language.” Coming up with a robust solution to name clashes can compromise the simplicity of a language. The user needs to learn the specialized rules that control dispatch, and since every method call is subject to these rules this can make understanding code difficult until fluency is achieved. C++ developers who fully understand “argument-dependent lookup” are few and far between.

But if your “little language” has grown up to the point where it is being used to ship serious software (hello, Python!), then you will definitely face these problems. If the language does not directly address them, then you will end up using convention to avoid the issues.

Strategy: Name Decoration

A simple approach is to decorate the method name m to avoid clashes in the first place. That is, rather than IFoo and IBar both having a method m, we instead have methods IFoo_m and IBar_m. Of course, since the pain is felt by downstream developers, both IFoo and IBar would need to have preemptively decorated their method names to avoid future clashes.

Python actually has a limited form of support for this approach built-in (mentioned briefly in the Python style guide). Field names that begin with a single underscore are automatically prefixed with the name of the enclosing class. Note that this is a very specific case of taking the lexical context into account (number 6 in my earlier list).

I hope it hasn’t escaped your attention that by amending the method name with the name of the enclosing type we have basically replicated the ability of a static language to dispatch based on the static type of the object. This came at the cost of more verbose method names everywhere – not just at the call sites that needed it.

Strategy: Module-Scoped Methods

The approach taken by CLOS and Dylan is to put the methods associated with a class into module scope, rather than nesting them inside the class. This means that even if an obj.m( a ) syntax is supported, it is just sugar for m( obj, a ), and the name m is always resolved by looking it up in the lexical environment.

The benefit of this approach is that the meaning of m can now be tailored to each call site – depending on which definition of m is in scope. The downside, of course, is that if more than one definition is in scope the reference once again becomes ambiguous.

Disambiguating these methods either requires explicitly qualifying the method with the name of a module: FooModule.m( obj, a ), or importing the method into the current scope with a decorated name: Foo_m( obj, a ). Thus we still end up having to explicitly decorate our references with information that the statically-typed language was able to glean from the current context.

Strategy: Avoid Inheritance

One each way to avoid ever making the obj.m( a ) operation ambiguous is to disallow certain forms of inheritance – either in the language, or institutionally. This might involve a blanket ban on implementation inheritance and multiple inheritance, or some more subtle middle ground.

If an object wants to support two different protocols like IFoo and IBar, it could do this by using the Adapter pattern. Two auxiliary objects would be created (one to implement IFoo, the other to implement IBar) and any place where we want to use our object through one of these interfaces, we pass the auxiliary object instead. Thus our original method call might be disambiguated as either obj.asIFoo.m( a ) or obj.asIBar.m( a ).

This form of disambiguation is not unlike inserting a cast to either IFoo or IBar in a statically-typed language (which is a common way of disambiguating a call). Compared to decorating method names it has the nice benefit that we can “cast” an object once, store the adapter object in a variable, and re-use the adapter for multiple calls (without decoration). Of course, this idiom then looks suspiciously like declaring a statically-typed variable.

Conclusion

When we are “programming in the large” and aggregating separately-developed code from multiple suppliers, names clashes seem inevitable. For most mainstream programming languages, it is possible to plan ahead and build libraries that are robust against these issues. Which language you are using, though, determines which strategies you can employ to achieve that robustness.

In many static languages, name-clash problems are anticipated by the language design, and there are explicit features (including method dispatch strategies) that make it relatively painless to cope with. In particular, users typically only have to amend their code in cases where their intent isn’t already clear from context. The price for this is a more complex set of rules in the language (for resolving potential ambiguities).

In all of the mainstream dynamic languages I am aware of, name clashes must instead be handled by diligent and defensive programming practices. In the best case, the idioms that work to avoid name clashes lead to slightly more verbose code. In the worst case, though, they amount to using ad-hoc mechanisms to express precisely the same information that the static types in other languages provide “for free.” The benefit is that for programs or libraries that don’t need this level of defensiveness, programmers benefit from intuitive language semantics.

For my part, I’d rather build (and use) languages that have a robust solution to problems like this built in. Dynamic typing advocates often complain about having to write extra annotations to “appease the compiler,” but that particular saw cuts both ways. If the context of my code makes my intention clear, I want my compiler to use that context to resolve potential ambiguities, rather than making me always add extra annotations to “appease the runtime.”

Advertisements

Chronicles in Failure: HP Customer Support

My intention had been to use this blog to discuss more useful stuff: programming languages, new games journalism, etc. Unfortunately, in the intervening months my free time has largely been occupied with a single effort – dealing with Hewlett-Packard’s customer service.

The setup is relatively simple. I needed a new desktop machine to work on my thesis project, and had decided to buy a PC from one of the online retailers to “save time.” I had never built a PC from scratch before and was worried about the complexity of shopping for each individual component. This is probably a situation that many users (even those that are technically proficient on the software side of things) find themselves in.

The end result of my limited comparison shopping was that I purchased the HP e9150 desktop. I probably should have done a bit more searching before making the purchase, though, since my own Google search for “HP e9150 problems” find this thread in the first page of results.

That thread goes on for (at last count) 146 pages of customers complaining about random freezes, lock-ups and BSODs relating to the e9150 and its big brother the e9180. Just to put the problem in perspective, in all my discussions with HP representatives they have all plead ignorance of the problems with this product line, despite me providing the URL (on their own customer-support forums) to everyone who will listen.

This is a broad problem with HP’s “top of the line” consumer desktop, and they have yet to acknowledge any responsibility. The best that the unfortunate customers of the product have been able to piece together is that this appears to be a problem with the motherboard used in the e9150 and e9180, but this is by no means a sure thing.

Having established that this line of HP products appears to be irrevocably broken, I’d now like to go over (in as concise of a fashion as my writing style allows) the customer service experience I suffered when trying to deal with this broken product.

Within weeks of getting the new machine I knew something wasn’t quite right. It tended to freeze every night (seemingly while connecting to my backup server) and would occasionally lock up/BSOD during ordinary use. One particular symptom was that after these crashes the BIOS settings would often reset, and I’d get an error message during POST.

I sent a message to HP’s email technical support giving a detailed description of the problems and found the following out:

  • HP’s first “layer” of customer service is clearly following a script where they pattern-match against words in your message. In my case they ignored 90% of my symptoms and homed in on the loss of BIOS settings.
  • HP expects their customers to spend their own time and money on wild-stab-in-the-dark “fixes” for the problems they report. The HP support reps sent me on a trip to Fry’s to buy a new CMOS battery and had me dig around in the PC chassis to reset the CMOS as well (which requires removing the graphics card and using tweezers to move jumpers around).
  • If you respond to the customer service rep to tell them that you jumped through the hoops they came up with, a completely different customer service rep will continue your complaint and come up with a new set of steps.

Once I’d jumped through enough hoops I was allowed to send my machine in for “bench repair.” I included a full page of typed instructions on the issues I was having and how to repro them, including the URL of the other customer complaints. The HP bench repair technicians apparently ignored my repro instructions, wiped the hard drive and shipped the computer back claiming that they couldn’t identify the problem. The paperwork I received when my computer was returned claimed that they had also made “costmetic” fixes to the computer’s case, which was apparently code for “removed one of the rubber feet so that the computer can’t actually stand upright any more.” I plugged the computer in, left it on overnight and found it with a BSOD in the morning.

After a few irate emails with magic words like “incompetent,” “escalate,” and “full refund,” I was finally sent to an HP “Case Manager.” The next layer (or is it “ring” … I think Dante’s Inferno had rings, right?) of the customer support process. My “Case Manager” was named “Faby R.”.

Faby was kind enough to explain that she (and HP) hadn’t the foggiest idea what was wrong with their products (despite me kindly forwarding the burgeoning thread of symptoms and repro cases on their own customer-support forums). What Faby was able to explain to me was the following:

  • HP does not offer customers 100% refunds when they have been inflicted with defective products. They will, however, deign to “buy back” your defective PC for 90% of what you paid for it.
  • HP’s “Case Managers” are (by their own description) the final line of customer service at HP. There is no way to speak to the manager of your “Case Manager,” no way to provide feedback on their performance, and no other way to address customer complaints.
  • HP’s “Case Managers” are (again, by their own description) not allowed to do whatever it takes to resolve a customer’s issues. Instead, they are just as scripted as the first ring of customer support.

Those last two are a frightening combination. HP has constructed a situation where the individuals who are (apparently) at the top of the customer-service food chain are not actually empowered to do their job (you know, serving customers). It should really be no surprise that everything that followed has been just as much of a train wreck as what came before.

I found out that the primary difference in dealing with an HP “Case Manager”  is that you are forever bound to them for the rest of your process. You have no way to get responses from them once they leave for the day (which happened at 3pm for my case manager and time zone), or for the weekend. If you call in to try to speak with your case manager (say after they have committed one of many monumental screw ups that costs you time and money) you will find that the HP receptionist will happily inform you that they:

  • Cannot look up information related to your case
  • Will not let you leave a voice message for your case manager
  • Cannot transfer your case to a new case manager (e.g. in the hopes of finding a literate/competent diamond in the rough)

So, let’s walk through what happened to me next. “Faby” informed me since the bench repair technicians were unable to repro any problems (though not even trying to repro them sounds like a poor excuse), and because HP apparently has no record of problems with this product line (which requires selective vision when it comes to their own customer-support forums), she could not offer me my money back for the broken product that I was now saddled with.

I was left with the choice between having HP “buy back” the PC for 90% of the purchase price, or letting them ship me a replacement PC. At the time, I was naive enough to believe that HP could still do right by me as a customer, and so allowed her to ship me a replacement PC (an e9180 with a higher-specced processor). You can probably guess (by the fact that I’ve written this screed) that the new machine worked no better than the first.

So I was still saddled with a defective product. I’d spent even more of my time trying to help HP fulfill their promise to deliver the functioning,  time-saving PC that I already paid them for. Still, I was informed by Faby that I couldn’t have my money back. I could either stay home from work to let an HP technician come and look at the PC to make sure that I (the customer/victim of HP) wasn’t just making up tall tales of defective PCs, or I could relent and submit to their clever 90% “buy back” scam.

Given the precedent of my bench repair process, where HP’s “technicians” had failed to follow the complicated crash repro instructions of “leave on for 12-48 hours straight, or use for ordinary web browsing and coding for 8-10 hours,” I was understandably dubious that whatever rent-a-geek HP sent on my house call would have the wherewithal to recognize an obvious hardware problem. After two months of doing all of the work in this “customer service” process, of trying to give important technical information to barely-literate service reps who just pattern-matched my email and followed their scripts, I was finally willing to let HP keep their 10%.

They didn’t/don’t deservice it in the least, but believe me: if you ever find yourself dealing with HP customer support, and have the option to spend $120 or so to never speak to them again, take it! It’s kind of like handing a bully (HP) your lunch money in the hopes that they won’t beat you up, I know. The fact is, though, that dealing with HP was actually making me more stressed and aggressive every day (I don’t suffer fools gladly, but who does, right?). My wife and I have a baby on the way, and the last thing I need is to be a gruff, angry person in those first weeks and months.

So yes, I’m letting the bully keep my lunch money. Congratulations, Hewlett-Packard – in the fight against your own customers you are winning.

You’d think that would be the end of it, right? I let HP keep their ill-gotten gains, and they let me FedEx their broken chunk of plastic and metal back to them (probably to pawn off on some other poor SOB, collect $120 in ransom, and keep the cycle going). How could they screw that up?

Well, they can screw that up by completely misinforming their customers (or lying to them, but I think incompetence is more likely than overt malice in this case). My good friend Faby told me to put the machine back in the box it shipped in, remove/cover the existing shipping labels (I tore them off), and write a special sequence of letters and numbers of all sides of the box. She arranged for a FedEx pickup, and I begrudgingly stayed home from work (losing even more money, but at least HP didn’t get to tally that on their bottom line) to wait for the FedEx guy.

Of course, once the FedEx guy got there, he explained that there was no way he could pick up a pacakge without tracking info, or an account number, or a shipping label. I drove the box over to a FedEx office, where the kind receptionist actually told me that this is a common problem with HP! She explained that apparently HP’s customer service people are under the delusion that they have some kind of special deal with FedEx where customers can just write the HP-internal return/remittance number on a box and FedEx will magically be able to accept it without any other accompanying information or documentation.

That’s right, my local FexEx has actually seen enough irate customers trying to return defective merchandise, all of whom have fallen victim to this mass delusion on HP’s part, that when I explained my plight they just said “oh no, not this again!”

I brought all of this to my good pal Faby’s attention, and she helped explain that:

  • Apparently all the FedEx employees are lying to me about how their own company operates. It of course makes sense that Faby would know more about when FedEx will or will not pick up a package than FedEx employees, right?
  • Despite having wasted a whole day of my time and money on her fool’s errand, Faby cannot offer me anything in compensation for her complete and utter failure to perform her stated job. I still only get to keep 90% of my money.

This is of course what happens when you don’t empower your employees to do their job. If your customer service staff aren’t actually allowed to serve customers, you should call them something else. I understand that “team of people designed to tire you out so you keep a broken product or at worst scam you out of 10% of the purchase price” isn’t as concise, and probably doesn’t look as good on advertising materials, but let’s be honest.

Maybe you read this whole thing, maybe you just skimmed for the highlights. If I’ve convinced you that buying a product from HP might not be as good of an idea as it first appears, I’ve done my job.