I had a student once that developed a system for selecting a bunch of things using a flashlight metaphor. By attenuating the intensity and spread of the light, it would select multiple objects in the flashlight’s focus.

Another of my favorite advanced selection paradigms are regular expressions. I’ve always thought that if there could be a visual language for regular expressions, that we’d really be in a different place on the computer.

Jeff Han’s done a nice job of building a kind of “grammar” for interacting with multiple objects on screen with his multi-touch work. Perhaps we are close to a more powerful means to pointing that even transcends the logic of our own two hands?

Consider folders for example. A file can be in one folder. This is a physical limitation. In the computer though there’s no particular reason to impose this limitation. We impose it because folders are simple and familiar, but sometimes the simplicity of the organization system results in added complexity of using the interface. We build multiple interfaces on top of folders to deal with the limitations of folders: search, virtual folders, tagging, etc. Would it not be simpler to have one simple abstraction that can handle these needs?

Point and click and drag tends to work well when you have a small number of items you want to work with, but it becomes unwieldy with a large number of items. Babies speak with points and grunts and cries. As adults we use language instead. It’s simpler to communicate most things with language than to express the same thing with pointing and grunting. Command line interfaces are overkill for a small number of items, but they are nice when working with lots of items, or when working with more complex linguistic constructs, such as conditionals, past/future, and hypothetical statements. Try expressing, “next wednesday, if it’s raining, bring me a sandwich.” Sometimes a more complex interface (e.g., language) can lead to easier communication, but it does require some investment to learn.

The iPhone does look nice though.