In this course, you will learn relevant fundamentals of human motor performance, perception, and cognition that inform effective interaction design. You will use these models of how people work to design more effective input and interaction techniques. You’ll apply these to both traditional graphic and gestural interfaces.
From Fingers to Screen
Loading...
From the course by University of California San Diego
Input and Interaction
584 ratings
From the lesson
Input
There’s more to interfaces than what’s on the screen. While often overlooked, input deserves to be on equal footing with its more popular sibling, output. Also, input is a rare case where we can model user behavior mathematically. In this module, you’ll learn how good input is more than just preference, trace input from the fingertip to the screen, and think about the diversity of possible input devices and their relative merits.
Meet the Instructors
Scott KlemmerProfessor
Cognitive Science & Computer Science
Here we have one of the most pedestrian and
ubiquitous input devices out there, the keyboard.
And what's interesting about the keyboard,
is that there's actually an enormous amount that happens,
between when I type with my fingers, and when something appears on the screen.
Through the magic of direct manipulation,
it looks like with a keypress to the screen appearance, it's instant.
But what are all the intermediate steps?
That's what we're gonna go through in today's video.
You take an everyday keyboard, like the one that you see in the image here, and
you peel it back, you get something that look like this.
It's not just a cool wallpaper pattern.
What we see in this layer here is a circuit that's able to
ascertain which key or keys have been pressed, at an impressively low cost.
So, if we take all of these keys and we pop this open,
what we'll see under the hood is that we've got a lot of rows.
And we've got a lot of columns.
So, these are the wires that we saw inside the keyboard.
So we have our rows.
And we have the columns here.
So altogether we might have eight or nine rows, we might have a dozen columns,
it'll depend obviously on the exact keyboard layout.
What's notable about this though is that you have many more keys on your keyboard
then there are rows plus columns.
So how do we take eight or nine rows, ten or 11 columns.
These are gonna go together to an encoder that sits over here and
listens to all of these wires.
But again there are more keys than there are wires.
So how does this all add up together?
If we zoom in, what you see here is that
each key represents the intersection of a row and a column.
When the key is up, all of these switches,
each key is a switch, the circuits are open, they're not closed.
When we pressed down, that will complete a circuit for
a particular column and for a particular row.
So if I press down this key here which might be A,
both this column is gonna light up and this row is gonna light up.
And so this is like if you played the board game Battleship
where you call out coordinates in row-column space.
Same idea here.
The encoder is going to see column one is hot and row three is hot.
The keyboard encoder knows that when column one is hot, and row three is hot,
that that corresponds to a particular physical key on the keyboard.
Any individual keyboard could be labeled with lots of different languages,
do lots of different stuff.
So we see with the key codes that the encoder talks about,
those are the physical keys, not yet the alphabet that will appear on the screen.
So in addition to A through Z and 0 through 9, we've got all of your F keys,
all the other stuff.
The keyboard I showed you here has a number keypad on the right.
The key code for that number one on the right is different than the key code for
the number one on the top left of the keyboard.
So far what we've done is we've taken the physical press of your finger,
we've used that to close the circuit, and we've used that to describe a key code.
Now we're gonna talk about how that key code bubbles up.
Your encoder is going to send to the computer the key code that is pressed.
So if I've got my computer right here
it's gonna hear from the keyboard.
1C, which might be the key code for A.
From there that might be interpreted by the computer to understand that,
on a US layout keyboard, that's what 1C means,
and then that's what we'll see on the screen.
This skips all of the graphics processing part of it.
But for today, we're just talking about how the input side of things work.
So let's say we have a different example.
Let's say that you press, column three row four here,
which might correspond to maybe C and
that will have its different corresponding key code and
then that's gonna light up a different row and column.
More interestingly, what happens if while key A is pressed,
you also press S, which might be in the same row but a different column?
Here, when I press on the S the encoder is seeing what
it saw before, column one row three for A.
And now it's also seeing column two row three.
If both column one row three and column two row three are showing up,
then we know that both the A and the S are pressed simultaneously.
For the devious among you,
because the row and column set here is a little bit under-specified,
it is theoretically possible that if you pressed two keys that were ambiguous,
two corners here, at exactly the same time, exactly the same time, your
encoder might not know whether it was this corner pressed or that corner pressed.
In reality few users are that devious or at least their timing isn't that perfect,
and so the encoder is able to tell which key is pressed.
We talked a little bit in our history about the role
of Grace Hopper in computing and her invention of the compiler.
One thing you might notice is that she is one early user
of having a keyboard to do programming input as opposed to punch cards or
wires or something that is much lower level.
At the time, this was a real advantage, but
in the intervening decades, we've been able to do a lot better.
But the real challenge is that for
a lot of the tasks that we want to be able to specify to each other, for
example this diagram I've just drawn here, it can be difficult to shoehorn that
purely into things that you could express with a keyboard.
If I had to use a keyboard to draw this diagram, it would be pretty slow.
So what we're gonna do is we've talked about it several places in this class is
that the big idea of direct manipulation is that you get input directly
on top of output and this is where Doug Engelbart and
team in the early 1960s when they invented the mouse.
This was the great leap forward.
Well let's flip that mouse upside down and see what's underneath there.
What you'll see, and this is kinda cool,
in this early mouse what we have are two different wheels.
These are in fact variable resistors.
So the way that we're going to encode the position of the mouse is we have have one
wheel, a variable resistor it's also called a potentiometer, that's going
to encode our x direction, and a separate wheel that's gonna encode our y direction.
This is pretty good, except that there will be a limit when we turn this wheel.
The resistor will turn [SOUND], and then it stops and you need to go back.
After a few decades of innovation and again, we can pop the hood on this,
we can see how things have evolved significantly.
This is a commodity mouse and what I'd like to use it as an example of
is not necessarily how your computer today works.
Of course, if you have a tablet, or you're using a trackpad, things may be different.
What I want to use this as an example of is
a way that you can take an analog signal, like you get from a mouse, and
in a low cost, high robustness way, convert that into a digital signal.
So that grey ball there is the trackball for the mouse.
How does that get converted into something digital?
Well, if you look at it, you'll see that again there is a wheel,
just like in the early mouse and
this wheel has solid parts and transparent parts.
That enables an infrared detector If you shine a light through this wheel
to see whether it's over a solid part or a transparent part.
This is the basis for what's known as a rotary encoder.
Rotary encoders are widely used throughout mechanical systems that need closed
loop feedback It
takes what used to be analog, and in a discrete sense, it now makes it digital.
You can either see through the wheel or you can't.
One and zero, high and low.
So we roll the wheel forward, we go from transparent to opaque.
Okay, high to low, good.
Let's say we roll it backward.
We go from opaque to transparent.
But did we roll it forward or did we roll it backward?
In the basic version, you can't tell.
So, now we're gonna get a little bit fancier and
we're going to add an offset encoder.
This way if we see a particular pattern,
we go from high high to low high, we know we're going in one direction.
By contrast, if we go the other way,
we go from low high to high high, we're going in the other direction.
Holy cow!
Now we have the ability, both in X and in Y, to be able to take our
analog motions on a surface and to turn those into digital movements.
That lets us now control a pointer on a screen.
If you think about a capacitive touch sensor like you might have in your
smartphone or tablet, or other sensing devices you may be using,
the details of this will vary somewhat but the principle that there is a structured
grid of electrical systems with an encoder of some kind, it could be through a rotory
encoding wheel like this or the encoder that we talked about in the keyboard.
That's taking your analog motions, your physical presses, your physical movements,
and turning those into something digital that can then get sent across the wire to
a computer, which lets you move your mouse and type the keyboard.
That's what we've seen is now possible today.
Explore our Catalog
Join for free and get personalized recommendations, updates and offers.
Coursera provides universal access to the world’s best education,
partnering with top universities and organizations to offer courses online.
© 2019 Coursera Inc. All rights reserved.
