You may have noticed that when you drag something on your screen, it will often lag behind your mouse or finger, as if attached by a rubber band. This is input latency: a delay in the computer's response to your actions.
Drag the box to see what bad input latency feels like.
Input latency affects every interaction with your computer, not just dragging. It can make modern computers feel sluggish compared to ancient 8-bit devices that are thousands of times slower.
It is unreasonably difficult to minimize input latency on modern devices. It's even difficult just to measure input latency. Many devices and apps we use every day have gratuitously large amounts of unnecessary latency.
Does it matter?â
Input latency is measured in milliseconds, and it may seem weird to worry about intervals of time that small. But to a computer a millisecond is a very long time.
On a normal 60 Hz display there are 16 milliseconds between each frame, during which time the computer can perform billions or even trillions of operations. And yet, many apps have 100 milliseconds of input latency or more. As John Carmack has said, "I can send an IP packet to Europe faster than I can send a pixel to the screen. How f'd up is that?". (He lives about 4,000 miles away from Europe.)
"Sure, computers are fast", you may think, "but humans are slow. Do a few millseconds of latency matter to humans?" The answer is definitively yes. Multiple research studies performed using displays with adjustable latency (here and here) have shown that humans can perceive a difference all the way down to 1 millisecond of latency. Which is far better than can be achieved on any modern computer display!
Your browser and device have some baseline level of latency. They don't allow apps to measure it accurately, so I can't tell you exactly how much. But I can add more latency on top. So here is the draggable box again, with a slider that lets you choose how much latency to add, so you can feel the effects yourself.
100ms extra latency (on top of unknown baseline)
Unfortunately this sample can't show you what truly low latency feels like. When you set it to "zero", you are likely still experiencing 100 milliseconds of latency or much more depending on your browser and device. If you compare the slider at 20 and 1 you probably won't notice a difference, but that's becuase it's hard to tell the difference between 120 and 101 milliseconds of latency. You would be able to tell the difference between 20 and 1, if your device could actually achieve 1 millisecond of latency.
How to fix itâ
As a userâ
As a user, it is difficult to tell how much input latency your device has and whether it is coming from software or hardware. But there are some things you can do:
- High refresh rate displays can help a lot. 120 Hz is common these days, twice as fast as the normal 60 Hz, and the difference is clearly noticeable, both on phones and on PCs. Some gaming displays even support up to 360 Hz now. Variable refresh rate (G-Sync/FreeSync) can help as well.
- Gaming mice with a 1000 Hz polling rate can be had for very reasonable prices, and do reduce latency over normal mice which run at 125 Hz by default. Some wireless devices add latency but others are practically as good as wired so there is no need to stick to wired. Personally I use a Logitech G305 which I think is excellent for $40.
- rtings.com has excellent latency tests for monitors, mice, keyboards, and wireless headphones that you can reference when choosing devices.
- On the software side, AMD and Nvidia GPU drivers have recently added latency-reducing features which can make a big difference in games.
As a developerâ
If you are building a native app using graphics APIs such as Vulkan, DirectX, Metal, or OpenGL, you might expect that you will get nice low latency automatically. Surely these low level graphics APIs are blazing fast by default? Unfortunately not. The defaults are optimized for throughput, and latency can suffer.
The good news is it is possible to do much better. Check Part 2 of this series for a deep dive into techiques to reduce latency in your apps!