Future Magic Mouse could detect gestures in the air like Apple Vision Pro TechTricks365

Apple has been researching a new Magic Mouse for the Mac that would detect gestures — not on its surface, but on the table near it.

When Apple introduced its Magic Mouse back in 2009, it featured multi-touch gestures. Since then it’s been redesigned, and there have been rumors of a further redesign.

Those rumors, though, have concentrated on the recurring issue that the USB-C charging port is on the bottom. There hadn’t been any idea that Apple has been looking at anything else, certainly not at adding more gestures to it.

But a newly-granted patent called “Electronic devices with extended input-output capabilities,” that is exactly what Apple has been doing. Only, alongside taps and swipes, the whole idea is literally about gestures alongside the mouse.

“A mouse or other electronic pointing device may have an array of sensors on a housing sidewall configured to gather information on the location of a finger of the user in an area that is laterally adjacent to the electronic pointing device,” says the patent. “The sensors may be optical sensors based on light-sources such as light-emitting diodes or lasers and light detectors.”

“Capacitive sensors may also be used in gathering user finger input in the area laterally adjacent to the electronic pointing device,” it continues.

The idea is that this would mean “three-dimensional gestures may be supplied by a user.” The gestures could be made by “moving the user’s fingers or hand through the air near an electronic device.”

So a future Magic Mouse could gain at least a subset of the type of gesture detection that is so key to the use of the Apple Vision Pro. Backing that up, the patent adds that the gestures could be detected using “one or more visible light cameras and/or one or more infrared cameras.”

As well as cameras, the patent talks about projectors. The mouse, or other device, “may include projectors… [which] may, for example, project an image… onto [a] surface.

“[The] images projected… may contain virtual keys or other content with visual elements that a user may select with finger input,” it continues. “As an example, [it] may project an image that contains multiple selectable regions… (e.g., visual elements such as file folder icons, keys such as alphanumeric keys, menu options, labels with 10 text, graphics, and/or moving image content, etc.).”

So exactly in the way that you can now get QWERTY keyboards that are laser-projected onto any surface, the mouse could present an image on your desk. You could then tap on that image, and the mouse’s cameras will register your position and infer which projected control you meant.

That sounds more like a MacBook Pro, for instance, projecting a keyboard or a mouse area for the user to move their fingers through. And there are illustrations that show exactly that — except they still include a regular keyboard and trackpad.

All patents do try to anticipate future uses and future patents from other firms, and as ever this one does so by listing more than a mouse. Typically Apple will use phrases like “a mouse or other pointing device,” but this time it goes very much further.

While it does concentrate on this facility being used in a mouse, it also briefly specifies that the same technology could be used in:

• trackpad, or other pointing device
• laptop computer
• computer monitor containing an embedded computer
• tablet computer
• cellular telephone
• media player
• wrist-watch
• pendant
• headphone or earpiece
• headset

And more than a dozen other examples, including “electronic equipment embedded in tables, chairs, desks, [or] other furniture.” So drumming your fingers next to your mouse might kickstart a Shortcut.

Putting a coffee mug down on your desk might turn on your Mac. Spreading your fingers on your table might act as a pinch and zoom on the device.

But Apple does seem to at least have an idea that such a mouse, or desk, would let a user supplement other input, rather than being a complete gesture by itself. The mouse, or chair, would determine “the position of a user’s finger or other body part moving through the air,” it says.

Then this “additional user input may help extend the input capabilities… and thereby supplement the information gathered using buttons, touch sensors, pointing device movement sensors.”

“User input may be used in manipulating visual objects (e.g., icons, etc.), may be used in supplying [the] system… with text,” it continues, “may be used in making menu selections, and/or may otherwise be used in operating the equipment of [the] system.”

So you could maybe click on a “Bid” button in an auction, and then wave your hand to say how high you’re willing to go.

Or perhaps more practically, a flick of the wrist near the mouse could be enough to move the cursor around the Mac’s screen. Just as now you can click even only roughly near a button and it will be detected, perhaps a slapping motion could tell the Mac to click the highlighted tick box.

As ever, though, the patent is chiefly concerned with how something might work. While it gives an unusually comprehensive list of possible devices, it doesn’t give examples of why a user might want to do any of this.

And then as ever with all patents, the fact that it has been granted does not guarantee that the idea will make it into a future product.

The three credited inventors on the patent include the extremely prolific Paul X. Wang. His very many previous patents and patent applications, include one for replacing the old Touch Bar with an Apple Pencil holder.