Voting resources, early voting, and poll worker information - VOTE. ... Adafruit is open and shipping.
0

USB HID on iPad (iOS)
Moderators: adafruit_support_bill, adafruit

Please be positive and constructive with your questions and comments.

USB HID on iPad (iOS)

by kmevans on Mon Aug 24, 2020 9:00 pm

Has anyone gotten USB keyboard/mouse HID to work on an iPad or any other iOS device? I've tried both CircuitPython 6.0.0 alpha 2 and the older 5.3 and am able to see mass storage mount via the Files app, but the USB HID keyboard/mouse functionality doesn't appear to work. I am using a USB C to 3-port USB 3.0 Hub to connect. (iHaper C003.) I plugged in an old USB keyboard (HHKB Lite 2) and that worked fine.

I'm using an ItsyBitsy M4 Express. The same setup worked fine with a OS X, and with a Raspberry Pi. The iPad is an iPad pro with iOS 13.6.1.

kmevans
 
Posts: 44
Joined: Mon Feb 06, 2017 4:32 pm

Re: USB HID on iPad (iOS)

by siddacious on Fri Aug 28, 2020 2:51 pm

If I recall correctly, it took convincing Apple to make some tweaks to IOS to make mass storage work correctly so I'm not all that surprised that HID doesn't just work.

I'm not certain though, so I'll reach out to folks that would know better than me.

siddacious
 
Posts: 305
Joined: Fri Apr 21, 2017 3:09 pm

Re: USB HID on iPad (iOS)

by danhalbert on Fri Aug 28, 2020 4:07 pm

It may require "BOOT" HID support, which we don't handle currently.

danhalbert
 
Posts: 2118
Joined: Tue Aug 08, 2017 12:37 pm

Re: USB HID on iPad (iOS)

by hathach on Fri Aug 28, 2020 11:09 pm

would you mind trying out the TinyUSB library on Arduino to see if it works. It is the same library used by circuitpython, but have user-configurable hid descriptor which make thing easier to try out. Please try hid_mouse, hid_keyboard, hid_composite to see which works

https://github.com/adafruit/Adafruit_Ti ... amples/HID

You will need to switch the USB Stack from Arduino IDE menu from Arduino -> TinyUSB

hathach
 
Posts: 1110
Joined: Tue Apr 23, 2013 1:02 am

Re: USB HID on iPad (iOS)

by kmevans on Sat Aug 29, 2020 4:20 pm

Ok let me first try the absolute latest releases of everything for CircuitPython, and then port to Arduino. I'll post sample code and results.

kmevans
 
Posts: 44
Joined: Mon Feb 06, 2017 4:32 pm

Re: USB HID on iPad (iOS)

by kmevans on Sat Aug 29, 2020 6:57 pm

First for reference I verified that a Logitech K270 mouse/keyboard plugged into an iHaper USB C003 USB-C to 3-Port USB 3.0 Hub works and provides about mouse and keyboard functionality. I used the Notes iOS app for testing.

* CircuitPython on ItsyBitsy M4 Express *

CircuitPython 6.0.0 Alpha 3 was announced here:
https://forums.adafruit.com/viewtopic.php?f=60&t=168889
But I couldn't find it here:
https://circuitpython.org/board/itsybitsy_m4_express/
So I'm sticking with 6.0.0 Alpha 2.

Code: Select all | TOGGLE FULL SIZE
Adafruit CircuitPython 6.0.0-alpha.2 on 2020-07-23; Adafruit ItsyBitsy M4 Express with samd51g19


I downloaded the latest libraries in adafruit-circuitpython-bundle-6.x-mpy-20200829 and installed.

Here is the CircuitPython test program:
Code: Select all | TOGGLE FULL SIZE
import adafruit_hid # So we can see __version__
from adafruit_hid.keyboard import Keyboard
from adafruit_hid.keyboard_layout_us import KeyboardLayoutUS
from adafruit_hid.keycode import Keycode
from adafruit_hid.mouse import Mouse
import board
from digitalio import DigitalInOut, Direction, Pull
import time
import usb_hid

button1 = DigitalInOut(board.D1)
button1.direction = Direction.INPUT
button1.pull = Pull.UP

button2 = DigitalInOut(board.D0)
button2.direction = Direction.INPUT
button2.pull = Pull.UP

kbd = Keyboard(usb_hid.devices)
layout = KeyboardLayoutUS(kbd)
mouse = Mouse(usb_hid.devices)

print("adafruit_hid.__version__", adafruit_hid.__version__)
# usb_hid - __version__ not available?
# print("usb_hid.__version__", usb_hid.__version__)

while True:
    if not button1.value:
        layout.write("# Hello")
        kbd.send(Keycode.ENTER)
        time.sleep(1)
    if not button2.value:
        # Double-click (not sure what this does on iPad?)
        mouse.click(Mouse.LEFT_BUTTON)
        mouse.click(Mouse.LEFT_BUTTON)
        time.sleep(1)

This works on Mac OS, but not on the iPad.

First ported to Arduino with the Arduino USB stack
Code: Select all | TOGGLE FULL SIZE
#include <Keyboard.h>
#include <Mouse.h>

void setup() {
  pinMode(0, INPUT_PULLUP); // Button1
  pinMode(1, INPUT_PULLUP); // Button2
  Mouse.begin();
  Keyboard.begin();
}

void loop() {
  if (digitalRead(1) == LOW) {
    Keyboard.println("# Hello");
    delay(1000);
  }
  if (digitalRead(0) == LOW) {
    Mouse.click();
    Mouse.click();
    delay(1000);
  }
}

This works on Mac OS X, and on the iPad.

Then I switched to the TinyUSB library, and installed https://github.com/cyborg5/TinyUSB_Mouse_and_Keyboard so that I wouldn't have to change the above much.

The includes were simply changed to:
Code: Select all | TOGGLE FULL SIZE
#include <TinyUSB_Mouse_and_Keyboard.h>

This works on Mac OS X, and still works on the iPad.

kmevans
 
Posts: 44
Joined: Mon Feb 06, 2017 4:32 pm

Re: USB HID on iPad (iOS)

by danhalbert on Sat Aug 29, 2020 7:28 pm

danhalbert wrote:It may require "BOOT" HID support, which we don't handle currently.


I looked at the Arduino library, and it's not a "boot" device, so there's something else that's different, and that iOS doesn't like.

danhalbert
 
Posts: 2118
Joined: Tue Aug 08, 2017 12:37 pm

Re: USB HID on iPad (iOS)

by kmevans on Sat Aug 29, 2020 8:12 pm

I didn't notice the "Please try hid_mouse, hid_keyboard, hid_composite to see which works" quote, but if you can let me know what still makes sense to try then I make changes.

kmevans
 
Posts: 44
Joined: Mon Feb 06, 2017 4:32 pm

Re: USB HID on iPad (iOS)

by danhalbert on Sat Aug 29, 2020 8:24 pm

Please do try hid_composite. iOS may not like the keyboard integrated into a composite device with other devices. I was thinking that might be the issue.

In the past there has also been an issue that the power requirements presented by the device during USB enumeration can be too high, but I believe CIRCUITPY is now working, so I think we fixed that problem.

danhalbert
 
Posts: 2118
Joined: Tue Aug 08, 2017 12:37 pm

Re: USB HID on iPad (iOS)

by kmevans on Sat Aug 29, 2020 8:32 pm

To change to composite, do I need to completely rewrite the above to something resembling this https://github.com/hathach/tinyusb/tree/master/examples/device/hid_composite/src?

kmevans
 
Posts: 44
Joined: Mon Feb 06, 2017 4:32 pm

Re: USB HID on iPad (iOS)

by hathach on Sun Aug 30, 2020 5:45 am

or you can just use the stock hid_composite example to see if that works.

hathach
 
Posts: 1110
Joined: Tue Apr 23, 2013 1:02 am

Re: USB HID on iPad (iOS)

by kmevans on Sun Aug 30, 2020 2:40 pm

I couldn't see all of the examples built into the Arduino environment, so I took the path of least resistance and built in Ubuntu 18.04.1.
Code: Select all | TOGGLE FULL SIZE
git clone https://github.com/hathach/tinyusb.git
git submodule update --init --recursive
cd examples/device/hid_composite
make BOARD=itsybitsy_m4 all uf2
...
Wrote 12288 bytes to _build/build-itsybitsy_m4/itsybitsy_m4-firmware.uf2.

Note that in hw/bsp/itsybitsy_m4/itsybitsy_m4.c there's a small typo in the comment for BUTTON_PIN:
Code: Select all | TOGGLE FULL SIZE
 61 #define LED_PIN      22
 62 #define BUTTON_PIN   18  // pin D5 -> really D7 aka PA18

This works on Mac OS X and on the iPad - if I push a button and force D7 low then it types an 'A' and moves the mouse.

kmevans
 
Posts: 44
Joined: Mon Feb 06, 2017 4:32 pm

Re: USB HID on iPad (iOS)

by danhalbert on Sun Aug 30, 2020 2:49 pm

Thanks for testing. This is an HID composite device, but the whole USB device is a composite MSC, HID, CDC, etc. We probably need to test that too.

danhalbert
 
Posts: 2118
Joined: Tue Aug 08, 2017 12:37 pm

Re: USB HID on iPad (iOS)

by kmevans on Sun Aug 30, 2020 10:33 pm

@hathach - is there any example of something like "composite MSC, HID, CDC, etc." I've never worked with TinyUSB before so it could take me awhile. Perhaps another angle is to rebuild CircuitPython with minor changes (disable CDC,...) and see if it makes a difference. Is it as simple as editing tools/gen_usb_descriptor.py?

kmevans
 
Posts: 44
Joined: Mon Feb 06, 2017 4:32 pm

Re: USB HID on iPad (iOS)

by danhalbert on Sun Aug 30, 2020 11:00 pm

OK, I have figured this out. It's due to the presence of the GAMEPAD HID device in the CircuitPython build. If I remove the GAMEPAD device, then a Trinket M0 works properly as a keyboard on an iPad.

I had forgotten about this problem, which either hathach or I discovered a year or more ago; I can't remember who figured it out.

You can remove the GAMEPAD device from the build for a board by adding a line like this to the mpconfigboard.mk file for the board:

Code: Select all | TOGGLE FULL SIZE
USB_HID_DEVICES=KEYBOARD,MOUSE,CONSUMER


The default is
Code: Select all | TOGGLE FULL SIZE
USB_HID_DEVICES=KEYBOARD,MOUSE,CONSUMER,GAMEPAD


@kmevans - if you would like a custom build of this for the Itsy Bitsy M4, I can make you one for testing.

danhalbert
 
Posts: 2118
Joined: Tue Aug 08, 2017 12:37 pm

Please be positive and constructive with your questions and comments.