USB HID on iPad (iOS) Moderators: adafruit_support_bill, adafruit

[kmevans]

Has anyone gotten USB keyboard/mouse HID to work on an iPad or any other iOS device? I've tried both CircuitPython 6.0.0 alpha 2 and the older 5.3 and am able to see mass storage mount via the Files app, but the USB HID keyboard/mouse functionality doesn't appear to work. I am using a USB C to 3-port USB 3.0 Hub to connect. (iHaper C003.) I plugged in an old USB keyboard (HHKB Lite 2) and that worked fine.

I'm using an ItsyBitsy M4 Express. The same setup worked fine with a OS X, and with a Raspberry Pi. The iPad is an iPad pro with iOS 13.6.1.

[siddacious]

If I recall correctly, it took convincing Apple to make some tweaks to IOS to make mass storage work correctly so I'm not all that surprised that HID doesn't just work.

I'm not certain though, so I'll reach out to folks that would know better than me.

[danhalbert]

It may require "BOOT" HID support, which we don't handle currently.

[hathach]

would you mind trying out the TinyUSB library on Arduino to see if it works. It is the same library used by circuitpython, but have user-configurable hid descriptor which make thing easier to try out. Please try hid_mouse, hid_keyboard, hid_composite to see which works

https://github.com/adafruit/Adafruit_Ti ... amples/HID

You will need to switch the USB Stack from Arduino IDE menu from Arduino -> TinyUSB

[kmevans]

Ok let me first try the absolute latest releases of everything for CircuitPython, and then port to Arduino. I'll post sample code and results.

[kmevans]

First for reference I verified that a Logitech K270 mouse/keyboard plugged into an iHaper USB C003 USB-C to 3-Port USB 3.0 Hub works and provides about mouse and keyboard functionality. I used the Notes iOS app for testing.

* CircuitPython on ItsyBitsy M4 Express *

CircuitPython 6.0.0 Alpha 3 was announced here:
https://forums.adafruit.com/viewtopic.php?f=60&t=168889
But I couldn't find it here:
https://circuitpython.org/board/itsybitsy_m4_express/
So I'm sticking with 6.0.0 Alpha 2.

Code: Select all | TOGGLE FULL SIZE

Adafruit CircuitPython 6.0.0-alpha.2 on 2020-07-23; Adafruit ItsyBitsy M4 Express with samd51g19

I downloaded the latest libraries in adafruit-circuitpython-bundle-6.x-mpy-20200829 and installed.

Here is the CircuitPython test program:

Code: Select all | TOGGLE FULL SIZE

import adafruit_hid # So we can see __version__
from adafruit_hid.keyboard import Keyboard
from adafruit_hid.keyboard_layout_us import KeyboardLayoutUS
from adafruit_hid.keycode import Keycode
from adafruit_hid.mouse import Mouse
import board
from digitalio import DigitalInOut, Direction, Pull
import time
import usb_hid

button1 = DigitalInOut(board.D1)
button1.direction = Direction.INPUT
button1.pull = Pull.UP

button2 = DigitalInOut(board.D0)
button2.direction = Direction.INPUT
button2.pull = Pull.UP

kbd = Keyboard(usb_hid.devices)
layout = KeyboardLayoutUS(kbd)
mouse = Mouse(usb_hid.devices)

print("adafruit_hid.__version__", adafruit_hid.__version__)
# usb_hid - __version__ not available?
# print("usb_hid.__version__", usb_hid.__version__)

while True:
    if not button1.value:
        layout.write("# Hello")
        kbd.send(Keycode.ENTER)
        time.sleep(1)
    if not button2.value:
        # Double-click (not sure what this does on iPad?)
        mouse.click(Mouse.LEFT_BUTTON)
        mouse.click(Mouse.LEFT_BUTTON)
        time.sleep(1)

This works on Mac OS, but not on the iPad.

First ported to Arduino with the Arduino USB stack

Code: Select all | TOGGLE FULL SIZE

#include <Keyboard.h>
#include <Mouse.h>

void setup() {
  pinMode(0, INPUT_PULLUP); // Button1
  pinMode(1, INPUT_PULLUP); // Button2
  Mouse.begin();
  Keyboard.begin();
}

void loop() {
  if (digitalRead(1) == LOW) {
    Keyboard.println("# Hello");
    delay(1000);
  }
  if (digitalRead(0) == LOW) {
    Mouse.click();
    Mouse.click();
    delay(1000);
  }
}

This works on Mac OS X, and on the iPad.

Then I switched to the TinyUSB library, and installed https://github.com/cyborg5/TinyUSB_Mouse_and_Keyboard so that I wouldn't have to change the above much.

The includes were simply changed to:

Code: Select all | TOGGLE FULL SIZE

#include <TinyUSB_Mouse_and_Keyboard.h>

This works on Mac OS X, and still works on the iPad.

[danhalbert]

It may require "BOOT" HID support, which we don't handle currently.

I looked at the Arduino library, and it's not a "boot" device, so there's something else that's different, and that iOS doesn't like.

[kmevans]

I didn't notice the "Please try hid_mouse, hid_keyboard, hid_composite to see which works" quote, but if you can let me know what still makes sense to try then I make changes.

[danhalbert]

Please do try hid_composite. iOS may not like the keyboard integrated into a composite device with other devices. I was thinking that might be the issue.

In the past there has also been an issue that the power requirements presented by the device during USB enumeration can be too high, but I believe CIRCUITPY is now working, so I think we fixed that problem.

[kmevans]

To change to composite, do I need to completely rewrite the above to something resembling this https://github.com/hathach/tinyusb/tree/master/examples/device/hid_composite/src?

[hathach]

or you can just use the stock hid_composite example to see if that works.

[kmevans]

I couldn't see all of the examples built into the Arduino environment, so I took the path of least resistance and built in Ubuntu 18.04.1.

Code: Select all | TOGGLE FULL SIZE

git clone https://github.com/hathach/tinyusb.git
git submodule update --init --recursive
cd examples/device/hid_composite
make BOARD=itsybitsy_m4 all uf2
...
Wrote 12288 bytes to _build/build-itsybitsy_m4/itsybitsy_m4-firmware.uf2.

Note that in hw/bsp/itsybitsy_m4/itsybitsy_m4.c there's a small typo in the comment for BUTTON_PIN:

Code: Select all | TOGGLE FULL SIZE

 61 #define LED_PIN      22
 62 #define BUTTON_PIN   18  // pin D5 -> really D7 aka PA18

This works on Mac OS X and on the iPad - if I push a button and force D7 low then it types an 'A' and moves the mouse.

[danhalbert]

Thanks for testing. This is an HID composite device, but the whole USB device is a composite MSC, HID, CDC, etc. We probably need to test that too.

[kmevans]

@hathach - is there any example of something like "composite MSC, HID, CDC, etc." I've never worked with TinyUSB before so it could take me awhile. Perhaps another angle is to rebuild CircuitPython with minor changes (disable CDC,...) and see if it makes a difference. Is it as simple as editing tools/gen_usb_descriptor.py?

[danhalbert]

OK, I have figured this out. It's due to the presence of the GAMEPAD HID device in the CircuitPython build. If I remove the GAMEPAD device, then a Trinket M0 works properly as a keyboard on an iPad.

I had forgotten about this problem, which either hathach or I discovered a year or more ago; I can't remember who figured it out.

You can remove the GAMEPAD device from the build for a board by adding a line like this to the mpconfigboard.mk file for the board:

Code: Select all | TOGGLE FULL SIZE

USB_HID_DEVICES=KEYBOARD,MOUSE,CONSUMER

The default is

Code: Select all | TOGGLE FULL SIZE

USB_HID_DEVICES=KEYBOARD,MOUSE,CONSUMER,GAMEPAD

@kmevans - if you would like a custom build of this for the Itsy Bitsy M4, I can make you one for testing.