Strangeness when sending raw HID keyboard reports with Bluef

General project help for Adafruit customers

Moderators: adafruit_support_bill, adafruit

Please be positive and constructive with your questions and comments.
Locked
User avatar
bcg
 
Posts: 11
Joined: Sun Oct 16, 2011 11:13 pm

Strangeness when sending raw HID keyboard reports with Bluef

Post by bcg »

Hi guys

I've been playing around with my Bluefruit (the one that doesn't support mouse) and so far I really like it. I'm almost finished with my project but I'm experiencing a bug that I can't solve and I'm hoping to rule out a problem or limitation with the Bluefruit itself.

I've taken an IBM Model M terminal keyboard and hooked it up to a atmega 32u4 using the awesome tmk_keyboard firmware by @tmk (https://github.com/tmk/tmk_keyboard)

I've reimplemented the output protocol to send the generated HID reports out to the Bluefruit over serial. The project is powered by the USB connection which is hooked up to a PC, and the 32u4 is running PJRC's "usb_debug_only" stack so that I can get info on what is going on inside the program. Eventually I will probably get your usb lipo charger and battery and make it so that I can toggle between Bluetooth and USB or even just charge the keyboard while I'm using Bluetooth.

Anyhow, those details of my project will be chronicled here as I progress further along: http://geekhack.org/index.php?topic=50183.0

Also at that link are some details relating to my question for you, which I will reiterate below:

Everything works fine as long as I only press one key at a time... if I press two, things get a little strange, I'll try to describe this as best I can...

If I press two keys at once, the first key is typed on my PC just fine, as is the second key. However if I release the first key, another HID report gets sent to and the other key gets typed out again (as if there was a key up in between). To better illustrate here is the debug output from pressing and holding 'g', then pressing and holding 'h', then releasing 'h', then releasing 'g':

Code: Select all

Waiting for device:.....
Listening:
Setting host driver...
Initializing serial...
Beginning main loop
rAA wFF [ack]
RESET_RESPONSE: rBF err
RESET: rBF wFF x01wFF x01wFF x01wFF x01wFF [ack]
RESET_RESPONSE: rAA [ok]
KBD_ID: rBF rBF 
CONFIG: wF8 [ack]
READY
r34 
+------------------------------------+
| HID report to Bluefruit via serial |
+------------------------------------+
| FD  00  00  0A  00  00  00  00  00 |
+------------------------------------+

gr33 
+------------------------------------+
| HID report to Bluefruit via serial |
+------------------------------------+
| FD  00  00  0A  0B  00  00  00  00 |
+------------------------------------+

ghrF0  r33 
+------------------------------------+
| HID report to Bluefruit via serial |
+------------------------------------+
| FD  00  00  0A  00  00  00  00  00 |
+------------------------------------+

grF0  r34 
+------------------------------------+
| HID report to Bluefruit via serial |
+------------------------------------+
| FD  00  00  00  00  00  00  00  00 |
+------------------------------------+
Each of these reports is sent to the Bluefruit verbatim, and as far as I can tell they look right based on the tutorial here: http://learn.adafruit.com/introducing-b ... via-serial

Two things seem to be wrong here:

1) When the second key is released and I send the updated HID report, whatever keys are still pressed get typed out a second time
2) When I hold down a key, it doesn't seem as if the computer is recognizing that the key is held down

In both cases it seems like the PC perceives that the held down keys were released when they in fact were not.

I hope I explained this well, I know it was verbose but I hope that I made it clear enough.

If you need to see any of my code just let me know, I will make it open source once its further along but in the meantime I can post whatever snippets you need to see here.

Thanks

User avatar
adafruit_support_rick
 
Posts: 35092
Joined: Tue Mar 15, 2011 11:42 am

Re: Strangeness when sending raw HID keyboard reports with B

Post by adafruit_support_rick »

I've passed your question along to the bluefruit engineering team.

adafruit
 
Posts: 12151
Joined: Thu Apr 06, 2006 4:21 pm

Re: Strangeness when sending raw HID keyboard reports with B

Post by adafruit »

When we run this code

Code: Select all

#include <SoftwareSerial.h>
SoftwareSerial BT = SoftwareSerial(3, 2);

void keyCommand(uint8_t modifiers, uint8_t keycode1, uint8_t keycode2 = 0, uint8_t keycode3 = 0, 
uint8_t keycode4 = 0, uint8_t keycode5 = 0, uint8_t keycode6 = 0) {
  BT.write(0xFD);       // our command
  BT.write(modifiers);  // modifier!
  BT.write((byte)0x00); // 0x00  
  BT.write(keycode1);   // key code #1
  BT.write(keycode2); // key code #2
  BT.write(keycode3); // key code #3
  BT.write(keycode4); // key code #4
  BT.write(keycode5); // key code #5
  BT.write(keycode6); // key code #6
}

void setup() {
  Serial.begin(9600);
  BT.begin(9600);

  delay(5000);
  keyCommand(0,0x0a,0x00,0,0,0,0);
  delay(2000);
  keyCommand(0,0x0a,0x0b,0,0,0,0);
  delay(2000);
  keyCommand(0,0x0a,0x00,0,0,0,0);
  delay(2000);
  keyCommand(0,0x00,0x00,0,0,0,0);
}

void loop() {}

the output we get: ggggggggggggggggggggggghhhhhhhhhhhhhhhhhhhh
Is exactly the same as what you get if you press and hold 'g' on the keyboard, then press and hold 'h' and then release 'h' - 'g' doesnt continue to repeat, this is an OS thing, not a bluetooth/bluefruit thing!

User avatar
bcg
 
Posts: 11
Joined: Sun Oct 16, 2011 11:13 pm

Re: Strangeness when sending raw HID keyboard reports with B

Post by bcg »

Thank you for your reply, I appreciate you looking into this.

The behavior you're describing is exactly what I'd like to see happen. Your example also gave me an idea to troubleshoot this.

When I connect the IO pins to ground as you describe, I get exactly the behavior that you mentioned. So at least I know it can work.

I decided to sniff the messages for the bluefruit by looking at the input from the bluetooth dongle in usbmon; so if I connect pin 11 to ground, then pin add pin 9, I get this output as keystrokes

Code: Select all

ddddddddddddddddddddddaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
In usbmon I get this (I added comments to denote each 'keystroke'):

Code: Select all

sudo cat /sys/kernel/debug/usb/usbmon/4u
eef81cc0 2818296287 C Bi:4:113:2 0 18 = 15200e00 0a004100 a1010000 07000000 0000
eef81cc0 2818296303 S Bi:4:113:2 -115 1028 <  # pin 11 connected to ground
eef813c0 2820921294 C Bi:4:113:2 0 18 = 15200e00 0a004100 a1010000 04070000 0000
eef813c0 2820921310 S Bi:4:113:2 -115 1028 <  # pin 9 connected to ground
eef81cc0 2823608290 C Bi:4:113:2 0 18 = 15200e00 0a004100 a1010000 07000000 0000
eef81cc0 2823608303 S Bi:4:113:2 -115 1028 <  # pin 9 disconnected
eef813c0 2825084291 C Bi:4:113:2 0 18 = 15200e00 0a004100 a1010000 00000000 0000
eef813c0 2825084305 S Bi:4:113:2 -115 1028 <  # pin 11 disconnected
So there were 4 reports, as I'd expect from those actions (key down, key down, key up, key up)

Next I ported your example to C and tried it again:

Code: Select all

#include <stdint.h>
#include <avr/interrupt.h>
#include <avr/wdt.h>
#include <avr/sleep.h>
#include <util/delay.h>
#include "serial.h"

void keyCommand(uint8_t, uint8_t, uint8_t, uint8_t, uint8_t, uint8_t, uint8_t); 

void keyCommand(uint8_t modifiers, uint8_t keycode1, uint8_t keycode2, uint8_t keycode3, uint8_t keycode4, uint8_t keycode5, uint8_t keycode6) {
    serial_send(0xFD);       // our command
    serial_send(modifiers);  // modifier!
    serial_send(0x00); // 0x00 
    serial_send(keycode1);   // key code #1
    serial_send(keycode2); // key code #2
    serial_send(keycode3); // key code #3
    serial_send(keycode4); // key code #4
    serial_send(keycode5); // key code #5
    serial_send(keycode6); // key code #6
}

int main(void)
{

    DDRD = _BV(PD5);
    DDRB = _BV(PB0);
    
    PORTD = _BV(PD5);
    PORTB = _BV(PB0);

    serial_init();

    _delay_ms(1000);
 
    PORTD = ~_BV(PD5);
    while (1) {
	_delay_ms(5000);
        keyCommand(0, 0x0a, 0x00, 0, 0, 0, 0);
        _delay_ms(2000);
        keyCommand(0, 0x0a, 0x0b, 0, 0, 0, 0);
        _delay_ms(2000);
        keyCommand(0, 0x0a, 0x00, 0, 0, 0, 0);
        _delay_ms(2000);
        keyCommand(0, 0x00, 0x00, 0, 0, 0, 0);
    }
}
and I received output like this for each iteration of the loop:

Code: Select all

gghg
And the usbmon output looks like this for 1 cycle (8 reports this time, instead of 4 like before):

Code: Select all

sudo cat /sys/kernel/debug/usb/usbmon/4u
eef813c0 2528214491 C Bi:4:113:2 0 18 = 15200e00 0a004100 a1010000 0a000000 0000
eef813c0 2528214507 S Bi:4:113:2 -115 1028 <
eef81cc0 2528275492 C Bi:4:113:2 0 18 = 15200e00 0a004100 a1010000 00000000 0000
eef81cc0 2528275504 S Bi:4:113:2 -115 1028 <
eef813c0 2530213493 C Bi:4:113:2 0 18 = 15200e00 0a004100 a1010000 0a0b0000 0000
eef813c0 2530213505 S Bi:4:113:2 -115 1028 <
eef81cc0 2530251493 C Bi:4:113:2 0 18 = 15200e00 0a004100 a1010000 00000000 0000
eef81cc0 2530251503 S Bi:4:113:2 -115 1028 <
eef813c0 2532233493 C Bi:4:113:2 0 18 = 15200e00 0a004100 a1010000 0a000000 0000
eef813c0 2532233507 S Bi:4:113:2 -115 1028 <
eef81cc0 2532241492 C Bi:4:113:2 0 18 = 15200e00 0a004100 a1010000 00000000 0000
eef81cc0 2532241502 S Bi:4:113:2 -115 1028 <
eef813c0 2534249495 C Bi:4:113:2 0 18 = 15200e00 0a004100 a1010000 00000000 0000
eef813c0 2534249507 S Bi:4:113:2 -115 1028 <
eef81cc0 2534251487 C Bi:4:113:2 0 18 = 15200e00 0a004100 a1010000 00000000 0000
eef81cc0 2534251492 S Bi:4:113:2 -115 1028 <
Finally, just to make sure, I uploaded your sketch to my Arduino Uno, and I still received the same results!
This is typed out:

Code: Select all

gghg
And usbmon records this (8 reports again):

Code: Select all

sudo cat /sys/kernel/debug/usb/usbmon/4u
eef81cc0 3666948615 C Bi:4:113:2 0 18 = 15200e00 0a004100 a1010000 0a000000 0000
eef81cc0 3666948634 S Bi:4:113:2 -115 1028 <
eef813c0 3666948641 C Bi:4:113:2 0 18 = 15200e00 0a004100 a1010000 00000000 0000
eef813c0 3666948643 S Bi:4:113:2 -115 1028 <
eef81cc0 3668940612 C Bi:4:113:2 0 18 = 15200e00 0a004100 a1010000 0a0b0000 0000
eef81cc0 3668940626 S Bi:4:113:2 -115 1028 <
eef813c0 3668978613 C Bi:4:113:2 0 18 = 15200e00 0a004100 a1010000 00000000 0000
eef813c0 3668978624 S Bi:4:113:2 -115 1028 <
eef81cc0 3670968613 C Bi:4:113:2 0 18 = 15200e00 0a004100 a1010000 0a000000 0000
eef81cc0 3670968626 S Bi:4:113:2 -115 1028 <
eef813c0 3670969609 C Bi:4:113:2 0 18 = 15200e00 0a004100 a1010000 00000000 0000
eef813c0 3670969616 S Bi:4:113:2 -115 1028 <
eef81cc0 3672984613 C Bi:4:113:2 0 18 = 15200e00 0a004100 a1010000 00000000 0000
eef81cc0 3672984626 S Bi:4:113:2 -115 1028 <
eef813c0 3673022615 C Bi:4:113:2 0 18 = 15200e00 0a004100 a1010000 00000000 0000
eef813c0 3673022627 S Bi:4:113:2 -115 1028 <
Sorry about the length and for being a pain... I just really like this product and would like to get it working and
show others how to do the same thing.

One last thing... do you think it is a problem that I am trying to do this with the version 1.0 of Bluefruit? I know
that mentions a version 1.1 in the tutorial so I'm just curious.

So in summary it appears that the OS should be able to support the functionality I'm describing, but for some reason it seems as if when I send a report to bluefruit it sends 2 commands instead of one, something like 'key down, key up' when I am expecting it will only send the key down (or key up, depending on circumstances of course)

Thanks again for looking into this

adafruit
 
Posts: 12151
Joined: Thu Apr 06, 2006 4:21 pm

Re: Strangeness when sending raw HID keyboard reports with B

Post by adafruit »

OK lets try sending you a v1.1 - in case that changes things! please email support@adafruit and request a new v1.1 when they come back into stock (within next 2 weeks)

User avatar
bcg
 
Posts: 11
Joined: Sun Oct 16, 2011 11:13 pm

Re: Strangeness when sending raw HID keyboard reports with B

Post by bcg »

adafruit wrote:OK lets try sending you a v1.1 - in case that changes things! please email support@adafruit and request a new v1.1 when they come back into stock (within next 2 weeks)
Thanks guys will do. Adafruit rocks!

User avatar
bcg
 
Posts: 11
Joined: Sun Oct 16, 2011 11:13 pm

Re: Strangeness when sending raw HID keyboard reports with B

Post by bcg »

Received the Bluefruit 1.2, dropped it into my circuit and it worked perfectly. My keyboard is working great now, and I also was able to enable the mouse support in the tmk firmware library that I'm using. This product is an absolute home run IMO

Anyhow one last question - in the description for the 1.2 version it says that you can map consumer keys to the IO pins... was wondering if it is also support to send consumer HID reports. If not then no sweat, I can just map the IO pins to the volume controls or whatever and use those instead.

As far as my project goes I just have a little bit of code cleanup and I also need to add a battery as well as implement switching between USB and bluetooth... but all in all I should be able to get this up on Github and write a tutorial in case anyone is curious what I did

Thanks again guys

Deqing
 
Posts: 37
Joined: Wed Nov 13, 2013 12:18 pm

Re: Strangeness when sending raw HID keyboard reports with B

Post by Deqing »

bcg wrote:Received the Bluefruit 1.2, dropped it into my circuit and it worked perfectly. My keyboard is working great now, and I also was able to enable the mouse support in the tmk firmware library that I'm using. This product is an absolute home run IMO

Anyhow one last question - in the description for the 1.2 version it says that you can map consumer keys to the IO pins... was wondering if it is also support to send consumer HID reports. If not then no sweat, I can just map the IO pins to the volume controls or whatever and use those instead.

As far as my project goes I just have a little bit of code cleanup and I also need to add a battery as well as implement switching between USB and bluetooth... but all in all I should be able to get this up on Github and write a tutorial in case anyone is curious what I did

Thanks again guys
Yes you can send although it is not in document yet.

Command for consumer keys are:
FD 00 02 ?? ?? 00 00 00 00
Question marks are bit map of
"Home", "KeyboardLayout", "Search", "Snapshot", "VolumeUp", "VolumeDown", "Play/Pause", "Fast Forward", "Rewind","Scan Next Track", "Scan Previous Track", "Random Play","Stop",

For example, if you want to hold home button:
FD 00 02 01 00 00 00 00 00
Then send
FD 00 02 00 00 00 00 00 00
to release it.

Or Stop
FD 00 02 00 10 00 00 00 00
Then send
FD 00 02 00 00 00 00 00 00
to release it.

You can hold multiple keys simultaneously with an "OR" logic because it is bit map.

User avatar
bcg
 
Posts: 11
Joined: Sun Oct 16, 2011 11:13 pm

Re: Strangeness when sending raw HID keyboard reports with B

Post by bcg »

Thanks guys, because of your help I was able to get this working. Here's the code I ended up using to send the consumer report:

Code: Select all

/*
+-----------------+-------------------+-------+
| Consumer Key    | Bit Map           | Hex   |
+-----------------+-------------------+-------+
| Home            | 00000001 00000000 | 01 00 |
| KeyboardLayout  | 00000010 00000000 | 02 00 |
| Search          | 00000100 00000000 | 04 00 |
| Snapshot        | 00001000 00000000 | 08 00 |
| VolumeUp        | 00010000 00000000 | 10 00 |
| VolumeDown      | 00100000 00000000 | 20 00 |
| Play/Pause      | 01000000 00000000 | 40 00 |
| Fast Forward    | 10000000 00000000 | 80 00 |
| Rewind          | 00000000 00000001 | 00 01 |
| Scan Next Track | 00000000 00000010 | 00 02 |
| Scan Prev Track | 00000000 00000100 | 00 04 |
| Random Play     | 00000000 00001000 | 00 08 |
| Stop            | 00000000 00010000 | 00 10 |
+-------------------------------------+-------+
*/
#define CONSUMER2BLUEFRUIT(usage) \
    (usage == AUDIO_MUTE           ? 0x0000  : \
    (usage == AUDIO_VOL_UP         ? 0x1000  : \
    (usage == AUDIO_VOL_DOWN       ? 0x2000  : \
    (usage == TRANSPORT_NEXT_TRACK ? 0x0002  : \
    (usage == TRANSPORT_PREV_TRACK ? 0x0004  : \
    (usage == TRANSPORT_STOP       ? 0x0010  : \
    (usage == TRANSPORT_STOP_EJECT ? 0x0000  : \
    (usage == TRANSPORT_PLAY_PAUSE ? 0x4000  : \
    (usage == AL_CC_CONFIG         ? 0x0000  : \
    (usage == AL_EMAIL             ? 0x0000  : \
    (usage == AL_CALCULATOR        ? 0x0000  : \
    (usage == AL_LOCAL_BROWSER     ? 0x0000  : \
    (usage == AC_SEARCH            ? 0x0400  : \
    (usage == AC_HOME              ? 0x0100  : \
    (usage == AC_BACK              ? 0x0000  : \
    (usage == AC_FORWARD           ? 0x0000  : \
    (usage == AC_STOP              ? 0x0000  : \
    (usage == AC_REFRESH           ? 0x0000  : \
    (usage == AC_BOOKMARKS         ? 0x0000  : 0)))))))))))))))))))

static void send_consumer(uint16_t data)
{
    static uint16_t last_data = 0;
    if (data == last_data) return;
    last_data = data;
    
    uint16_t bitmap = CONSUMER2BLUEFRUIT(data);
    
#ifdef BLUEFRUIT_TRACE_SERIAL	
	dprintf("\nData: "); 
	debug_hex16(data); 
	dprintf("; bitmap: "); 
	debug_hex16(bitmap); 
	dprintf("\n");
	dprintf("+------------------------------------+\n| HID report to Bluefruit via serial |\n+------------------------------------+\n|");
#endif
	bluefruit_serial_send(0xFD);
	bluefruit_serial_send(0x00);
	bluefruit_serial_send(0x02);
	bluefruit_serial_send((bitmap>>8)&0xFF);
	bluefruit_serial_send(bitmap&0xFF);	
	bluefruit_serial_send(0x00);
	bluefruit_serial_send(0x00);
	bluefruit_serial_send(0x00);
	bluefruit_serial_send(0x00);
#ifdef BLUEFRUIT_TRACE_SERIAL
	dprintf("|\n+------------------------------------+\n\n");
#endif
}
Also... one last question (I hope!) - do you have any plans to get this device certified in Japan or other countries as well as the US?

Thanks for all your help with this

User avatar
AVROduino
 
Posts: 7
Joined: Mon Aug 25, 2014 4:49 pm

Re: Strangeness when sending raw HID keyboard reports with B

Post by AVROduino »

I'm trying to hold down a modifier key using an Arduino and the EZKey.
I want to press on a GPIO pin switch and have the modifier "latch".

And then later send a command to release.

Its pretty straightforward on a 32u4 with Keyboard.press and Keyboard.release.
And of course Keyboard.write.

I tried changing in the keyCommand(),
BT.write to BT.press in the following.

#include <SoftwareSerial.h>
SoftwareSerial BT = SoftwareSerial(8, 9);

void setup()
{
pinMode(switchPin, INPUT_PULLUP);
BT.begin(9600);
}

void keyCommand(uint8_t modifiers, uint8_t keycode1, uint8_t keycode2 = 0, uint8_t keycode3 = 0,
uint8_t keycode4 = 0, uint8_t keycode5 = 0, uint8_t keycode6 = 0)
{
BT.write(0xFD); // our command
BT.write(modifiers); // modifier!
BT.write((byte)0x00); // 0x00
BT.write(keycode1); // key code #1
BT.write(keycode2); // key code #2
BT.write(keycode3); // key code #3
BT.write(keycode4); // key code #4
BT.write(keycode5); // key code #5
BT.write(keycode6); // key code #6
}

If you have any suggestions, please let me know.

Thanks.

User avatar
adafruit_support_rick
 
Posts: 35092
Joined: Tue Mar 15, 2011 11:42 am

Re: Strangeness when sending raw HID keyboard reports with B

Post by adafruit_support_rick »


Locked
Please be positive and constructive with your questions and comments.

Return to “General Project help”