Starting to port Forking keyboard driver to Windows

Introduction

 After 20 years of developing a "forking" keyboard driver for Xfree86/Xorg, I decided to finally attempt to make it available on Windows. 

 As a first step I split the code into portable core, and the interfacing glue for Xorg.  In a couple of days I implemented such a glue for the "libinput" to be used in Weston (Wayland).

Plan for Windows

Encouraged by the speedy adaptation, I started to assemble the tools for Windows:

1.  wdk
2.  visual studio
3.  a "target" machine (KVM+Qemu  running Windows). 

Requirements for 3. included public IP for the guest VM (ping should be possible).

virt-install --name win11 --cdrom Win11_24H2_EnglishInternational_x64.iso 

virt-viewer win11

Host machine setup

Installing VS was easy, and included a misleadingly named "wdk-extension" which did not include WDK itself. So  ntddk.h header was missing during the first builds.      Installing WDK properly solved that.

Target machine

Regarding  preparation ... just re-read and execute correctly the steps.

I didn't install any SDK or VStudio on the target PC! Only the WDK.

For "non-enhanced mode" ....I didn't do anything with my KVM/Qemu setup.

WDK Test Target Setup MSI ... indeed provided by WDK. (If by chance you install SDK, you will find other MSI files but those are irrelevant.)

Provisioning made inside VS (on the host) ... sometimes fails. Repeating can fix it.

This create the WDKRemoteUser account (on the target) and logs as it. On one physical machine I couldn't login as it has an unknown password.

Deploying

I started to try-out Sample drivers. In particular the kbfilter. Deploy failed.

• one can see the XML logs... one has to understand that  "Driver Test Group Explorer" tab is sibling of "Git Changes" and "Solution Explorer".
• Or you can make a video of the terminals opened on the target -- they disappear quickly.
  

The message is the same in both "Cannot create a stable subkey under a volatile parent key". Other failures could be present too.

So re-read the Readme and the INF file template

- Set the hardware ID in the inx file

I didn't see any 

> WdfCoinstaller010xx.dll The coinstaller for version 1.xx of KMDF.

Manual build

Using MSBuild ... does work, but does not help with the deployment.

 Manual deployment

So, while searching for solutions of the aforementioned issues I learned:

* dbgview  seems similar to dmesg of linux. But it crashes sometimes (disappears). How to see messages during the boot?
To see kernel message it must be run "as administrator" !

* devcon tool -- mysterious tool installed during the provisioning (by VS)

https://github.com/hawku/TabletDriver/issues/225

I looked at the github issues of the sample drivers -- for example:

about devcon install.

* regedit  .... indeed ... there is a key which is crucial:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\{4D36E96B-E325-11CE-BFC1-08002BE10318}\UpperFilters

But beware .... the format while editing is ..... one item per line

"kbdclass kbfilter" .... this indeed makes it work !!!

Wrong value and you lose keyboard !!!!!

* devmgr one can observe, manually remove.

So, what works?

deploy (from VS). Then  just "Update" the driver of the PS/2 keyboard device...

if it says "Already up-to-date" then your HW-id is wrong!!! So update the *inf with correct value.

and Registry -- irrelevant.

WdfDriverCreate was failing with 0xC000009A so disable the verification.

Story of abandoning own tool in favour of learning multiple projects and combining them

Adopting Debian Installer  (time scale)

In 2016 I started to formalize the way I want to replicate my debian installation on other PCs. I think I needed that once per year.

That is indicated by git commits of a repo with numbered scripts -- stages of installing debian & my expected packages & modifications, after booting with a usb flash drive.

Disk-partitioning was hard (shrinking Windows partitions), configuring apt also forever evolving.

Prior to that I would have just reattach the disks and copied the filesystem.

After 6 years, in 2022 (having used those tools about 10 times) I started to suspect I could spend my time better by learning & fixing a publicly developed tool.

And indeed in April 2023 I tried out the debian-installer (the "hd-media" option), with some pre-seeding.

Then I needed more than pre-seed & targeting unstable/SID: I wanted to understand all possible disk encryption setups, and using my additional apt repository (+ apt configuration).

Experiments

So, I was using the build system in debian-installer (to build customizations) and decided to use a virtual PC (KVM-qemu) to quickly test the result.

I had to overcome bug 940801

I could test the partitioning with encryption on a virtual disk (10GB).

• Started to wonder about having "udeb" packages in my reprepro.
• Not really sure what is inside "mini.iso", maybe all is inside the initrd?

Setting up PXE

I wanted to also avoid using flash drives/sd-cards  -- using PXE seems more elegant. Also using netboot rather than hd-media seems a nice simplification.

But there's a catch: this requires modifying DHCP server (which I cannot do on my main modem/route), and I didn't want to set a separate dhcp just for this use-case (on a linux box, with the only connection to the booting PC).

Anyway, I started still with KVM -- a new "bridge" with dhcp pointing at local tftp.

I had to use signed grub to boot with secure-boot.

Going for physical PC

So I installed dd-wrt on another router/wifi access point, and detached from its "bridge" between all LAN connectors one connector which is then serviced by a new dhcp server, and advertises the necessary option for PXE boot.

 

First successful run

I included a custom udeb: I needed a modified "choose-mirror" to debug still flaky pre-seed configuration of the net interface.

Partitioning seems intuitive (once you want to use the same key for both Swap & Root fs).

Choose the minimal (ssh server, standard), and after the reboot  do still some manual copying, apt configuration, and finally install my  "top" virtual packages.

логическая загадка

Есть 5 космических кораблей (из разных стран мира). У каждого есть номер (1-5), цвет, маскот и каждый привозит сувенир из Луны. Они не стартуют в своей стране но приземляются в ней.  Нам известны следующие факты: 

 

1. Синий корабль стартует в Хирошиме (Япония)
2. Русский корабль привезёт с Луны "флаг США" (как сувенир) 
3. В Люнбурге (в каком же государстве ... дочитать до конца и узнаешь) стартует корабль номер 1. 
4. Кошка путешествует в корабле с номером на 1 меньше чем номер золотого корабля. 
5. Соловей является маскотом корабля, с номером на 1 больше чем у зелёного корабля. 
6. Корабль со средним номером привозит "лунарный пыл" (как сувенир). 
7. Улитка маскот корабля, который стартует на Байконуре (будем считать городом в России). 
8. Корабль который стартовал на мысе Каневерале (США) привезет "старую лунарную камеру" (=сувенир). 
9. Маскотом на красном корабле является жаба. 
10. Зелённый корабль принадлежит Китае. 
11. У Японии номер 2. 
12. Серебристый корабль принадлежит Германии. 
13. Корабль США стартует в Пекинге. 
14. Германия привезёт в Люнбург с Луны "космическоя авто" и имеет номер на 1 меньше чем Россия. 
    

 

Вся эта информация хватит для того, чтобы ответить на следующие 2 вопроса:

1. A. Который корабль имеет зебру как маскота? 
2. B. Который корабль привезёт "женшину" как сувенир?
   

 Working on Proof-of-concept

Based on my git segments/sums, I can construct a 

poset of modifications (micro-features), and 

study them in isolation. For the whole-picture study I use (modified) xdot with an extension to handle "gitk://" actions, so I can open gitk (restricted to the segment history) by clicking on single nodes:

git-hierarchy contains the "git-graph -x" command to open such a graph.

Keeping up with debian Sid

My small build system

I need to replace some distro packages with my improvements.

For this I need to patch the sources -- I only handle git. I need to be notified, warned, when a new version is needed.

So I patched apt to report when a switch from my version to the upstream would happen (due to  "apt dist-upgrade").

In such a case I run my  git-rebase-origin which rebases my feature poset (of feature branches/segments mixed into "merges") over a refreshed remote branch.

Then I run my scripts to invoke gbp(1) with a suitable version string, and create the deb packages & put them in reprepro.

the sequence of commands to maintain apt itself:


$ cd ~/git/apt/
$ git-rebase-origin  
# this deduces the upstream -- base of my feature branches.

$ release
# this detects I'm building my extension of upstream debian package, so deduces the correct version, build, uploads, tags, pushes, installs.

When I only want to test some changes, I run 

$ snap

which creates a .deb package, installs it, withouth any reprepro etc.

Logický test

5 lodí, každá má číslo, barvu, maskota a suvenýr z Měsíce.
Nestartují ze své země, ale přistávají v ní.

1. Modrá loď startuje v Hirošimě
2. SSSR přiveze z Měsíce vlajku USA
3. v Lunburgu startuje loď s číslem 1.
4. Kočka cestuje v lodi s číslem o 1 menším než je číslo zlaté lodi.
5 Slavík je maskotem lodi, jejíž start. číslo je o 1 vyšší než číslo zelené lodi.
6 Loď s prostředním číslem veze zpět měsíční prach.
7 Hlemýžd je maskot lodi, jež startuje na Bajkonuru.
8 Stát, který odstartoval na mysu Caneveral, příveze zpět starou měsíční kameru.
9 Maskot v červené lodi je žába.
10 Zelená loď patří Číně.
11 Japonsko má startovní číslo 2.
12 Stříbrná loď patří NSR.
13 Loď USA startuje v Pekingu.
14 NSR přiveze do Lunburgu z měsíce kosmické auto a má startovní číslo o 1 nižší než SSSR.


Tyto infomace stačí k tomu, abychom zodpověděli dvě otázky:

A. Která loď má jako maskota zebru?
B. Která loď přiveze zpět z měsíce ženu?

Upgrading Nubia 7 mini

I recently bought this  phone on marktplaats, already upgraded to Mokee Android 7. I liked the features, my previous phone used Android 4.2.
So I bought another one for my wife, this time I had to make the upgrade myself. It was not straightforward:

The first nubia used as recovery  TWRP 3.0.2-0, the other is with clockwork-mod (v6.0.5.0).

• I had to resize the big "grow" partition manually, i.e. merge it with "userdata".
• suddenly I saw disconnections of Wifi, and that was due to the same MAC address of the 2 nubias.

MAC address

So I searched for a solution to tune the MAC address. I found the hint on Intf0MacAddress in WCNSS_qcom_cfg.ini. But it did not work.

/data/misc/wifi/WCNSS_qcom_cfg.ini  nor  /system/etc/wifi/WCNSS_qcom_cfg.ini

So I had a workaround to

echo 98:6c:f5:63:ec:d0 > /sys/devices/fb000000.qcom,wcnss-wlan/wcnss_mac_addr

while in airplane mode.
In the end I  found /persist/WCNSS_qcom_cfg.ini which indeed works, and overrides the other 2 files.

ANDROID_SERIAL to distinguish them?

To have them accessible via adb at the same time, I need to distinguish by this serial number, which is unfortunately identical. For now I overwrite the kernel cmdline in img_info with "androidboot.serialno=XXX"

Proximity sensor

After receiving some (whatsapp) calls, I noticed it was almost impossible to get the screen back on, as expected.

So, as a workaround I turned off  in Phone> Settings> Accessibility> Use proximity sensor.

But Whatsapp does not seem to have this option.

So I installed Sensors test and some app to "reset/calibrate" but to no effect.

So I started to look for the kernel driver, to fix a possible bug. There are confusingly multiple kernel sources on github:

1. https://github.com/nx507j-dev/android_kernel_nubia_nx507j
2. https://github.com/ztemt/NX507J_5.1_kernel
3. https://github.com/MoKee/android_kernel_nubia_nx507j.git
4. https://github.com/MoKee/android_kernel_zte_nx507j.git

The 3rd is the one used on Mokee (my fixes in https://github.com/mmaruska/android_kernel_nubia_nx507j). The other ones fail when (Android) "init" configures (selinux) sepolicy.

Btw. failures of the other kernels can be seen thanks to android ram-console: after the failure, the phone reboots to recovery and 

adb shell cat /proc/last_kmsg

shows the problem.

In the end the problems is only the calibration, so 

 echo 500 > prox_threshold_low  

fixes the issue. To see this value, I did:

cd /sys/class/proximity/proximity/
echo 1 > prox_debug

and "dmesg -c" while running the sensor test app, to see:

[43580.286869] [SENSOR_ALS_PROX] [taos_prox_threshold_set: 2338] proxdata = 396

I guess it's stored in /persist/proxdata/threshold ?

Other issues

calendar/alarm database -- crash when entering the Alarm app:

android.database.sqlite.SQLiteException: Can't downgrade database from version 12 to 8

I removed the relevant file.

SystemUI crash due to wallpaper using too big image

java.lang.RuntimeException: Canvas: trying to draw too large(143769600bytes) bitmap.
....
AppCrashReceiver: com.android.systemui stopped unexpectedly...

rm  /data/system/user/0/wallpaper_*



My improvements (divided into feature-segments)  https://github.com/mmaruska/android_kernel_nubia_nx507j/network