Add --ble-adapter support to CHIP REPL and update docs

Add support to select the Bluetooth adapter using the common
--ble-adapter command line argument.

Update the advanced docs for the Python Controller.

Author: agners

docs/guides/python_chip_controller_advanced_usage.md

@@ -7,8 +7,9 @@ tool or Matter accessories on Linux.
 
 <hr>
 
--   [Bluetooth LE virtualization on Linux](#bluetooth-le-virtualization-on-linux)
--   [Debugging with gdb](#debugging-with-gdb)
+-   [Using Python CHIP Controller advanced features](#using-python-chip-controller-advanced-features)
+    -   [Bluetooth LE virtualization on Linux](#bluetooth-le-virtualization-on-linux)
+    -   [Debugging with gdb](#debugging-with-gdb)
 
 <hr>
 
@@ -62,38 +63,38 @@ interfaces working as Bluetooth LE central and peripheral, respectively.
        TX bytes:3488 acl:95 sco:0 commands:110 errors:0
     ```
 
-4. Run the Python CHIP Controller with Bluetooth LE adapter defined from a
+4. Run the Python CHIP Controller REPL with Bluetooth LE adapter defined from a
    command line:
 
-    For example, add `--bluetooth-adapter=hci2` to use the virtual interface
-    `hci2` listed above.
+    For example, add `--ble-adapter=2` to use the virtual interface `hci2`
+    listed above.
 
     ```
-    chip-device-ctrl --bluetooth-adapter=hci2
+    chip-repl --ble-adapter=2
     ```
 
 <hr>
 
 ## Debugging with gdb
 
-You can run the chip-device-ctrl under GDB for debugging, however, since the
-Matter core support library is a dynamic library, you cannot read the symbols
-unless it is fully loaded.
+You can run the chip-repl under GDB for debugging, however, since the Matter SDK
+library is a dynamic library, you cannot read the symbols unless it is fully
+loaded.
 
 The following block is a example debug session using GDB:
 
 ```
 # GDB cannot run scripts directly
-# so you need to run Python3 with the path of device controller
-# Here, we use the feature from bash to get the path of chip-device-ctrl without typing it.
-$ gdb --args python3 `which chip-device-ctrl`
-GNU gdb (Ubuntu 10.1-2ubuntu2) 10.1.90.20210411-git
-Copyright (C) 2021 Free Software Foundation, Inc.
+# so you need to run Python3 with the path of device controller REPL
+# Here, we use the feature from bash to get the path of chip-repl without typing it.
+$ gdb --args python3 `which chip-repl`
+GNU gdb (GDB) 14.2
+Copyright (C) 2023 Free Software Foundation, Inc.
 License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
 This is free software: you are free to change and redistribute it.
 There is NO WARRANTY, to the extent permitted by law.
 Type "show copying" and "show warranty" for details.
-This GDB was configured as "aarch64-linux-gnu".
+This GDB was configured as "x86_64-pc-linux-gnu".
 Type "show configuration" for configuration details.
 For bug reporting instructions, please see:
 <https://www.gnu.org/software/gdb/bugs/>.
@@ -103,6 +104,12 @@ Find the GDB manual and other documentation resources online at:
 For help, type "help".
 Type "apropos word" to search for commands related to "word"...
 Reading symbols from python3...
+
+This GDB supports auto-downloading debuginfo from the following URLs:
+  <https://debuginfod.archlinux.org>
+Enable debuginfod for this session? (y or [n]) n
+Debuginfod has been disabled.
+To make this setting permanent, add 'set debuginfod enabled off' to .gdbinit.
 (No debugging symbols found in python3)
 (gdb)
 ```
@@ -119,38 +126,68 @@ library, let run the Matter device controller first.
 
 ```
 (gdb) run
-Starting program: /usr/bin/python3 /home/ubuntu/.local/bin/chip-device-ctrl
+Starting program: /home/sag/projects/project-chip/connectedhomeip/out/venv/bin/python3 /home/sag/projects/project-chip/connectedhomeip/out/venv/bin/chip-repl
 [Thread debugging using libthread_db enabled]
-Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
-CHIP:DIS: Init admin pairing table with server storage.
-CHIP:IN: local node id is 0x000000000001b669
-CHIP:DL: MDNS failed to join multicast group on wpan0 for address type IPv4: Inet Error 1016 (0x000003F8): Address not found
-CHIP:ZCL: Using ZAP configuration...
-CHIP:ZCL: deactivate report event
-CHIP:CTL: Getting operational keys
-CHIP:CTL: Generating operational certificate for the controller
-CHIP:CTL: Getting root certificate for the controller from the issuer
-CHIP:CTL: Generating credentials
-CHIP:CTL: Loaded credentials successfully
-CHIP:DL: Platform main loop started.
-Chip Device Controller Shell
+Using host libthread_db library "/usr/lib/libthread_db.so.1".
+Python 3.11.9 (main, Apr 29 2024, 11:59:58) [GCC 13.2.1 20240417]
+Type 'copyright', 'credits' or 'license' for more information
+IPython 8.24.0 -- An enhanced Interactive Python. Type '?' for help.
+[1716395111.775747][364405:364405] CHIP:CTL: Setting attestation nonce to random value
+[1716395111.776196][364405:364405] CHIP:CTL: Setting CSR nonce to random value
+InitBLE 0[1716395111.776809][364405:364405] CHIP:DL: writing settings to file (/tmp/chip_counters.ini-T7hX27)
+[1716395111.776854][364405:364405] CHIP:DL: renamed tmp file to file (/tmp/chip_counters.ini)
+[1716395111.776860][364405:364405] CHIP:DL: NVS set: chip-counters/reboot-count = 9 (0x9)
+[1716395111.777261][364405:364405] CHIP:DL: Got Ethernet interface: eno2
+[1716395111.777555][364405:364405] CHIP:DL: Found the primary Ethernet interface:eno2
+[1716395111.777868][364405:364405] CHIP:DL: Got WiFi interface: wlp7s0
+[1716395111.777877][364405:364405] CHIP:DL: Failed to reset WiFi statistic counts
+────────────────────────────────────────────────────────────────────────────────────────────────────────── Matter REPL ──────────────────────────────────────────────────────────────────────────────────────────────────────────
+
+
+
+            Welcome to the Matter Python REPL!
+
+            For help, please type matterhelp()
+
+            To get more information on a particular object/class, you can pass
+            that into matterhelp() as well.
+
+
+─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
+2024-05-22 18:25:11 allenwind PersistentStorage[364405] WARNING Initializing persistent storage from file: /tmp/repl-storage.json
+2024-05-22 18:25:11 allenwind PersistentStorage[364405] WARNING Loading configuration from /tmp/repl-storage.json...
+2024-05-22 18:25:11 allenwind CertificateAuthorityManager[364405] WARNING Loading certificate authorities from storage...
+2024-05-22 18:25:11 allenwind CertificateAuthority[364405] WARNING New CertificateAuthority at index 1
+2024-05-22 18:25:11 allenwind CertificateAuthority[364405] WARNING Loading fabric admins from storage...
+2024-05-22 18:25:11 allenwind FabricAdmin[364405] WARNING New FabricAdmin: FabricId: 0x0000000000000001, VendorId = 0xFFF1
+2024-05-22 18:25:11 allenwind FabricAdmin[364405] WARNING Allocating new controller with CaIndex: 1, FabricId: 0x0000000000000001, NodeId: 0x000000000001B669, CatTags: []
+
+
+The following objects have been created:
+        certificateAuthorityManager:    Manages a list of CertificateAuthority instances.
+        caList:                         The list of CertificateAuthority instances.
+        caList:                 A specific FabricAdmin object at index m for the nth CertificateAuthority instance.
 
-chip-device-ctrl >
+
+Default CHIP Device Controller (NodeId: 112233): has been initialized to manage caList[0].adminList[0] (FabricId = 1), and is available as devCtrl
+
+In [1]:
 ```
 
-The prompt `chip-device-ctrl >` indicates that the Matter core library is loaded
-by Python, you can browse the symbols in the Matter core library, setting
-breakpoints on functions and many other functions provided by GDB.
+The prompt `In [1]: ` indicates that the Matter SDK library has been loaded and
+initialized by the Python Controller REPL, you can browse the symbols in the
+Matter core library, setting breakpoints on functions and many other functions
+provided by GDB.
 
-You can use `Ctrl-C` to send SIGINT to the controller anytime you want so you
-can set breakpoints.
+You can use `Ctrl-Z` to send SIGSTP to the Python 3 REPL process anytime you
+want so you can set breakpoints (unfortunately Ctrl+C seems to be captured by
+the REPL).
 
-> (`Ctrl-C` pressed here.)
+In [1]: (`Ctrl-Z` pressed here.)
 
 ```
-Thread 1 "python3" received signal SIGINT, Interrupt.
-0x0000fffff7db79ec in __GI___select (nfds=<optimized out>, readfds=0xffffffffe760, writefds=0x0, exceptfds=0x0, timeout=<optimized out>) at ../sysdeps/unix/sysv/linux/select.c:49
-49	../sysdeps/unix/sysv/linux/select.c: No such file or directory.
+Thread 1 "python3" received signal SIGTSTP, Stopped (user).
+0x00007ffff7650ceb in kill () from /usr/lib/libc.so.6
 (gdb)
 ```
 
@@ -159,40 +196,27 @@ command in GDB (`b` for short)
 
 ```
 (gdb) b DeviceCommissioner::PairDevice
-Breakpoint 1 at 0xfffff5b0f6b4 (2 locations)
+Breakpoint 1 at 0x7fffed453943: DeviceCommissioner::PairDevice. (4 locations)
 (gdb)
 ```
 
-Type `continue` (`c` for short) to continue the device controller, you may need
-another hit of `Enter` to see the prompt.
+Type `signal SIGCONT` to continue the device controller after stopping it with
+signal stop, you may need another hit of `Enter` to see the prompt.
 
 ```
-(gdb) c
-Continuing.
-
-chip-device-ctrl >
+(gdb) signal SIGCONT
+Continuing with signal SIGCONT.
+In [1]:
 ```
 
 Let do pairing over IP to see the effect of the breakpoint we just set.
 
 ```
-chip-device-ctrl > connect -ip 192.168.50.5 20202021 1
-Device is assigned with nodeid = 1
-
-Thread 1 "python3" hit Breakpoint 1, 0x0000fffff5b0f6b4 in chip::Controller::DeviceCommissioner::PairDevice(unsigned long, chip::RendezvousParameters&)@plt ()
-   from /home/ubuntu/.local/lib/python3.9/site-packages/chip/_ChipDeviceCtrl.so
-(gdb)
-```
-
-The `@plt` symbol means it is a symbol used by dynamic library loader, type `c`
-(for `continue`) and it will break on the real function.
+In [1]: devCtrl.CommissionWithCode("MT:-24J0AFN00KA0648G00", 1234)
 
-```
-(gdb) c
-Continuing.
-
-Thread 1 "python3" hit Breakpoint 1, chip::Controller::DeviceCommissioner::PairDevice (this=0xd28540, remoteDeviceId=1, params=...) at ../../src/controller/CHIPDeviceController.cpp:827
-827	{
+Thread 5 "python3" hit Breakpoint 1.1, chip::Controller::DeviceCommissioner::PairDevice (this=0x7fffd8003a90, remoteDeviceId=1234, setUpCode=0x7ffff453d490 "MT:-24J0AFN00KA0648G00", params=...,
+    discoveryType=chip::Controller::DiscoveryType::kAll, resolutionData=...) at ../../src/controller/CHIPDeviceController.cpp:646
+646     {
 (gdb)
 ```
 
@@ -201,46 +225,44 @@ then you can use `bt` (for `backtrace`) to see the backtrace of the call stack.
 
 ```
 (gdb) bt
-#0  chip::Controller::DeviceCommissioner::PairDevice(unsigned long, chip::RendezvousParameters&) (this=0xd28540, remoteDeviceId=1, params=...)
-    at ../../src/controller/CHIPDeviceController.cpp:827
-#1  0x0000fffff5b3095c in pychip_DeviceController_ConnectIP(chip::Controller::DeviceCommissioner*, char const*, uint32_t, chip::NodeId)
-    (devCtrl=0xd28540, peerAddrStr=0xfffff467ace0 "192.168.50.5", setupPINCode=20202021, nodeid=1) at ../../src/controller/python/ChipDeviceController-ScriptBinding.cpp:234
-#2  0x0000fffff7639148 in  () at /lib/aarch64-linux-gnu/libffi.so.8
-#3  0x0000fffff7638750 in  () at /lib/aarch64-linux-gnu/libffi.so.8
-#4  0x0000fffff7665a44 in  () at /usr/lib/python3.9/lib-dynload/_ctypes.cpython-39-aarch64-linux-gnu.so
-#5  0x0000fffff7664c7c in  () at /usr/lib/python3.9/lib-dynload/_ctypes.cpython-39-aarch64-linux-gnu.so
-#6  0x00000000004a54f0 in _PyObject_MakeTpCall ()
-#7  0x000000000049cb10 in _PyEval_EvalFrameDefault ()
-#8  0x0000000000496d1c in  ()
-#9  0x00000000004b1eb0 in _PyFunction_Vectorcall ()
-#10 0x0000000000498264 in _PyEval_EvalFrameDefault ()
-#11 0x00000000004b1cb8 in _PyFunction_Vectorcall ()
-#12 0x0000000000498418 in _PyEval_EvalFrameDefault ()
-#13 0x0000000000496d1c in  ()
-#14 0x00000000004b1eb0 in _PyFunction_Vectorcall ()
-#15 0x0000000000498418 in _PyEval_EvalFrameDefault ()
-#16 0x00000000004b1cb8 in _PyFunction_Vectorcall ()
-#17 0x00000000004c6bc8 in  ()
-#18 0x0000000000498264 in _PyEval_EvalFrameDefault ()
-#19 0x00000000004b1cb8 in _PyFunction_Vectorcall ()
-#20 0x0000000000498418 in _PyEval_EvalFrameDefault ()
-#21 0x00000000004966f8 in  ()
-#22 0x00000000004b1f18 in _PyFunction_Vectorcall ()
-#23 0x0000000000498418 in _PyEval_EvalFrameDefault ()
-#24 0x00000000004b1cb8 in _PyFunction_Vectorcall ()
-#25 0x0000000000498264 in _PyEval_EvalFrameDefault ()
-#26 0x00000000004966f8 in  ()
-#27 0x0000000000496490 in _PyEval_EvalCodeWithName ()
-#28 0x0000000000595b7c in PyEval_EvalCode ()
-#29 0x00000000005c6a5c in  ()
-#30 0x00000000005c0a70 in  ()
-#31 0x00000000005c69a8 in  ()
-#32 0x00000000005c6148 in PyRun_SimpleFileExFlags ()
-#33 0x00000000005b60bc in Py_RunMain ()
-#34 0x0000000000585a08 in Py_BytesMain ()
-#35 0x0000fffff7d0c9d4 in __libc_start_main (main=
-    0x5858fc <_start+60>, argc=2, argv=0xfffffffff498, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=<optimized out>) at ../csu/libc-start.c:332
-#36 0x00000000005858f8 in _start ()
+(gdb) bt
+#0  chip::Controller::DeviceCommissioner::PairDevice
+    (this=0x7fffd8003a90, remoteDeviceId=1234, setUpCode=0x7fffef2555d0 "MT:-24J0AFN00KA0648G00", params=..., discoveryType=chip::Controller::DiscoveryType::kAll, resolutionData=...)
+    at ../../src/controller/CHIPDeviceController.cpp:646
+#1  0x00007fffed040825 in pychip_DeviceController_ConnectWithCode (devCtrl=0x7fffd8003a90, onboardingPayload=0x7fffef2555d0 "MT:-24J0AFN00KA0648G00", nodeid=1234, discoveryType=2 '\002')
+    at ../../src/controller/python/ChipDeviceController-ScriptBinding.cpp:395
+#2  0x00007ffff6ad5596 in ??? () at /usr/lib/libffi.so.8
+#3  0x00007ffff6ad200e in ??? () at /usr/lib/libffi.so.8
+#4  0x00007ffff6ad4bd3 in ffi_call () at /usr/lib/libffi.so.8
+#5  0x00007ffff6aeaffc in ??? () at /usr/lib/python3.11/lib-dynload/_ctypes.cpython-311-x86_64-linux-gnu.so
+#6  0x00007ffff6aeb4b4 in ??? () at /usr/lib/python3.11/lib-dynload/_ctypes.cpython-311-x86_64-linux-gnu.so
+#7  0x00007ffff794a618 in _PyObject_MakeTpCall () at /usr/lib/libpython3.11.so.1.0
+#8  0x00007ffff78f3d03 in _PyEval_EvalFrameDefault () at /usr/lib/libpython3.11.so.1.0
+#9  0x00007ffff7adef90 in ??? () at /usr/lib/libpython3.11.so.1.0
+#10 0x00007ffff79ebc0b in _PyObject_FastCallDictTstate () at /usr/lib/libpython3.11.so.1.0
+#11 0x00007ffff79ebe02 in _PyObject_Call_Prepend () at /usr/lib/libpython3.11.so.1.0
+#12 0x00007ffff79ec114 in ??? () at /usr/lib/libpython3.11.so.1.0
+#13 0x00007ffff794a618 in _PyObject_MakeTpCall () at /usr/lib/libpython3.11.so.1.0
+#14 0x00007ffff78f3d03 in _PyEval_EvalFrameDefault () at /usr/lib/libpython3.11.so.1.0
+#15 0x00007ffff7adef90 in ??? () at /usr/lib/libpython3.11.so.1.0
+#16 0x00007ffff7955b97 in PyObject_Vectorcall () at /usr/lib/libpython3.11.so.1.0
+#17 0x00007ffff6aea174 in ??? () at /usr/lib/python3.11/lib-dynload/_ctypes.cpython-311-x86_64-linux-gnu.so
+#18 0x00007ffff6aea28c in ??? () at /usr/lib/python3.11/lib-dynload/_ctypes.cpython-311-x86_64-linux-gnu.so
+#19 0x00007ffff6ad5152 in ??? () at /usr/lib/libffi.so.8
+#20 0x00007ffff6ad57b8 in ??? () at /usr/lib/libffi.so.8
+#21 0x00007fffed5de848 in chip::DeviceLayer::Internal::GenericPlatformManagerImpl<chip::DeviceLayer::PlatformManagerImpl>::_DispatchEvent
+    (this=0x7fffed88dc90 <chip::DeviceLayer::PlatformManagerImpl::sInstance+16>, event=0x7fffe6fffe30) at ../../src/include/platform/internal/GenericPlatformManagerImpl.ipp:304
+#22 0x00007fffed5dd90d in chip::DeviceLayer::PlatformManager::DispatchEvent (this=0x7fffed88dc80 <chip::DeviceLayer::PlatformManagerImpl::sInstance>, event=0x7fffe6fffe30) at ../../src/include/platform/PlatformManager.h:503
+#23 0x00007fffed5df45b in chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::ProcessDeviceEvents
+    (this=0x7fffed88dc90 <chip::DeviceLayer::PlatformManagerImpl::sInstance+16>) at ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:185
+#24 0x00007fffed5dee64 in chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::_RunEventLoop (this=0x7fffed88dc90 <chip::DeviceLayer::PlatformManagerImpl::sInstance+16>)
+--Type <RET> for more, q to quit, c to continue without paging--
+    at ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:227
+#25 0x00007fffed5dd888 in chip::DeviceLayer::PlatformManager::RunEventLoop (this=0x7fffed88dc80 <chip::DeviceLayer::PlatformManagerImpl::sInstance>) at ../../src/include/platform/PlatformManager.h:403
+#26 0x00007fffed5df3fe in chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::EventLoopTaskMain (arg=0x7fffed88dc90 <chip::DeviceLayer::PlatformManagerImpl::sInstance+16>)
+    at ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:256
+#27 0x00007ffff76a6ded in ??? () at /usr/lib/libc.so.6
+#28 0x00007ffff772a0dc in ??? () at /usr/lib/libc.so.6
 (gdb)
 ```
 

src/controller/python/chip/ChipReplStartup.py

@@ -94,6 +94,8 @@ def main():
         "-d", "--debug", help="Set default logging level to debug.", action="store_true")
     parser.add_argument(
         "-t", "--trust-store", help="Path to the PAA trust store.", action="store", default="./credentials/development/paa-root-certs")
+    parser.add_argument(
+        "-b", "--ble-adapter", help="Set the Bluetooth adapter index.", type=int, default=None)
     args = parser.parse_args()
 
     if not os.path.exists(args.trust_store):
@@ -128,7 +130,7 @@ def main():
         # nothing we can do ... things will NOT work
         return
 
-    chip.native.Init()
+    chip.native.Init(bluetoothAdapter=args.ble_adapter)
 
     global certificateAuthorityManager
     global chipStack