Error using nerfcapture2nerf.py script

[ashutoshmishra1014]

Hello,

Many thanks for your great work!
Unfortunately, I am not able to get any connections on the Nerfcapture app.
I started the nerfcapture2nerf.py script on my computer and then opened the nerfcapture app on my iPhone.
Both the devices are on the same wifi network.

I tried it with 2 different iPhones, still the same issue. Am I missing anything?

Any suggestions, how to resolve it?

[yusuf-wadi]

Any luck?

[jc211]

Sorry for the late reply, I'll look into this tomorrow and report back

[jc211]

Delete this line:
https://github.com/NVlabs/instant-ngp/blob/e45134b9bcf50d0c04f27bc3ab3cde57c27f5bc8/scripts/nerfcapture2nerf.py#L61
and it should work for you.

Alternatively, if you are on linux, set the maximum receive buffer size, rmem_max, by running:

sudo sysctl -w net.core.rmem_max=2147483647

Sending raw RGB over wifi is a bit ridiculous and I'm working on fixing that. Thats why the buffer sizes need to be so big.

[yusuf-wadi]

I have everything working up to this point. Cyclonedds recognizes the script, the only thing missing is the iphone connection.
Picture for reference:

I would love to cite this amazing project as part of my research, but just this one issue is blocking me.

[yusuf-wadi]

I had an issue with only seeing ground truth, resolved by pressing 'g'

[ashutoshmishra1014]

@jc211 I tried both the options, but unfortunately didn't work. I tried using wifi and bluetooth as networks for both desktop and my iPhone but no luck. I do not see my iPhone registered with the cyclonedds neither do I see any available connections in the nerfcapture online mode.

Delete this line: https://github.com/NVlabs/instant-ngp/blob/e45134b9bcf50d0c04f27bc3ab3cde57c27f5bc8/scripts/nerfcapture2nerf.py#L61 and it should work for you.

Alternatively, if you are on linux, set the maximum receive buffer size, rmem_max, by running:

sudo sysctl -w net.core.rmem_max=2147483647

Sending raw RGB over wifi is a bit ridiculous and I'm working on fixing that. Thats why the buffer sizes need to be so big.

[yusuf-wadi]

I had it working in one network environment but not working in others. My guess is that issue lies not in the devices but in the networks.

[ashutoshmishra1014]

@jc211 I have been trying to use the Nerfcapture again on my iPhone connected via the cable with my Linux machine. After launching the nerfcapture2nerf.py script --stream, I run cyclonedds ls. I do see 2 entities being returned but it keeps on fluctuating between 2 and 0, the connection is not stable. I used cable instead of WiFi connection specifically to avoid any connection issues. Also, when I run cyclonedds ps I do not get any entry but cyclonedds ls does show entities being discovered but very unstable, it switches between 1,2 or 0 every half a second.
I have also tried restarting the app and the script multiple times but no help.

Any guidance in this regard will be very helpful. Waiting for your earliest response.

[ashutoshmishra1014]

@jc211 I recently also attempted to connect my iPhone/iPad via USB-C cable with my Windows laptop but no luck. The python script does not appear in the cyclonedds ps output when connected via WiFi or the USB-C cable. When "NetworkInterface" is set to "auto" in the cyclonedds config, the python script does appear in the cyclonedds ps output but the interface selected is "Ethernet 2" which is not expected.

Do you have any idea what am I missing?

[jc211]

Sorry for the late reply. If you are compiling this from source, the lines that need to be changed are

NeRFCapture/NeRFCapture/DDSWriter.swift

Line 42 in 312cb01

<NetworkInterface name="en0" />

"en0" is the WiFi interface on apple devices. The ethernet interface is called something else. In XCode's console, you can check the interfaces available when cyclonedds connects. Messing with the source code to get the configuration you want is obviously not the best solution and it really should be a setting within the app. But this might be the fastest way to resolve your issue. Other configuration settings can be found here: https://cyclonedds.io/docs/cyclonedds/latest/config/index.html

[ashutoshmishra1014]

I have finally managed to resolve the issue. Now I have another question though.
I want to disable broadcasting and rather use unicasting. When I enable unicast in the dds config in the nerfcapture2nerf.py file suddenly the iPad does not connect. Does any changes need to be done in the dds config of the app itself as well?

The usecase is that the nerfcapture should talk to only one specific machine and not use broadcast as any broadcasting is restricted in the network I am using.