problem with field IDs

[dandanio]

After a couple months of consistently going over my quota, I am looking for a way to monitor my usage. Happy to find your repo, but it does not work for me. Check this out:

[+] Starting Chrome
[+] Getting Login page
[+] Waiting for fields to load
Traceback (most recent call last):
  File "/usr/bin/coxusage.py", line 29, in <module>
    username = WebDriverWait(driver, 60).until(EC.visibility_of_element_located((By.ID, 'okta-signin-username')))
  File "/usr/local/lib/python3.8/site-packages/selenium/webdriver/support/wait.py", line 80, in until
    raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:

I have disabled 2FA. What am I doing wrong?

[dejanzelic]

Grrr this is frustrating. Looks like Cox changed their login flow again. I'll have to look more deeply into why this is happening. I'll report back with what I find

[dandanio]

Thank you @dejanzelic If there is anything I can help you with, lmk. I could really use your script. 🥇

[dejanzelic]

Bad news, It looks like Cox started to use "Incapsula" to block our requests. I'm not 100% sure how they are detecting that we are using a script since we are opening a browser page with selenium and navigating. It should be, in theory, able to be bypassed but that might be entering a grey area.

I think the root of the issue is that Cox is activity trying to prevent us from having access to our usage data in a way that's much more meaningful then there UI. They're notifications are not customizable and they cannot warn us if there is any anomaly in our data usage like we can if we have our own scripts.

There is another project which is experiencing the same issue: dcshoecomp/homeassistant-cox#11

I think our best bet is to request that Cox create an api endpoint that we can use for this so we're not always having to play wack-a-mole. They're probably not going to do it because it would mean they be able to scam people out of less money.

We all know they're datacaps are bullshit. With their Gigabit plan, if you were to continuously download for 3 hours you would hit the monthly data cap. But the idea of actually getting a full gigabit is even more of a joke then their datacaps.

They've raised the datacap by 280 GB in the last couple years. With the rise of 4k streaming, Work from home, massive video game sizes, etc. I doubt that 280GB is enough. They have always used the line that only a small percentage of customers ever hit the datacap. But in dockerhub, my image and ntalekt's image has been downloaded over 4,000 times. I don't think that "only a small percentage" of their customer hit the data cap. I imagine that it's much higher.

Anyway, if you'd like to join me, I'll be filing an FCC complaint against Cox for making it more difficult for us to detect data usage anomalies: https://consumercomplaints.fcc.gov/hc/en-us/articles/115002206106-Internet-Complaints

[vutang50]

It looks like there may be a workaround described here. Also it looks like the data url has been updated. I'll try playing around with it.

[dejanzelic]

Woah nice find!

[jefferystone]

Even with the changes @vutang50 suggested it still doesn't work for me.

@vutang50 were you able to get it to work?

[dandanio]

Anyway, if you'd like to join me, I'll be filing an FCC complaint against Cox for making it more difficult for us to detect data usage anomalies: https://consumercomplaints.fcc.gov/hc/en-us/articles/115002206106-Internet-Complaints

Go ahead, I will do that as well. (It won't be my first, and, honestly, the first time I heard from the FCC and Cox and I believe my complaint led to breakup of 2 supernodes we had in town into 5. I can't tell you guys how much that improved the peak time slowdowns!)

[dandanio]

I also wonder, if we can sniff out the URL Cox uses in their smartphone app and/or the widget (it does not require authentication relying on some session persistence).

[jefferystone]

Since there's no current workaround, I've created a macro using UI Vision RPA (plugin for Chrome).

This does not use Selenium or Chromedriver.

You can still use python to launch Chrome with the RPA plugin and choose the macro to run. After the macro is done Chrome will exit.

The macro can gather elements and export them to csv. The you can use csv reader in python to load the data.

Works good for me so far. If you think this warrants a GitHub project for sharing then let me know.

I have tested with both headless Chrome and normal Chrome UI (running from cronjob it will require "export DISPLAY=:0.0" for Chrome UI). I've also found that headless Chrome doesn't work 100% of time, but I'll be doing more testing.