Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: on_target: add pkk test #96

Merged
merged 1 commit into from
Mar 6, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/build-and-target-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ jobs:
test:
permissions:
actions: read
contents: read
contents: write
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing permissions from contents: read to contents: write can introduce security risks if not properly managed. Ensure that the workflow is secure and that only trusted code can execute with these permissions.

packages: read
uses: ./.github/workflows/target-test.yml
needs: build
Expand Down
43 changes: 31 additions & 12 deletions .github/workflows/target-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,16 +42,19 @@ jobs:
# This will create multiple jobs, one for each target defined in the matrix
strategy:
matrix:
# nrf9151dk not available yet
device: [cia-trd-thingy91x]
include:
- device: cia-trd-thingy91x
- device: ppk_thingy91x
# Only run PPK tests when slow marker is specified
if: ${{ inputs.pytest_marker == 'slow' }}

# Self-hosted runner is labeled according to the device it is linked with
runs-on: ${{ matrix.device }}
name: Target Test - ${{ matrix.device }}

permissions:
actions: read
contents: read
contents: write
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the build-and-target-test.yml, changing permissions from contents: read to contents: write requires careful consideration. Verify that this change is necessary and that appropriate security measures are in place.

packages: read

container:
Expand Down Expand Up @@ -109,17 +112,25 @@ jobs:
run: |
mkdir -p results

if [[ '${{ inputs.pytest_marker }}' == 'no_markers_flag' || '${{ inputs.pytest_marker }}' == '' ]]; then
pytest_marker_arg=()
if [[ '${{ matrix.device }}' == 'ppk_thingy91x' ]]; then
# For PPK device, only run test_power.py
pytest -v tests/test_ppk/test_power.py \
--junit-xml=results/test-results.xml \
--html=results/test-results.html --self-contained-html
else
pytest_marker_arg=(-m "${{ inputs.pytest_marker }}")
# For other devices, use normal marker logic
if [[ '${{ inputs.pytest_marker }}' == 'no_markers_flag' || '${{ inputs.pytest_marker }}' == '' ]]; then
pytest_marker_arg=()
else
pytest_marker_arg=(-m "${{ inputs.pytest_marker }}")
fi

echo pytest -v "${pytest_marker_arg[@]}"
pytest -v "${pytest_marker_arg[@]}" \
--junit-xml=results/test-results.xml \
--html=results/test-results.html --self-contained-html \
${{ inputs.pytest_path }}
fi

echo pytest -v "${pytest_marker_arg[@]}"
pytest -v "${pytest_marker_arg[@]}" \
--junit-xml=results/test-results.xml \
--html=results/test-results.html --self-contained-html \
${{ inputs.pytest_path }}
shell: bash
env:
SEGGER: ${{ env.RUNNER_SERIAL_NUMBER }}
Expand All @@ -131,6 +142,14 @@ jobs:
MEMFAULT_ORGANIZATION_SLUG: ${{ secrets.MEMFAULT_ORGANIZATION_SLUG }}
MEMFAULT_PROJECT_SLUG: ${{ secrets.MEMFAULT_PROJECT_SLUG }}

- name: Commit and Push Badge File to gh-pages Branch
if: always()
continue-on-error: true
working-directory: asset-tracker-template
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: ./tests/on_target/scripts/commit_badge.sh

- name: Results
if: always()
uses: pmeier/pytest-results-action@v0.7.1
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,6 @@

#### Nightly:
[![Target_tests](https://github.com/nrfconnect/Asset-Tracker-Template/actions/workflows/build-and-target-test.yml/badge.svg?event=schedule)](https://github.com/nrfconnect/Asset-Tracker-Template/actions/workflows/build-and-target-test.yml?query=branch%3Amain+event%3Aschedule)

[![Power Consumption Badge](https://img.shields.io/endpoint?url=https://nrfconnect.github.io/Asset-Tracker-Template/power_badge.json)](https://nrfconnect.github.io/Asset-Tracker-Template/power_measurements_plot.html)

The Asset Tracker Template is under development.
233 changes: 233 additions & 0 deletions tests/on_target/tests/test_ppk/test_power.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,233 @@
##########################################################################################
# Copyright (c) 2025 Nordic Semiconductor
# SPDX-License-Identifier: LicenseRef-Nordic-5-Clause
##########################################################################################

import os
import time
import json
import types
import pytest
import csv
import pandas as pd
import plotly.express as px
from tests.conftest import get_uarts
from ppk2_api.ppk2_api import PPK2_API
from utils.uart import Uart
from utils.flash_tools import flash_device, reset_device, recover_device
import sys
sys.path.append(os.getcwd())
from utils.logger import get_logger

logger = get_logger()

UART_TIMEOUT = 60 * 30
POWER_TIMEOUT = 60 * 15
MAX_CURRENT_PSM_UA = 10
SAMPLING_INTERVAL = 0.01
CSV_FILE = "power_measurements.csv"
HMTL_PLOT_FILE = "power_measurements_plot.html"
SEGGER = os.getenv('SEGGER')


def save_badge_data(average):
badge_filename = "power_badge.json"
logger.info(f"Minimum average current measured: {average}uA")
if average < 0:
pytest.skip(f"current cant be negative, current average: {average}")
elif average <= 10:
color = "green"
elif average <= 50:
color = "yellow"
else:
color = "red"

badge_data = {
"label": "🔗 PSM current uA",
"message": f"{average}",
"schemaVersion": 1,
"color": f"{color}"
}

# Save the JSON data to a file
with open(badge_filename, 'w') as json_file:
json.dump(badge_data, json_file)

logger.info(f"Minimum average current saved to {badge_filename}")


def save_measurement_data(samples):
# Generate timestamps for each sample assuming uniform sampling interval
timestamps = [round(i * SAMPLING_INTERVAL, 2) for i in range(len(samples))]

with open(CSV_FILE, 'w', newline='') as csvfile:
fieldnames = ['Time (s)', 'Current (uA)']
writer = csv.DictWriter(csvfile, fieldnames=fieldnames)

writer.writeheader()
for t, current in zip(timestamps, samples):
writer.writerow({'Time (s)': t, 'Current (uA)': current})

logger.info(f"Measurement data saved to {CSV_FILE}")


def generate_time_series_html(csv_file, date_column, value_column, output_file="time_series_plot.html"):
"""
Generates an HTML file with an interactive time series plot from a CSV file.

Parameters:
- csv_file (str): Path to the CSV file containing the time series data.
- date_column (str): Name of the column containing date or time information.
- value_column (str): Name of the column containing the values to plot.
- output_file (str): Name of the output HTML file (default is "time_series_plot.html").

Returns:
- str: The path to the generated HTML file.
"""
# Load the CSV file
df = pd.read_csv(csv_file, parse_dates=[date_column])

title = "Asset Tracket Template Current Consumption Plot\n\n"
note_text = "Note: application is still in development, not reaching target psm current yet"
title += f"<br><span style='font-size:12px;color:gray;'>{note_text}</span>"

# Create an interactive Plotly line chart
fig = px.line(df, x=date_column, y=value_column, title=title)

# Save the plot to an HTML file
fig.write_html(output_file)

logger.info(f"HTML file generated: {output_file}")
return output_file


@pytest.fixture(scope="module")
def thingy91x_ppk2():
'''
This fixture sets up ppk measurement tool.
'''
ppk2s_connected = PPK2_API.list_devices()
ppk2s_connected.sort()
if len(ppk2s_connected) == 2:
ppk2_port = ppk2s_connected[0]
ppk2_serial = ppk2s_connected[1]
logger.info(f"Found PPK2 at port: {ppk2_port}, serial: {ppk2_serial}")
elif len(ppk2s_connected) == 0:
pytest.skip("No ppk found")
else:
pytest.skip(f"PPK should list 2 ports, but found {ppk2s_connected}")

ppk2_dev = PPK2_API(ppk2_port, timeout=1, write_timeout=1, exclusive=True)

# get modifier might fail, retry 15 times
for _ in range(15):
try:
ppk2_dev.get_modifiers()
break
except Exception as e:
logger.error(f"Failed to get modifiers: {e}")
time.sleep(5)
else:
pytest.skip("Failed to get ppk modifiers after 10 attempts")

ppk2_dev.set_source_voltage(3300)
ppk2_dev.use_ampere_meter() # set ampere meter mode
ppk2_dev.toggle_DUT_power("ON") # enable DUT power

time.sleep(10)
for i in range(10):
try:
all_uarts = get_uarts()
logger.warning(f"momo all uarts {all_uarts}")
if not all_uarts:
logger.error("No UARTs found")
log_uart_string = all_uarts[0]
break
except Exception as e:
logger.warning(f"Exception: {e}")
ppk2_dev.toggle_DUT_power("OFF") # disable DUT power
time.sleep(2)
ppk2_dev.toggle_DUT_power("ON") # enable DUT power
time.sleep(5)
continue
else:
pytest.skip("NO uart after 10 attempts")

t91x_uart = Uart(log_uart_string, timeout=UART_TIMEOUT)

yield types.SimpleNamespace(ppk2_dev=ppk2_dev, t91x_uart=t91x_uart)

t91x_uart.stop()
recover_device(serial=SEGGER)
ppk2_dev.stop_measuring()

@pytest.mark.slow
def test_power(thingy91x_ppk2, hex_file):
'''
Test that the device can reach PSM and measure the current consumption

Current consumption is measured and report generated.
'''
flash_device(os.path.abspath(hex_file), serial=SEGGER)
reset_device(serial=SEGGER)
try:
thingy91x_ppk2.t91x_uart.wait_for_str("Connected to Cloud", timeout=120)
except AssertionError:
pytest.skip("Device unable to connect to cloud, skip ppk test")

thingy91x_ppk2.ppk2_dev.start_measuring()

start = time.time()
min_rolling_average = float('inf')
rolling_average = float('inf')
samples_list = []
last_log_time = start
psm_reached = False

# Initialize an empty pandas Series to store samples over time
samples_series = pd.Series(dtype='float64')
while time.time() < start + POWER_TIMEOUT:
try:
read_data = thingy91x_ppk2.ppk2_dev.get_data()
if read_data != b'':
ppk_samples, _ = thingy91x_ppk2.ppk2_dev.get_samples(read_data)
sample = sum(ppk_samples) / len(ppk_samples)
sample = round(sample, 2)
samples_list.append(sample)

# Append the new sample to the Pandas Series
samples_series = pd.concat([samples_series, pd.Series([sample])], ignore_index=True)

# Log and store every 3 seconds
current_time = time.time()
if current_time - last_log_time >= 3:
# Calculate rolling average over the last 3 seconds
window_size = int(3 / SAMPLING_INTERVAL)
rolling_average_series = samples_series.rolling(window=window_size).mean()
rolling_average = rolling_average_series.iloc[-1] # Get the last rolling average value
rolling_average = round(rolling_average, 2) if not pd.isna(rolling_average) else rolling_average
logger.info(f"Average current over last 3 secs: {rolling_average} uA")

if rolling_average < min_rolling_average:
min_rolling_average = rolling_average

last_log_time = current_time

# Check if PSM target has been reached
if rolling_average < MAX_CURRENT_PSM_UA and rolling_average > 0:
psm_reached = True

except Exception as e:
logger.error(f"Catching exception: {e}")
pytest.skip("Something went wrong, unable to perform power measurements")

time.sleep(SAMPLING_INTERVAL) # lower time between sampling -> less samples read in one sampling period

# Save measurement data and generate HTML report
save_badge_data(min_rolling_average)
save_measurement_data(samples_list)
generate_time_series_html(CSV_FILE, 'Time (s)', 'Current (uA)', HMTL_PLOT_FILE)

# Determine test result based on whether PSM was reached
if not psm_reached:
pytest.fail(f"PSM target not reached after {POWER_TIMEOUT / 60} minutes, only reached {min_rolling_average} uA")
Loading