Skip to content

Commit

Permalink
more search and replace, sans underscore
Browse files Browse the repository at this point in the history
  • Loading branch information
stephenlienharrell committed Jan 10, 2025
1 parent 1fe4139 commit 4ff096c
Show file tree
Hide file tree
Showing 7 changed files with 34 additions and 34 deletions.
28 changes: 14 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ The hpcperfstats package provides the tools to monitor resource usage of HPC sys

The package is split into an `autotools`-based `monitor` subpackage and a Python `setuptools`-based `hpcperfstats` subpackage. `monitor` performs the online data collection and transmission in a production environment while `hpcperfstats` performs the data curation and analysis in an offline environment.

Building and installing the `hpcperfstats-2.3.5-1.el7.x86_64.rpm` package with the `taccstats.spec` file will build and install a systemd service `taccstats`. This service launches a daemon with an overhead of 3% on a single core when configured to sample at a frequency of 1Hz. It is typically configured to sample at 5 minute intervals, with samples taken at the start and end of every job as well. The TACC Stats daemon, `hpcperfstatsd`, is controlled by the `taccstats` service and sends the data directly to a RabbitMQ server over the administrative ethernet network. RabbitMQ must be installed and running on the server in order for the data to be received.
Building and installing the `hpcperfstats-2.3.5-1.el7.x86_64.rpm` package with the `hpcperfstats.spec` file will build and install a systemd service `hpcperfstats`. This service launches a daemon with an overhead of 3% on a single core when configured to sample at a frequency of 1Hz. It is typically configured to sample at 5 minute intervals, with samples taken at the start and end of every job as well. The TACC Stats daemon, `hpcperfstatsd`, is controlled by the `hpcperfstats` service and sends the data directly to a RabbitMQ server over the administrative ethernet network. RabbitMQ must be installed and running on the server in order for the data to be received.

Installing the `hpcperfstats` module will setup a Django-based web application along with tools for extracting the data from the RabbitMQ server and feeding them into a PostgreSQL database.

Expand All @@ -45,7 +45,7 @@ First ensure the RabbitMQ library and header file are installed on the build and

`./configure; make; make install` will then successfully build the `hpcperfstatsd` executable for many systems. If Xeon Phi coprocessors are present on your system they can be monitored with the `--enable-mic` flag. Additionally the configuration options, `--disable-infiniband`, `--disable-lustre`, `--disable-hardware` will disable infiniband, Lustre Filesystem, and Hardware Counter monitoring which are all enabled by default. Disabling RabbitMQ will result in a legacy build of `hpcperfstatsd` that relies on the shared filesystem to transmit data. This mode is not recommended and currently used for testing purposes only. If libraries or header files are not found than add their paths to the include and library paths with the `CPPFLAGS` and/or `LDFLAGS` vars as is standard in autoconf based installations.

There will be a configuration file, `/etc/taccstats/taccstats.conf`, after installation. This file contains the fields
There will be a configuration file, `/etc/hpcperfstats/hpcperfstats.conf`, after installation. This file contains the fields

`server localhost`

Expand All @@ -58,17 +58,17 @@ There will be a configuration file, `/etc/taccstats/taccstats.conf`, after insta

`server` should be set to the hostname or IP hosting the RabbitMQ server, `queue` to the system/cluster name that is being monitored, `port` to the RabbitMQ port (5672 is default), and `freq` to the desired sampling frequency in seconds. The file and settings can be reloaded into a running `hpcperfstatsd` daemon with a SIGHUP signal.

An RPM can be built for deployment using the `taccstats.spec` file. The most straightforward approach to build this is to setup your rpmbuild directory then run
An RPM can be built for deployment using the `hpcperfstats.spec` file. The most straightforward approach to build this is to setup your rpmbuild directory then run

`rpmbuild -bb taccstats.spec`
`rpmbuild -bb hpcperfstats.spec`

The `taccstats.spec` file `sed`s the `taccstats.conf` file to the correct server and queue. These can be changed by modifying these two lines
The `hpcperfstats.spec` file `sed`s the `hpcperfstats.conf` file to the correct server and queue. These can be changed by modifying these two lines

`sed -i 's/localhost/stats.frontera.tacc.utexas.edu/' src/taccstats.conf`
`sed -i 's/localhost/stats.frontera.tacc.utexas.edu/' src/hpcperfstats.conf`

`sed -i 's/default/frontera/' src/taccstats.conf`
`sed -i 's/default/frontera/' src/hpcperfstats.conf`

`hpcperfstatsd` can be started, stopped, and restarted using `systemctl start taccstats`, `systemctl stop taccstats`, and `systemctl restart taccstats`.
`hpcperfstatsd` can be started, stopped, and restarted using `systemctl start hpcperfstats`, `systemctl stop hpcperfstats`, and `systemctl restart hpcperfstats`.

In order to notify `hpcperfstats` of a job beginning, echo the job id into `/var/run/TACC_jobid` on each node where the job is running. It order to notify
it of a job ending echo `-` into `/var/run/TACC_jobid` on each node where the job is running. This can be accomplished in the job scheduler prolog and
Expand Down Expand Up @@ -154,12 +154,12 @@ and the `hpcperfstats` Python module
$ sudo su - postgres
$ psql
# CREATE DATABASE machinename_db;
# CREATE USER taccstats WITH PASSWORD 'taccstats';
# ALTER ROLE taccstats SET client_encoding TO 'utf8';
# ALTER ROLE taccstats SET default_transaction_isolation TO 'read committed';
# ALTER ROLE taccstats SET timezone TO 'UTC';
# ALTER DATABASE machinename_db OWNER TO taccstats;
# GRANT ALL PRIVILEGES ON DATABASE machinename_db TO taccstats;
# CREATE USER hpcperfstats WITH PASSWORD 'hpcperfstats';
# ALTER ROLE hpcperfstats SET client_encoding TO 'utf8';
# ALTER ROLE hpcperfstats SET default_transaction_isolation TO 'read committed';
# ALTER ROLE hpcperfstats SET timezone TO 'UTC';
# ALTER DATABASE machinename_db OWNER TO hpcperfstats;
# GRANT ALL PRIVILEGES ON DATABASE machinename_db TO hpcperfstats;
# \q
```

Expand Down
2 changes: 1 addition & 1 deletion hpcperfstats/dbload/globus_sync.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@
# For more information:
# https://docs.globus.org/api/auth/developer-guide/#register-app
CLIENT_ID = '900c3e14-cc2e-4a8d-819f-7eb2cf2ad854'
TOKEN_FILE = 'taccstats-tokens.json'
TOKEN_FILE = 'hpcperfstats-tokens.json'
DATA_FILE = 'transfer-data.json'
REDIRECT_URI = 'https://auth.globus.org/v2/web/auth-code'
SCOPES = ('openid email profile '
Expand Down
2 changes: 1 addition & 1 deletion hpcperfstats/dbload/test_utils/query_timescaledb.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
#pandas.set_option('display.max_rows', 100)


CONNECTION = "dbname=taccstats user=postgres port=5433"
CONNECTION = "dbname=hpcperfstats user=postgres port=5433"

conn = psycopg2.connect(CONNECTION)
print(conn.server_version)
Expand Down
6 changes: 3 additions & 3 deletions monitor/src/Makefile.am
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ AM_DISTCHECK_CONFIGURE_FLAGS = \

if HAVE_SYSTEMD
systemduserunit_DATA = \
taccstats.service \
taccstats.conf
hpcperfstats.service \
hpcperfstats.conf
endif

bin_PROGRAMS = hpcperfstatsd
Expand Down Expand Up @@ -171,4 +171,4 @@ stats.o: stats.x
stats.x: Makefile
echo '$(patsubst %.c, X(%),$(sort $(TYPES)))' > $@

EXTRA_DIST = taccstats.conf taccstats.service
EXTRA_DIST = hpcperfstats.conf hpcperfstats.service
4 changes: 2 additions & 2 deletions monitor/src/monitor.c
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ static char *server = NULL;
static char *queue = "default";
static char *port = "5672";

static char *dumpfile_dir = "/tmp/taccstats";
static char *dumpfile_dir = "/tmp/hpcperfstats";
static double freq = 300;
static int max_buffer_size = 300; // 25 hours
static int allow_ring_buffer_overwrite = 0;
Expand Down Expand Up @@ -551,7 +551,7 @@ static void usage(void)
" -s [SERVER] or --server [SERVER] Server to send data.\n"
" -q [QUEUE] or --queue [QUEUE] Queue to route data to on RMQ server. \n"
" -p [PORT] or --port [PORT] Port to use (5672 is the default).\n"
" -t [TMP_DIR] or --tmp [TMP_DIR] Directory for dumpfiles (/tmp/taccstats is the default).\n"
" -t [TMP_DIR] or --tmp [TMP_DIR] Directory for dumpfiles (/tmp/hpcperfstats is the default).\n"
" -b [BUFFER] or --buffer [BUFFER] Max size (in # of stats) for temporary in-memory storage (288 is the default).\n"
" -f [FREQUENCY] or --frequency [FREQUENCY] Frequency to sample (300 seconds is the default).\n"
,
Expand Down
2 changes: 1 addition & 1 deletion monitor/src/stats_buffer.c
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ static int send(struct stats_buffer *sf)
}

amqp_login(conn, "/", 0, 131072, 0, AMQP_SASL_METHOD_PLAIN,
"taccstats", "taccstats");
"hpcperfstats", "hpcperfstats");
amqp_channel_open(conn, 1);
amqp_get_rpc_reply(conn);

Expand Down
24 changes: 12 additions & 12 deletions monitor/taccstats.spec
Original file line number Diff line number Diff line change
Expand Up @@ -21,29 +21,29 @@ along with a systemd script to provide control.
%build
./configure
make
sed -i 's/CONFIGFILE/\%{_sysconfdir}\/taccstats\/taccstats.conf/' src/taccstats.service
sed -i 's/localhost/stats.frontera.tacc.utexas.edu/' src/taccstats.conf
sed -i 's/default/frontera/' src/taccstats.conf
sed -i 's/CONFIGFILE/\%{_sysconfdir}\/hpcperfstats\/hpcperfstats.conf/' src/hpcperfstats.service
sed -i 's/localhost/stats.frontera.tacc.utexas.edu/' src/hpcperfstats.conf
sed -i 's/default/frontera/' src/hpcperfstats.conf

%install
mkdir -p %{buildroot}/%{_sbindir}/
mkdir -p %{buildroot}/%{_sysconfdir}/taccstats/
mkdir -p %{buildroot}/%{_sysconfdir}/hpcperfstats/
mkdir -p %{buildroot}/%{_unitdir}/
install -m 744 src/hpcperfstatsd %{buildroot}/%{_sbindir}/hpcperfstatsd
install -m 644 src/taccstats.conf %{buildroot}/%{_sysconfdir}/taccstats/taccstats.conf
install -m 644 src/taccstats.service %{buildroot}/%{_unitdir}/taccstats.service
install -m 644 src/hpcperfstats.conf %{buildroot}/%{_sysconfdir}/hpcperfstats/hpcperfstats.conf
install -m 644 src/hpcperfstats.service %{buildroot}/%{_unitdir}/hpcperfstats.service

%files
%{_sbindir}/hpcperfstatsd
%{_sysconfdir}/taccstats/taccstats.conf
%{_unitdir}/taccstats.service
%dir %{_sysconfdir}/taccstats
%{_sysconfdir}/hpcperfstats/hpcperfstats.conf
%{_unitdir}/hpcperfstats.service
%dir %{_sysconfdir}/hpcperfstats

%post
%systemd_post taccstats.service
%systemd_post hpcperfstats.service

%preun
%systemd_preun taccstats.service
%systemd_preun hpcperfstats.service

%postun
%systemd_postun_with_restart taccstats.service
%systemd_postun_with_restart hpcperfstats.service

0 comments on commit 4ff096c

Please sign in to comment.