Author: kittycool only

by

Adding cgroups control to GPGPU Servers for PSB-Professional

After adding GPGPU node to PBS Professional, you have to make sure, it is in the right queue first

qmgr -c "set node gpu-node resources_available.Qlist = gpu_v100"

Locate the cgroups.json2 script in the directory you have placed in. Check by doing the following 

ll cgroups.json2

If there, edit the file. 

vim cgroups.json2

Find the place where the “run_only_on_hosts” and add the node

"run_only_on_hosts" : [ "gpu-node1", "gpu-node2", "gpu-node3", "gpu-node4],
        "cgroup":
......
......
......

Use the qmgr to import the file

qmgr -c "import hook cgroups application/x-config default cgroups.json2"

Check that the PBS has detected the node correctly

pbsnodes -aSj |grep gpu-node1
by

Disabling ipv6 on Rocky Linux 8 with Ansible

If you wish to disable ipv6 on Rocky Linux 8, there is a wonderful writeup on the script found at https://github.com/juju4/ansible-ipv6/blob/main/tasks/ipv6-disable.yml which you may find useful. If you just need to disable it temporarily without disruption (assuming you have not been using ipv6 at all)

- name: Disable IPv6 with sysctl
  ansible.posix.sysctl:
    name: "{{ item }}"
    value: "1"
    state: "present"
    reload: "yes"
  with_items:
    - net.ipv6.conf.all.disable_ipv6
    - net.ipv6.conf.default.disable_ipv6
    - net.ipv6.conf.lo.disable_ipv6

If you can tolerate a bit of disruption, you may want to take a look at putting it at the network configuration and restarting it

- name: RedHat | disable ipv6 in sysconfig/network
  ansible.builtin.lineinfile:
    dest: /etc/sysconfig/network
    regexp: "^{{ item.regexp }}"
    line: "{{ item.line }}"
    mode: '0644'
    backup: true
    create: true
  with_items:
    - { regexp: 'NETWORKING_IPV6=.*', line: 'NETWORKING_IPV6=NO' }
    - { regexp: 'IPV6INIT=.*', line: 'IPV6INIT=no' }
  notify:
    - Restart network
    - Restart NetworkManager
  when: ansible_os_family == 'RedHat'
by

Using Mellanox ConnectX VPI Ports to Ethernet or InfiniBand

The Mellanox ConnectX5 VPI adapter supports both Ethernet and InfiniBand port modes, which must be configured.

Check Status

# mst status -v
MST modules:
------------
    MST PCI module is not loaded
    MST PCI configuration module is not loaded
PCI devices:
------------
DEVICE_TYPE             MST                           PCI       RDMA            NET                                     NUMA  
ConnectX4(rev:0)        /dev/mst/mt4115_pciconf3      8b:00.0   mlx5_3                                                  1     

ConnectX4(rev:0)        /dev/mst/mt4115_pciconf2      84:00.0   mlx5_2                                                  1     

ConnectX4(rev:0)        /dev/mst/mt4115_pciconf1      0c:00.0   mlx5_1                                                  0     

ConnectX4(rev:0)        /dev/mst/mt4115_pciconf0      05:00.0   mlx5_0                                                  0                                                 1

Start MST

# mst start
Starting MST (Mellanox Software Tools) driver set
Loading MST PCI module - Success
Create devices
Unloading MST PCI module (unused) - Success

Change the port type to Ethernet (LINK_TYPE = 2)

# mlxconfig -d /dev/mst/mt4115_pciconf2 set LINK_TYPE_P1=2

Check that the port type was changed to Ethernet

# ibdev2netdev
mlx5_0 port 1 ==> ens1np0 (Down)
mlx5_1 port 1 ==> enp12s0np0 (Down)
mlx5_2 port 1 ==> enp132s0np0 (Up)
mlx5_3 port 1 ==> enp139s0np0 (Down)

References:

by

Using Mac Terminal App to SSH X-Forward

If you are using a MacOS and you are planning to do a X-forwarding, you may notice an error like this. In the Pix, below, after SSH, I wanted to X-forward Firefox Browser from Rocky Linux 8

ssh -X user1@192.168.0.1

There are a few settings we need to do. Let’s fix the “Locale not supported by C library. Using the fallback ‘C’ locale”

Issue 1 – Locale not support by C library. Using the fallback ‘C’ locale

To fix it, go to Top-Left-Hand Corner of your Mac-Desktop to look for the Terminal App,

  1. Click “Settings” (Pix 1)
  2. Click the Profiles Icons (Pix 2)
  3. Uncheck the “Set locale environment variable on startup” (Pix 2)

Close and Quit Terminal App and come in again. You should fix the first problem.

Issue 2 – Install XQuartz

X11 used to be included with macOS. Apple no longer includes X11 with macOS. That is why you see an error like this “Error: no DISPLAY environment variable specified.”

You have to download the XQuartz App which is an open-source effort to develop a version of the X.Org X Window System that runs on macOS. Downloading and Installing is very straight-forward and easy. I’ve captured selected screen-shots….

After installing, you may want to let is run in the background

Log-Off from Your Mac Completely to allow the settings to reinitate

Issue 3 – Fixing the libGL error: failed to load driver: swrast on Rocky Linux 8

You may find an issue similar to the one in the earlier blog. That is how you may want to fix it.

by

Using Ansible to get Flexlm License Information and copy to Shared File Environment

You can use Ansible to extract Flexlm information from a remote license server, which is stored in a central place where you can display the information.

I use crontab to extract the information every 15 min and place it in a central place so that users can check the license availability. 

- name: Extract Information from ANSYS Lic Server and extract to file
  block:
    - name: Get FlexLM License Info
      ansible.builtin.shell: "/usr/local/ansys_inc/shared_files/licensing/linx64/lmutil lmstat -c ../license_files/ansyslmd.lic -a"
      register: lmstat_output

    - name: Save FlexLM License Output to File on ANSYS Lic Server
      copy:
        content: "{{ lmstat_output.stdout }}"
        dest: "/var/log/ansible_logs/ansys_lmstat.log"

    - name: Get FlexLM Output from Remote Server
      fetch:
        src: "/var/log/ansible_logs/ansys_lmstat.log"
        dest: "/usr/local/lic_lmstat_log/ansys_lmstat.log"
        flat: yes

The fetch command is useful for fetching files from remote machines and storing them locally in a file tree. For more information, do take a look at Fetch files from remote nodes

At crontab, I fetch the file every 15min

*/15 * * * * /root/ansible_cluster/run_lmstat_licsvr.sh

The run_lmstat_licsvr.sh is simply to call the ansible playbook to run the ansible script above.