Setting up Git Repository for Ansible

Install DNF on the Client Side

$ dnf install git
================================================================================
 Package             Architecture  Version               Repository        Size
================================================================================
Upgrading:
 git                 x86_64        2.31.1-3.el8_7        appstream        160 k
 git-core            x86_64        2.31.1-3.el8_7        appstream        4.8 M
 git-core-doc        noarch        2.31.1-3.el8_7        appstream        2.6 M
 perl-Git            noarch        2.31.1-3.el8_7        appstream         77 k

Transaction Summary
================================================================================
Upgrade  4 Packages

Total download size: 7.6 M
Is this ok [y/N]: 

At github.com,

Do sign up for an account if you have not done so. At github.com, create a new Repository and give it a new name. Look at the pix for more information

Once you have created the repository, you should have something like this

At the profile Icon, click settings, you should enter the page where “SSH and GPG Keys” are present

Click the new SSH icon

Type the name of the key (Your Choice) and copy your ssh public keys into the key box. This can be found at ~/.ssh/id_rsa.pub or ~/.ssh/ed25519.pub. Copy the whole string and paste into the key box

Git Clone the Repository URL

Go back to your repository and copy the code

Next

At the Linux Server….

git clone the URL

$ git clone git@github.com:XXXXXXXXXX/ansible_cluster.git             
Cloning into 'ansible_cluster'...
Warning: Permanently added 'github.com,20.205.243.166' (ECDSA) to the list of known hosts.
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (3/3), done.

At your directory go to the ansible_cluster folder, you should be able to see the README.md file which is same as your git repository.

$ cd ansible_cluster
$ cat README.md
# ansible_cluster

Setting some basic git information like user name and email address

$ git config --global user.name "user1"
$ git config --global user.email "user1@hello.com"
$ cat ~/.gitconfig
[user]
        name = melvin soh
        email = melvin@ntu.edu.sg
[init]
        defaultBranch = main

Pushing and cloning to the git repository

Let’s make some changes to the README.md

$ vim README.md
#ansible_tutorial

Hello

Checking their Status

$ git status
On branch main
Your branch is up to date with 'origin/main'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   README.md

no changes added to commit (use "git add" and/or "git commit -a")

Add the changes, coming and pushing the changes to git

$ git add README.md
$ git status
On branch main
Your branch is up to date with 'origin/main'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        modified:   README.md

$ git commit -m "Updated Readme file on 8/5/2023"
[main 06583ec] Updated Readme file on 8/5/2023
 1 file changed, 3 insertions(+), 1 deletion(-)

$ git push origin main
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Writing objects: 100% (3/3), 308 bytes | 102.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
To github.com:kittycooldew/ansible_cluster.git
   5524665..62b9b0a  main -> main

References:

  1. Top 20 Git Commands With Examples
  2. Learn Linux TV Chapter 3

Getting Notification from Ansible

I have been learning from this book Fabio Alessandro Locati, published under Packt>. The example can be found at https://github.com/PacktPublishing/Learning-Ansible-2.X-Third-Edition/tree/master/Chapter06

Today I am learning about working with notification. One of the biggest advantages of Ansible is its ability, compared to a bash script to notify. The 6 tools which Ansible could easily work with include

  • Email Notification
  • Ansible XMPP/Jabber
  • Slack and Rocket Chat
  • Sending a message to an IRC Channel
  • Amazon Simple Notification Service
  • Nagios

For the full list of notification modules, we can refer to https://docs.ansible.com/ansible/2.8/modules/list_of_notification_modules.html

Email:

---
- hosts: localhost 
  connection: local
  tasks: 
    - name: Read the machine uptime 
      command: uptime -p 
      register: uptime 
    - name: Send the uptime via e-mail 
      mail: 
        host: mail.fale.io 
        username: ansible@fale.io 
        password: PASSWORD 
        to: me@fale.io 
        subject: Ansible-report 
        body: 'Local system uptime is {{ uptime.stdout }}.' 

To sent the email, we will need the SMTP host, credential and content of the email. Do note that mail modules support the following

  • The attachment parameter: To attach attachments
  • The port parameter: port to use by the email server.

Working with Roles in Ansible

I have been learning from this book Fabio Alessandro Locati, published under Packt>. The example can be found at https://github.com/PacktPublishing/Learning-Ansible-2.X-Third-Edition/tree/master/Chapter04

Today I am learning about working with roles. According to the author, the definition of a role is a set of playbooks, templates, files or variables used to achieve a specific goal. For example, the database role and the web server role can be cleanly separated.

You can see the structure in https://github.com/PacktPublishing/Learning-Ansible-2.X-Third-Edition/tree/master/Chapter04

According to the author, he recommends 3 files in the root folder

  • ansible.cfg: A small configuration file to explain to Ansible where to find the files in the folder structure
  • hosts: host files
  • master.yml: A playbook that aligns the whole infrastructure.

2 more folders

  • playbooks: This will contain the playbooks and a folder called groups for groups management
  • roles: Contain all the roles required.

Working with the Conditional feature in Ansible

I have been learning from this book Fabio Alessandro Locati, published under Packt>. The example can be found at https://github.com/PacktPublishing/Learning-Ansible-2.X-Third-Edition/tree/master/Chapter04

Today I am learning about working with local_action feature. This feature allows us to run certain tasks locally on the machine that runs Ansible rather than logging into a remote box and running these commands.

--- 
- hosts: database 
  remote_user: vagrant
  tasks: 
    - name: Count processes running on the remote system 
      shell: ps | wc -l 
      register: remote_processes_number 
    - name: Print remote running processes 
      debug: 
        msg: '{{ remote_processes_number.stdout }}' 
    - name: Count processes running on the local system 
      local_action: shell ps | wc -l 
      register: local_processes_number 
    - name: Print local running processes 
      debug: 
        msg: '{{ local_processes_number.stdout }}' 

Delegating a Task

If you wish to execute an action a different system. For example, you may want to do something on a database node while working on an application node or a localhost, you can use the delegate_to: HOST Property. This is useful when there are complex procedures need to be executed by the local machine or any other machine

--- 
- hosts: database 
  remote_user: vagrant
  tasks: 
    - name: Count processes running on the remote system 
      shell: ps | wc -l 
      register: remote_processes_number 
    - name: Print remote running processes 
      debug: 
        msg: '{{ remote_processes_number.stdout }}' 
    - name: Count processes running on the local system 
      shell: ps | wc -l 
      delegate_to: localhost 
      register: local_processes_number 
    - name: Print local running processes 
      debug: 
        msg: '{{ local_processes_number.stdout }}' 

Working with Conditionals

Ansible provides conditional statements to run a task only when a specified condition(s) is met

--- 
- hosts: webserver 
  remote_user: vagrant
  tasks: 
    - name: Print the ansible_os_family value 
      debug: 
        msg: '{{ ansible_os_family }}' 
    - name: Ensure the httpd package is updated 
      yum: 
        name: httpd 
        state: latest 
      become: True 
      when: ansible_os_family == 'RedHat' 
    - name: Ensure the apache2 package is updated 
      apt: 
        name: apache2 
        state: latest 
      become: True 
      when: ansible_os_family == 'Debian' 

Boolean Conditionals

Apart from matching string, you can check whether a variable is true. Ansible provides a way to check whether a variable is defined. The below features allow us to put the Ansible playgroup in a failure state if the backup_folder is not set

--- 
- hosts: all 
  remote_user: ansible 
  vars: 
    backup: True 
  tasks: 
    - name: Check if the backup_folder is set 
      fail: 
        msg: 'The backup_folder needs to be set' 
      when: backup_folder is not defined 
    - name: Copy the crontab in tmp if the backup variable is true 
      copy: 
        src: /etc/crontab 
        dest: '{{ backup_folder }}/crontab' 
        remote_src: True 
      when: backup 

Working with Handlers

Every handler will run at the end of the playbook if notified. Ansible will make sure, how many times you notify the service, it will call that task once after all other tasks has completed.

--- 
- hosts: webserver 
  remote_user: vagrant
  tasks: 
    - name: Ensure the HTTPd package is installed 
      yum: 
        name: httpd 
        state: present 
      become: True 
    - name: Ensure the HTTPd service is enabled and running 
      service: 
        name: httpd 
        state: started 
        enabled: True 
      become: True 
    - name: Ensure HTTP can pass the firewall 
      firewalld: 
        service: http 
        state: enabled 
        permanent: True 
        immediate: True 
      become: True 
    - name: Ensure HTTPd configuration is updated 
      copy: 
        src: website.conf 
        dest: /etc/httpd/conf.d 
      become: True 
      notify: Restart HTTPd 
  handlers: 
    - name: Restart HTTPd 
      service: 
        name: httpd 
        state: restarted 
      become: True 

Listing and Sorting Files

Listing files by access date.

You can use the “-ltur” option. The “u” enforces the “by access date” listing order.  The “t” option to list files in order of age. The “r” option is to reverse the option

$ ls -ltur
-rwxr-xr-x  1 user1 users 1622374400 Mar 10 00:08 yyyy.iso
-rw-rw-r--  1 user1 users   18387452 Mar 13 14:06 xxxx.rpm
-rwxr-xr-x  1 user1 users        303 Mar 30 16:32 visJob.pbs
-rw-------  1 user1 users        762 Mar 30 16:35 visJob.o1475403
-rw-------  1 user1 users         59 Mar 30 16:35 visJob.e1475403

Listing Files by Modified Date and Time

You can use the “-ltr” option. The “r” is the reverse order.

$ ls -ltr
-rw-rw-r--  1 melvin melvin   18387452 Mar 13 14:03 xxxx.rpm
-rwxr-xr-x  1 melvin melvin        303 Mar 13 17:00 vis.pbs
-rw-rw-r--  1 melvin melvin       1084 Mar 23 16:52 yyy.pem

Listing files by owner

Use the output of the ls command to sort and pick out the owner column by adding “-k3” to sort on the third field.

$ ls -l | sort -k3 | more
drwx------  29 www           users                 1005 Sep 20  2022 www-home
drwx------  16 yyy           users                  684 Apr  3 14:07 yyy-home
drwx------  16 zzz           users                  746 Mar 21 11:31 zzz-home

Listing files by group

Sort files by the associated groups, you can pass the output from a long listing to the sort command and tell it to sort on column 4.

$ ls -l | sort -k4 | more
drwx------  29 www           users1                 1005 Sep 20  2022 www-home
drwx------  16 yyy           users2                  684 Apr  3 14:07 yyy-home
drwx------  16 zzz           users3                  746 Mar 21 11:31 zzz-home

Listing files by size

To sort files by Size, use the “-S” Option. If you wish to put the largest files at the end, use the “-r”. Use the “-h” to be human readable.

ls -lSrh
-rw-rw-r--  1 www users 1.9G Nov 15 11:04 integr8_120.xml.gz
-rw-rw-r--  1 www users 6.0G Jul 26  2022 20220712_msconvert.zip
-rw-rw-r--  1 www users 6.9G May 31  2022 ELECTRONICS.tgz

Working with Inventory File in Ansible

I have been learning from this book Fabio Alessandro Locati, published under Packt>. The example can be found at https://github.com/PacktPublishing/Learning-Ansible-2.X-Third-Edition/tree/master/Chapter03

Basics

Today I am learning on working with Inventory Files. This time we are dealing with multiple hosts. These hosts have to be placed in the inventory file. An example is pasted here. In hosts.ini, we have

[Compute] 
node01.example.com 
node02.example.com
 
[Login] 
login.example.com

To run the ansible playfile

ansible-playbook -i hosts.ini firstrun.yaml

firstrun.yaml is taken from the site listed. It is to ensure the ansible user exist, accept the SSH keys and provided with sudoers rights with no password.

 hosts: all 
  user: vagrant 
  tasks: 
    - name: Ensure ansible user exists 
      user: 
        name: ansible 
        state: present 
        comment: Ansible 
      become: True
    - name: Ensure ansible user accepts the SSH key 
      authorized_key: 
        user: ansible 
        key: https://github.com/fale.keys 
        state: present 
      become: True
    - name: Ensure the ansible user is sudoer with no password required 
      lineinfile: 
        dest: /etc/sudoers 
        state: present 
        regexp: '^ansible ALL\=' 
        line: 'ansible ALL=(ALL) NOPASSWD:ALL' 
        validate: 'visudo -cf %s'
      become: True

Regular Expressions

If you have a larger number of servers with predictable names, you may want to consider the following expression. You can save 100 lines of listing the server with the following expression

[Compute] 
node[01:100].example.com 

 
[Login] 
login.example.com

Group Variables

If you wish to set a variable for the whole group, you may want to set a variable that is valid for the whole group,. A quick note from the book is that the host variables will override the group variables if the same variable is declared in both spaces.

[Compute] 
node[01:100].example.com 

[compute:vars]
firewalld_enabled=false
 
[Login] 
login.example.com

Working with iterates in Ansibles. For example in a un-iterates codes below

- name: Ensure the HTTP can pass the firewall 
      firewalld: 
        name: http 
        state: enabled 
        permament: True
        immediate: True
    - name: Ensure the HTTPS can pass the firewall 
      service: 
        name: https 
        state: enabled 
        enabled: True 
      become: True 

The codes can be shortened with the following with_items

- name: Ensure HTTP and HTTPS can pass the firewall 
      firewalld: 
        service: '{{ item }}'  
        state: enabled 
        permanent: True 
        immediate: True 
      become: True
      with_items:
        - http
        - https

Using nested loops – with_nested

If you need to iterate all elements of a list with all items from other lists. For example, you may want to create multiple folderw in multiple paths

--- 
- hosts: all 
  remote_user: ansible
  vars: 
    users: 
      - alice 
      - bob 
    folders: 
      - mail 
      - public_html 
  tasks: 
    - name: Ensure the users exist 
      user: 
        name: '{{ item }}' 
      become: True 
      with_items: 
        - '{{ users }}' 
    - name: Ensure the folders exist 
      file: 
        path: '/home/{{ item.0 }}/{{ item.1 }}' 
        state: directory 
      become: True 
      with_nested: 
        - '{{ users }}' 
        - '{{ folders }}' 

Fileglobs loop – with_fileglobs

If you want to perform an action on every file present in a certain folder like copying multiples files with similar names from one folder to another, you can do the following

--- 
- hosts: all 
  remote_user: ansible
  tasks: 
    - name: Ensure the folder /tmp/iproute2 is present 
      file: 
        dest: '/tmp/iproute2' 
        state: directory 
      become: True 
    - name: Copy files that start with rt to the tmp folder 
      copy: 
        src: '{{ item }}' 
        dest: '/tmp/iproute2' 
        remote_src: True 
      become: True 
      with_fileglob: 
        - '/etc/iproute2/rt_*' 

Thinking Strategy – Creative Thinker

Everyone in a Development Team has a preferred way of working and thinking. Thinking can be broken down into 3 dimensions. a. Creative Thinker, Understanding Thinking and Decision Thinking According to the Book “Git for Teams”

What are the Characteristics of a Creative Thinker:

Envision

  • To see an alternative future (whether it’s good or bad). This is useful for long-term strategy work.

Reframe

  • See the current situation from different perspectives

Brainstorming

  • Brainstorming is almost the ability to doodle through a problem

Employ flash of insight

  • A Flash of Insight happens when you’re not thinking about the problem. It happens when you’re out for a walk or in the shower.

Challenge

  • To question the status quo

Flow

  • Ignore Distraction and focus wholly on a given task

Recognising Creative Thinkers

You can recognise the creative thinkers from their key phrases

“Can we try?”

Have you thought about doing this instead?

I had this great idea?

Europe’s Leading Quantum Computer Manufacturer Launches Free Online Course for All

IQM Quantum Computers (IQM), a European leader in building quantum computers, today launched a global initiative, “IQM Academy,” to offer a free online quantum training course to educate and prepare talent for quantum workforce development.

IQM aims to reach high school and university students, educators, and enthusiasts who are curious to start learning about the fundamentals of quantum computing.

For more information, do take a look at https://academy.meetiqm.com/

Using Find to Search Better

Basic Use of FIND

If you are looking to find a file, one of the most common tools is Find. Here is a recap.

OFILE TYPEDESCRIPTION
1type -fLimits search results to files only
2type -dLimits search results to directories only
3type -lLimits search results to symbolic links only

For example, search for a case-insensitive file named “hello.mov”

$ find $HOME -type -iname "Hello.mov"

Parameters

NOPARAMETERSDESCRIPTION
1-namePerform a case-sensitive search for “files”
2-inamePerform a case-insensitive search for “files”
3size +nMatches files of size larger than size n
4size -nMatches files of size smaller than size n
5-mtime nMatches files or directories whose contents were last modified n*24 hours ago
6-atime nMatches files last access n*24 hours ago

For example, search for all case-insensitive files with the extension *mov 2 days ago

$ find $HOME -type -iname "*.mov" -mtime 2

Operators

S/NOOPERATOREXPLANATION
1-andMatch for both sides of the operators
2-orMatch for either test of the operators
3-noteDon’t match the test of the operators

For example, search for all files with Hello*, but excl ude pdf and jpg

$ find \( -name "Hello*" -mtime 2 \) -and -not \( -iname "*.jpg" -or -iname "*.pdf" \)

When using the () to combine tests, remember to escape the (\) brackets. You will need to leave a space after you open and close the brackets

find -type f -iname "*.mov" -exec chmod +x {} \;

The first part find -type f -iname”*.mov” will not be explained….. Executed commands must end with \; (a backslash and semi-colon) and may use {} (curly braces) as a placeholder for each file that the find command locates.

References:

  1. Linux Format – March Edition
  2. Use the Unix find command to search for files