23_Linux Basics - ansible2

I. Cloud Computing#

Cloud Computing
openstack
docker
kvm
Virtualization technology: virtualizing cloud hosts based on physical machines

The only downside is that security is not high, and data is stored with others (not a big impact)#

Troubleshooting ability, problem-solving ability#

Note: ssh enables 2 types of authentication - first public key authentication, then password authentication; if password authentication fails, it prompts permit deny#

II. Ansible is not a daemon#

Ansible is not a daemon; it only executes when the ansible command is run#

Daemon: always in memory, waiting for others to connect#

ssh is a daemon#

Executing a command is not a daemon#

Ansible is not a daemon (daemon: runs in memory after starting, waiting for others to access)#

Ansible is just a command script written in Python#

#[root@cPen_A ansible]# sudo -i # Note: directly switch to root user

III. Ansible Modules#

Ansible Modules

1. Copy Module#

Copy files from local to the directory host path
Parameter description:
src= source file path # Note: source

dest= destination path # Note: destination

Note that if src= path ends with /, it means copying all contents inside to the target directory; if it does not end with /, it recursively copies the directory
content= file content to fill in
owner owner
group group
mode permissions

Example 1: shell module, copy module
---------------------------------------------------------------------------------------------------------------------------------
#Example 1: Use ansible to create /lianxi directory on hosts B and C
#-m specify module
#-a parameters passed
ansible target hosts -m specify module -a parameters to execute
[root@cPen_A ansible]# ansible all -m shell -a "mkdir /lianxi -p"
#Note: rc ==> return code --> 0 indicates success
#Note: rc ==> return code --> 1 indicates failure

#Example 2: Copy ansible-copy file to hosts B and C in /lianxi/ansible directory, specify permissions and owner array
[root@cPen_A lianxi]# vim ansible-copy
[root@cPen_A lianxi]# ansible all -m copy -a "src=/lianxi/ansible-copy dest=/lianxi/ansible mode=777 owner=sanchuang group=sanchuang"
#Note: Confirm that the specified user sanchuang and group sanchuang exist on hosts B and C
#md5sum generally is the information summary to determine if the file has been modified

Example 2: Specify webser group, copy /etc/passwd to /tmp directory on hosts, specify permissions 777
---------------------------------------------------------------------------------------------------------------------------------
[root@cPen_A lianxi]# ansible webser -m copy -a "src=/etc/passwd dest=/tmp mode=777"

===========================================================================================

Example 3: Difference between src path with and without /
---------------------------------------------------------------------------------------------------------------------------------
#Note: if src= path ends with /, it means copying all contents inside to the target directory; if it does not end with /, it recursively copies the directory
#Note: without /
[root@cPen_A copy_dir]# ansible all -m copy -a "src=/lianxi/copy_dir dest=/lianxi/ansible"
#Note: with /
[root@cPen_A copy_dir]# ansible all -m copy -a "src=/lianxi/copy_dir/ dest=/lianxi/ansible"

#With / means copying sub-files or sub-directories under the directory
#Without / means copying the entire directory

2. Fetch Module#

#Note: Pull files from remote hosts to local

Example
---------------------------------------------------------------------------------------------------------------------------------
[root@cPen_A lianxi]# ansible webser -m fetch -a "src=/etc/hostname dest=/lianxi mode=644"
192.168.0.22 | CHANGED => {
    "changed": true, 
    "checksum": "c468f7340237d9f531122f5b03345ce8ac5641b4", 		  # Note: will compare after pulling
    "dest": "/lianxi/192.168.0.22/etc/hostname", 
    "md5sum": "8d23b0947f191eb9c20728677271eb4d", 
    "remote_checksum": "c468f7340237d9f531122f5b03345ce8ac5641b4", # Note: compare with remote to check if content has changed
    "remote_md5sum": null
}
#Note: If a large file transfer is suddenly interrupted, the file is not fully transferred; the saved md5sum and its md5sum are different
#Note: If the two values are consistent, it indicates that the transfer is complete
[root@cPen_A lianxi]# ls				# Note: the path automatically adds a folder named after the IP address
192.168.0.20  192.168.0.22  ansible-copy  copy_dir
[root@cPen_A lianxi]# cd 192.168.0.20
[root@cPen_A 192.168.0.20]# ls
etc

Pull files from remote hosts to local
fetch will automatically add the remote host's naming directory structure after the specified dest directory followed by the src directory structure
The directory structure stored locally: dest + remote host name + src

#Note: md5 can determine 1. whether the file has been modified; 2. whether the file has been fully transferred
#Note: or use any of the hash algorithms to compare if the values are equal

3. Command Module#

Execute commands on remote hosts, belonging to bare execution, non-key-value display; does not perform shell parsing

Example 1: command, shell
---------------------------------------------------------------------------------------------------------------------------------
[root@cPen_A 192.168.0.20]# ansible all -m command -a "ifconfig"
[root@cPen_A 192.168.0.20]# ansible all -m shell -a "ifconfig"

Example 2: command bare execution (the parameters passed later will not be parsed)
---------------------------------------------------------------------------------------------------------------------------------
#Note: treat ifconfig|grep inet as a single command to execute
[root@cPen_A 192.168.0.20]# ansible all -m command -a "ifconfig|grep inet"
192.168.0.22 | FAILED | rc=2 >>
[Errno 2] No such file or directory
#It is bare execution and will not parse its pipe symbol, treating ifconfig|grep inet as a command

#Note: shell can (the parameters passed later will be parsed)
[root@cPen_A 192.168.0.20]# ansible all -m shell -a "ifconfig|grep inet"

============================================================================================
[root@cPen_A ansible]# ifconfig			# Note: view network card (more information)
#Note: can see received packets, returned packets, bytes, how many errors
[root@cPen_A ansible]# ip a				# Note: view network card

4. Shell Module#

Example
---------------------------------------------------------------------------------------------------------------------------------
[root@cPen_A 192.168.0.20]# ansible all -m shell -a "ifconfig|grep inet"
Same as command, but the shell module can parse functions like pipes

5. File Module#

Set file attributes (create files)
Common parameters:
path target path
state directory for directory, link for soft link
group directory group
owner owner
Others can be obtained through ansible-doc -s file

state --
absent delete files and directories
directory directory
touch create an empty file
link soft link
hard hard link

Example 1: View file help information
--------------------------------------------------------------------------------------------
[root@cPen_A lianxi]# ansible-doc -s file			# Note: view file help information
============================================================================================

Example 2: Create a directory in the target host's /tmp directory
#Note: create file directory
---------------------------------------------------------------------------------------------------------------------------------
[root@cPen_A lianxi]# ansible all -m file -a "path=/tmp/sanchuang state=directory"
============================================================================================

Example 3: Set file attribute modification
---------------------------------------------------------------------------------------------------------------------------------
[root@cPen_A lianxi]# ansible all -m file -a "path=/tmp/passwd owner=sanchuang"

6. Cron Module#

Generate scheduled tasks on target hosts through the cron module
Common parameters:
In addition to minute, hour, day, month, week
name: name of this scheduled task
state: present generate (default) | absent delete (based on name)

#Note: First check if the program is running
[root@cPen_A ~]# ps -ef |grep crond
root        702      1  0 10:59 ?        00:00:00 /usr/sbin/crond -n
root       3582   3553  0 14:41 pts/3    00:00:00 grep --color=auto crond
#Note: Learn to check logs
[root@cPen_A ~]# cd /var/log/
[root@cPen_A log]# less cron
Nov 25 12:01:01 cPen_A run-parts(/etc/cron.hourly)[3498]: finished 0anacron

Example 1: Add scheduled task
---------------------------------------------------------------------------------------------------------------------------------
#Note: Output the current time every three minutes to /tmp/time.txt file
[root@cPen_A log]# ansible all -m cron -a "minute=*/3 job='date >>/tmp/time.txt' name=date_test state=present"		# Note: minute=*/3 specifies every 3 minutes date >>/tmp/time.txt command
[root@cPen_B ~]# crontab -l
#Ansible: date_test
*/3 * * * * date >>/tmp/time.txt

[root@cPen_A log]# ansible 192.168.0.20 -m cron -a "minute=*/3 job='date >>/tmp/time.txt' name=date_test state=present"		# Note: can set for a specific host, not necessarily batch processing

============================================================================================

Example 2: Delete scheduled task
---------------------------------------------------------------------------------------------------------------------------------
[root@cPen_A log]# ansible 192.168.0.20 -m cron -a  "name=date_test state=absent"

NTP service is a time management server
Example: NTP service system time management service
[root@cPen_A log]# yum install ntp				# Note: centos7
[root@cPen_B ~]# yum install chrony				# Note: in centos8, ntp has been replaced by chrony
[root@cPen_A log]# ntpdate ntp1.aliyun.com		# Note: synchronize time with Alibaba Cloud
#Note: Use NTP client to synchronize with NTP server to sync time

7. Yum Module#

As the name suggests, it is the module for installing software packages using yum;
Common parameter descriptions:
enablerepo, disablerepo indicate enabling and disabling certain repo libraries
name package name
state (present or installed, latest) indicates installation, (absent or removed) indicates deletion

#Note: name of the package to be installed state whether to install or uninstall

Example: Install wget
[root@cPen_A log]# ansible all -m yum -a "name=wget state=installed"
Example: Uninstall wget
[root@cPen_A log]# ansible all -m yum -a "name=wget state=absent"

8. Service Module#

Service management module
Common parameters:
name: service name
state: service status started (start) stopped (stop) restarted (restart) reloaded (reload)
enabled: whether to start on boot true|false
runlevel: startup level (ignored in systemd mode)

#Install file transfer service vsftpd
[root@cPen_A log]# ansible all -m yum -a "name=vsftpd state=installed"

Example: Stop vsftpd service
[root@cPen_A log]# ansible all -m service -a "name=vsftpd state=stopped"
Example: Start vsftpd service
[root@cPen_A log]# ansible all -m service -a "name=vsftpd state=started"

9. Script Module#

Transfer local scripts to remote execution; the premise is that the script can be executed on the remote end, do not sync Linux scripts to execute on Windows;

#Note: Only execute scripts on the remote server, do not upload scripts to the remote server

[root@cPen_A ~]# vim test.sh
#!/bin/bash
echo "test ansible" >>/tmp/ansible.txt
[root@cPen_A ~]# ansible all -m script -a "/root/test.sh"	# Note: /root/test.sh script path

--------------------------------------------------------------------------------------------
Example: If execution is slow and there are many machines, how to solve it
Answer: You can use multiple processes to execute
-f 6		Specify 6 processes to execute

The command module cannot recognize pipe symbols, while the shell module can recognize them
The copy module pushes
The script module does not need to upload scripts, it executes scripts on the remote server

IV. Check System Version#

Example: Check system version
[root@cPen_B ~]# uname -r						# Note: Check system kernel
4.18.0-193.el8.x86_64
[root@cPen_B ~]# cat /etc/redhat-release 		# Note: Check /etc/redhat-release to see system version
CentOS Linux release 8.2.2004 (Core)

V. Links#

Example: Soft link
[root@cPen_A lianxi]# ln -s ansible-copy ansible-copy-likcs-s	#Note: original file in front, link file in back
[root@cPen_A lianxi]# ls -al 		# Note: -s soft link
lrwxrwxrwx   1 root root  12 Nov 25 11:30 ansible-copy-likcs-s -> ansible-copy
#Note: Soft link is equivalent to a shortcut
#Note: Deleting a soft link does not affect the original file; if the original file of the soft link is deleted, it will affect the link file

Example: Hard link
[root@cPen_A lianxi]# ln ansible-copy ansible-copy-link
[root@cPen_A lianxi]# ls -al
-rw-r--r--   2 root root  20 Nov 25 10:12 ansible-copy-link
rm -rf deleting the source file still leaves the hard link file intact
#Note: Hard links can be used for backups
#Note: Deleting the original file of a hard link does not affect the link file

#Soft link, hard link
#Creating a hard link increases the link count of the file by 1
#Deleting a hard link file or the source file only decreases the link count by 1; the file will not be truly deleted
#Creating a hard link
[root@cPen_A lianxi]# ls -al
-rw-r--r--   2 root root  20 Nov 25 10:12 ansible-copy		# Note: link count is 2
#Note: Soft link
lrwxrwxrwx   1 root root  12 Nov 25 11:30 ansible-copy-likcs-s -> ansible-copy # Note: link count is 1
#Note: Hard link
-rw-r--r--   2 root root  20 Nov 25 10:12 ansible-copy-link	# Note: link count is 2
Symbolic link count of 1 means
rm -rf deletes the file name and the link to data in the disk
Data still exists on the disk; after a while, new data will overwrite the data
#Note: A count of 1 indicates that it is gone after deletion, while a count of 2 indicates that it still exists after deletion

VI. Playbook#

If ansible's various modules (which can achieve various functions) are tools in a workshop; playbook is the instruction manual, and the target remote host is the inventory and raw material object
#Note: Syntax is YAML format configuration

1. Core Elements of Playbook
hosts: the hosts that the playbook configuration file acts on
tasks: task list
variables: variables
templates: text files containing template syntax
handlers: tasks triggered by specific conditions
roles: used to organize playbooks hierarchically and structurally. Roles can automatically load variable files, tasks, and handlers based on a hierarchical structure
2. Playbook Running Methods
ansible-playbook --check only detects possible changes but does not actually execute operations
ansible-playbook --list-hosts lists the hosts running tasks
ansible-playbook --syntax-check playbook.yaml syntax check
ansible-playbook -t TAGS_NAME playbook.yaml only executes TAGS_NAME tasks
ansible-playbook playbook.yaml run

Example: Write/Execute playbook
--------------------------------------------------------------------------------------------
#Note: Copy /etc/passwd from machine A to /tmp/passwd_tmp on other machines
[root@cPen_A ~]# vim ansible_playbook_sc.yaml		#Note: Write playbook
- hosts: all
  remote_user: root
  tasks:
  - name: up file
    copy: src=/etc/passwd dest=/tmp/passwd_tmp
#Execute playbook
[root@cPen_A ~]# ansible-playbook ansible_playbook_sc.yaml 
============================================================================================

Example: How to write a YAML file
--------------------------------------------------------------------------------------------
#Step 1: First install python3: yum install python3
#Step 2: Use pip3 to install the PyYaml module
[root@cPen_A ~]# pip3 install PyYaml
[root@cPen_A ~]# python3
>>> import yaml
>>> fp = open("ansible_playbook_sc.yaml")
>>> dict = yaml.load(fp)
>>> dict
[{'hosts': 'all', 'remote_user': 'root', 'tasks': [{'name': 'up file', 'copy': 'src=/etc/passwd dest=/tmp/passwd_tmp'}]}]

#Note: Use
1. Create a YAML file
[root@cPen_A ~]# cat ansible_playbook_sc.yaml 	# Note: similar to dictionary form - equivalent to list; inside is equivalent to dictionary
- hosts: all								# Note: effective for all hosts (any group)
  remote_user: root							# Note: executed by root user
  tasks:									# Note: what it should do
  - name: up file							# Note: the task is named up file
    copy: src=/etc/passwd dest=/tmp/passwd_tmp	# Note: use copy module to execute source and destination
2. You can use Python to parse YAML
>>> import yaml
>>> fp = open("ansible_playbook_sc.yaml")
>>> dict = yaml.load(fp)
>>> dict
[{'hosts': 'all', 'remote_user': 'root', 'tasks': [{'name': 'up file', 'copy': 'src=/etc/passwd dest=/tmp/passwd_tmp'}]}]
#Note: Use copy module to specify src and dest

Example: Multiple Operations
--------------------------------------------------------------------------------------------
[root@cPen_A ~]# vim ansible_playbook_sc.yaml					# Note: Write
- hosts: all
  remote_user: root
  tasks:
  - name: up file
    copy: src=/etc/passwd dest=/tmp/passwd_tmp
  - name: download redis
    yum: name=redis state=installed
- hosts: webser
  tasks:
  - name: remove file
    shell: rm -rf /tmp/passwd_tmp
[root@cPen_A ~]# ansible-playbook ansible_playbook_sc.yaml 		# Note: Run

VII. Summary#

Ansible can manage the configuration of remote hosts through commonly used modules in the command line, without the need for a proxy client program, but requires the target host to have ssh and python2.4+; based on the ssh protocol, it can use both username and password or private key, it is recommended to use private key;
On Windows, you need to install PowerShell and winrm service to achieve this; regarding this aspect, you can refer to my previous blog Ansible Automated Management of Windows
Use ansib-doc to obtain module information and specific module help information;
ansible-playbook is configured based on YAML syntax; it can test, parse, and execute playbook files applied to specified remote hosts; it is very convenient for us to uniformly orchestrate and distribute management of remote hosts;

VIII. Exercises#

Example: Check if /etc/passwd file has been modified every day at 1 AM and generate a check report

Example: Write a script to implement ansible node (managed host) server backup, back up /var/log/messages log to /backup directory, named 2020-11-25-01-log.tar.gz, executed once every hour
############################################################################################
Example: Check if /etc/passwd file has been modified every day at 1 AM and generate a check report
-----------------------------------------------------------------------------------------------------------
Method 1: md5sum check if the file has been modified md5sum -c --status
check_file.sh
md5sum -c --status d.md5 &&echo "File has not been modified" ||echo "File has been modified"
md5sum /etc/passwd &> d.md5			#Note: the file name must be d.md5
-----------------------------------------------------------------------------------------------------------
Method 2
check_ectpasswd.sh
a='md5sum /etc/passwd'				#Note: current md5 value
b='tail -n1 /root/b' 				#Note: last md5 value
[[ $a==$b ]]&&echo "File has not been modified!" >> /root/c||echo "File has been modified!" >> /root/c
$a  >> /root/b
-----------------------------------------------------------------------------------------------------------
Method 3
touch /tmp/passwd.log
touch /tmp/passwd.txt
a=`md5sum /etc/passwd`
b=`md5sum /tmp/passwd.txt`
c=${a:0:32}							#Note: because md5sum is arbitrary length input, fixed length output
d=${b:0:32}
if [ $c = $d ] 
then
    echo "$(date) File has not been modified" >>/tmp/passwd.log  
    cat /etc/passwd >/tmp/passwd.txt
else
    echo "$(date) File has been modified" >>/tmp/passwd.log
    cat /etc/passwd >/tmp/passwd.txt
fi 
===========================================================================================================
ansible_playbook_zuoye.yaml
 - hosts: all
   remote_user: root
   tasks: 
   - name: check file
     cron: hour=1 day=*/1 job='/usr/bin/bash ~/check_file.sh'
#Note: The script needs to be on each virtual machine; this step is not written

Example: Write a script to implement ansible node (managed host) server backup, back up /var/log/messages log to /backup directory, named 2020-11-25-01-log.tar.gz, executed once every hour
 - hosts: all
   remote_user: root
   tasks: 
   - name: backup messages
     cron: hour=1 job='tar czf /backup/$(date +%Y-%m-%d-%H-log.tar.gz) /var/log/messages'

Example: md5sum check if the file has been modified md5sum -c --status
[root@cPen_A lianxi]# man md5sum
       -c, --check
              read MD5 sums from the FILEs and check them
       --status
              don't output anything, status code shows success
[root@cPen_A lianxi]# md5sum /etc/passwd > d.md5		#Note: write into this file, the file name must be this, cannot change
[root@cPen_A lianxi]# cat d.md5 
28b2503e7fb565ddc0b8ec1f9ad6a9c7  /etc/passwd
[root@cPen_A lianxi]# md5sum -c --status d.md5			#Note: check if the file has been modified
[root@cPen_A lianxi]# echo $?							#Note: if the file has not been modified, return 0
0
[root@cPen_A lianxi]# useradd chen223344
[root@cPen_A lianxi]# md5sum -c --status d.md5
[root@cPen_A lianxi]# echo $?							#Note: if the file has been modified, return 1
1