Linux Administrator Real time issues

  1. How to rejoin the server to domain ?
    • mv /etc/sssd/sssd.conf /etc/sssd/sssd.conf_bak$(date +%F)
    • realm leave
    • rm -rf /etc/krb5.keytab
    • realm join –user=(AD ADMIN OU ID) < FQDN of DNS Server >
    • systemctl stop sssd; rm -rf /var/lib/sss/db/* ; mv /etc/sssd/sssd.conf_bak$(date +%F) /etc/sssd/sssd.conf ; systemctl start sssd

2. User domain id not visible in the server

Below are the steps we need to perform :

  1. cd /var/lib/sss/
  2. cp -pr db db_bkp
  3. cp -pr mc mc_bkp
  4. ls -ltr
  5. systemctl stop sssd ; rm -rf /var/log/sssd/* /var/lib/sss/{db,mc}/* ; systemctl start sssd
  6. journalctl -xeu sssd.service –no-pager |tail -40
  7. sssd -d 10 -i
  8. chmod -R 755 /var/lib/sss/
  9. cd db_bkp/
  10. ls -ltr
  11. cp config.ldb /var/lib/sss/db/
  12. ls -ltr /var/lib/sss/db/config.ldb
  13. systemctl start sssd
  14. systemctl status sssd
  15. id -a <Domain ID>


3. User is not able to login with domain id,How to fix the below error when we see in the sssd status ?

[root@xxx002 ~]# systemctl status sssd
โ— sssd.service – System Security Services Daemon
Loaded: loaded (/usr/lib/systemd/system/sssd.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2025-09-23 09:18:53 EDT; 20h ago
Main PID: 2214422 (sssd)
Tasks: 5 (limit: 190818)
Memory: 111.1M
CGroup: /system.slice/sssd.service
โ”œโ”€2214422 /usr/sbin/sssd -i –logger=files
โ”œโ”€2214423 /usr/libexec/sssd/sssd_be –domain TCSdev.dev –uid 0 –gid 0 –logger=files
โ”œโ”€2214424 /usr/libexec/sssd/sssd_nss –uid 0 –gid 0 –logger=files
โ”œโ”€2214425 /usr/libexec/sssd/sssd_pam –uid 0 –gid 0 –logger=files
โ””โ”€2214426 /usr/libexec/sssd/sssd_pac –uid 0 –gid 0 –logger=files

Sep 23 09:18:53 xxx002.motivadev.dev systemd[1]: Started System Security Services Daemon.
Sep 23 09:19:44 xxx002.motivadev.dev sssd_be[2214423]: Could not autodiscover AD site value using DNS and ad_site option was not set in configurati>
Sep 23 09:22:48 xxx002.TCSdev.dev sssd_nss[2214424]: Enumeration requested but not enabled
Sep 23 09:24:05 xxx002.TCSdev.dev sssd_be[2214423]: Could not autodiscover AD site value using DNS and ad_site option was not set in configurati>
Sep 23 09:24:39 xxx002.TCSdev.dev sssd_be[2214423]: Could not autodiscover AD site value using DNS and ad_site option was not set in configurati>
Sep 23 09:27:40 xxx002.TCSdev.dev sssd_be[2214423]: Could not autodiscover AD site value using DNS and ad_site option was not set in configurati>
Sep 23 09:34:16 xxx002.TCSdev.dev adcli[2216882]: GSSAPI client step 1
Sep 23 09:34:16 xxx002.TCSdev.dev adcli[2216882]: GSSAPI client step 1
Sep 23 09:34:16 xxx002.TCSdev.dev adcli[2216882]: GSSAPI client step 1
Sep 23 09:36:30 xxx002.TCSdev.dev sssd_be[2214423]: Could not autodiscover AD site value using DNS and ad_site option was not set in configuration>

Add below lines in /etc/sssd/sssd.conf

----
[domain/TCSdev.dev] <-- BELOW THIS
ad_gpo_access_control = disabled <--- add this
ad_gpo_ignore_unreadable = true <-- add this
----

- Restart sssd service and clear cache

----
# systemctl stop sssd ; rm -f /var/lib/sss/db/* /var/log/sssd/* ; systemctl start sssd
----

- Check if user login works now.

4. Unable to install packages and Subscription issues on Server

If you want to register your system to customer portal, please try below commands and share the output of each command to check further.

Update the below parameters in /etc/rhsm/rhsm.conf

Server hostname:

hostname = subscription.rhsm.redhat.com
 
# Server prefix:

prefix = /subscription

##############################################################

# cp /etc/rhsm/rhsm.conf.kat-backup /etc/rhsm/rhsm.conf
# subscription-manager config
# subscription-manager clean 
# yum clean all
# yum repolist -v



After running the clean command, you need to register, so the yum will collect required data from CDN.

 # subscription-manager register 
 # subscription-manager refresh
 # subscription-manager status

 # yum clean all
 # yum repolist -v
 # yum check-update

5.Unable to mount /var/log filesystem & Change Permissions on Production server

Run the below command to fix the issue :

First remount these as read-write.

# mount -o remount,rw /var

# mount -o remount,rw /tmp

# mount -o remount,rw /var/tmp

After this is done, try mounting /var/log and /var/log/audit again

# mount -t ext4 -o rw /dev/mapper/vg_toptech-var_log /var/log

# mount -t ext4 -o rw /dev/mapper/vg_toptech-var_log_audit /var/log/audit

# Reboot the server

6.Unable to Patch RHEL 7.9 Server

Error while doing patching on RHEL 7.9 server

[root@hostname003 ~]# yum update –enablerepo=rhel-7-server-els-rpms -y
Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager
Repo rhel-7-server-els-rpms forced skip_if_unavailable=True due to: %(ca_cert_dir)sredhat-uep.pem
Repo rhel-7-server-rpms forced skip_if_unavailable=True due to: %(ca_cert_dir)sredhat-uep.pem
https://cdn.redhat.com/content/els/rhel/server/7/7Server/x86_64/os/repodata/repomd.xml: [Errno 14] curl#77 – “Problem with the SSL CA cert (path? access rights?)”
Trying other mirror.
https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os/repodata/repomd.xml: [Errno 14] curl#77 – “Problem with the SSL CA cert (path? access rights?)”
Trying other mirror.
No packages marked for update

Below Steps we need to perform for fixing the pacthing issue on RHEL 7.9

Error: More than one release product certificate installed. Certificate paths: /etc/pki/product/69.pem, /etc/pki/product/204.pem

To address this, please run the following commands on host hostname003 to disable the extra key and then try to register this system:

# mv /etc/pki/product/69.pem /etc/pki/product/69.pem.old

# subscription-manager clean

# subscription-manager register –force

# yum repolist

# curl -vvv https://subscription.rhsm.redhat.com –cacert /etc/rhsm/ca/redhat-uep.pem

# curl -vvv https://cdn.redhat.com –cacert /etc/rhsm/ca/redhat-uep.pem

# env|grep -i proxy

check that the /etc/rhsm/rhsm.conf file is missing or not

# mv /etc/rhsm/rhsm.satellite.back /etc/rhsm/rhsm.conf

# subscription-manager clean

# subscription-manager register

# yum clean all

# yum repolist -v

Since this is a RHEL 7.9 system, therefore we also need to enable the RHEL 7 ELS repo on it to get the updates. To do that please execute the below commands.

# subscription-manager refresh

# subscription-manager repos –enable rhel-7-server-els-rpms

# yum clean all

# yum repolist -v

7. LVM commands are not working

We are facing issue while creating lvm on RHEL 9 server.
Below commands are not working.

[root@hostname002 ~]# pvs
-bash: pvs: command not found
[root@hostname002 ~]# pvdisplay
-bash: pvdisplay: command not found
[root@hostname002 ~]# lvdisplay
-bash: lvdisplay: command not found
[root@hostname002 ~]# vgs
-bash: vgs: command not found
[root@hostname002 ~]# vgdisplay
-bash: vgdisplay: command not found
[root@hostname002 ~]#

Resolution :

System seems to be missing the lvm binary files under /sbin folder.

Reinstall the lvm2 package in rescue mode and check to see if all binaries are now made available in /sbin folder.
# yum reinstall lvm2

Root Cause:


lvdisplay display binaries were missing.

ls -al /sbin/lv*

ls: /sbin/lv*: No such file or directory


Diagnostic Steps


Check the package integrity
rpm -aV

# which lvdisplay

/sbin/lvdisplay

# ls -al /sbin/lv*

ls: /sbin/lv*: No such file or directory

8. Server is rebooted with old kernel after patch and reboot

[root@hostname002 boot]# uname -r
3.10.0-1160.108.1.el7.x86_64

[root@hostname002 boot]# awk -F\’ /^menuentry/{print\$2} /etc/grub2.cfg
Red Hat Enterprise Linux Server (3.10.0-1160.108.1.el7.x86_64) 7.9 (Maipo)
Red Hat Enterprise Linux Server (3.10.0-1160.105.1.el7.x86_64) 7.9 (Maipo)
Red Hat Enterprise Linux Server (0-rescue-df37947086814d26b84cd4dde0dbc862) 7.2 (Maipo)
[root@hostname002 boot]#12:15:08

[root@hostname002 boot]# rpm -q kernel
kernel-3.10.0-1160.105.1.el7.x86_64
kernel-3.10.0-1160.108.1.el7.x86_64
kernel-3.10.0-1160.114.2.el7.x86_64
[root@hostname002 boot]#

Resolution :

$ cat installed-rpms | grep kernel
kernel-3.10.0-1160.114.2.el7.x86_64 Tue Mar 26 07:46:34 2024

$ cat sos_commands/yum/yum_history
Loaded plugins: product-id, search-disabled-repos, subscription-manager,
: versionlock

ID | Login user | Date and time | Action(s) | Altered

71 | user1 <user1>      | 2024-03-26 07:46 | E, I, U        |    7 **

Hence we see missing initramfs file , grub entry for latest kernel.

$ cat sos_commands/boot/ls_-lanR_.boot | egrep “vmli|init” | egrep -v “kdump|hmac|rescue|ply”
-rw——- 1 0 0 21878325 Jan 22 05:01 initramfs-3.10.0-1160.105.1.el7.x86_64.img
-rw——- 1 0 0 21841607 Feb 19 05:13 initramfs-3.10.0-1160.108.1.el7.x86_64.img
-rwxr-xr-x 1 0 0 7056216 Nov 6 06:06 vmlinuz-3.10.0-1160.105.1.el7.x86_64
-rwxr-xr-x 1 0 0 7056216 Jan 4 14:02 vmlinuz-3.10.0-1160.108.1.el7.x86_64
-rwxr-xr-x 1 0 0 7056216 Mar 3 07:26 vmlinuz-3.10.0-1160.114.2.el7.x86_64

# yum history info 71

To resolve the issue, reinstall the kernel.

# yum reinstall kernel-3.10.0-1160.114.2.el7.x86_64

# Reboot the server # shutdown -r now

9. AD users are not able to login to the server

User is getting access denied error while trying with domain id :

1) Login into issue server as root.

2) The system needs to be re-joined to fetch the current keytab.

Below are reference steps.

# cp -p /etc/sssd/sssd.conf /etc/sssd/sssd.conf.bkp
# realm leave domain_name
# mv /etc/krb5.keytab /etc/krb5.keytab.bkp
# realm join domain_name
# cp -p /etc/sssd/sssd.conf.bkp /etc/sssd/sssd.conf

4) Restart SSSD and clear cache

  # service sssd stop ; rm -f /var/lib/sss/db/* /var/log/sssd/* ; service sssd start

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top