Fix AWS EC2 Booting Issues with incorrect fstab entries

Goal This project to fix the EC2 booting issues using below methods

1. EC2 Serial Console
2. Rescue instance

Pre-Requisites:

1. You must have an AWS account to create infrastructure resources on AWS cloud.

Booting Issues

1. In this problem statement you are fixing the incorrect entries in the /etc/fstab which is causing the booting to get halted and not to initialize the network for you to login.

Reproduce the Issue:

1. First you create a problem with incorrect entries in the /etc/fstab.
2. Login to AWS and create 5GB EBS volume in the same AZ of the EC2 instance and attach EBS volume with the name /dev/xvdf to the Instance.
3. Create File System ext4  on the new EBS volume attached in the previous step
4. Mount the File System on /data directory
5. Verify File System utilization using ‘df -h’ command – This command must show /data file system
6. Create file ‘f1’ in the /data file system.
7. Now to auto mount this file system after the EC2 restart the File System entry must be updated in the /etc/fstab – Add below [note the below syntax is incorrect and ec2 boot issues will arise ]
   1. /dev/xvdf /data ext4 defaults 1  [Note that some times the volume name could be nvme1nx on nitro system architecture systems]
8. Restart EC2 Instance from AWS console. 
9. Now Verify the EC2 status from instance screenshot that instance halted in booting process and you can not connect to EC2 instance from SSH client.

Fix the Booting Issues:

Use below methods to fix the booting issues based on the type of the issues that are reported by the EC2 instance. You can get the issue type from the Instance screenshot.

EC2 Serial Console

1. Open Serial Console from AWS console to start the EC2 instance without network services like SSH will be started .
2. Once the Serial Console is available follow the steps to fix the incorrect /etc/fstab entries.
   1. /dev/xvdf /data ext4 defaults 1 1 [Note that some times the volume name could be nvme1n1 on nitro system architecture systems]

Rescue instance

1. Stop original problematic EC2 Instance
2. Detach Root Volume
3. Launch new EC2 instance which will be now a Rescue Instance
4. Attached the root volume  that was detached in step 2 as data volume to Rescue Instance
5. Mount as file system without being formatted.
6. Correct the fstab syntax issues
   1. /dev/xvdf /data ext4 defaults 1 1 [Note that some times the volume name could be nvme1n1 on nitro system architecture systems]
7. Detach the volume from the rescue instance.
8. Terminate the Rescue Instance as this instance is not required anymore.
9. Now, Attach the root volume back again to the original EC2 Instance.
10. Start EC2 instances. Now the Instance should come up cleanly. 

Validation

1. Verify that the EC2 instance is now started successfully.

Please comment below which type of issues can be fixed in Serial Console and Which issues can be fixed in using Rescue Instance. 

Solution

Follow the LINK to check for the solution. 

© Edwiki Trainings – Click HERE If you are interested to learn more on Cloud & DevOps stack.