Automate Server Setup: VMware, Linux, Ansible, Terraform

by Pedro Alvarez 57 views

Hey guys! Ever found yourself setting up multiple servers on virtual machines and thought, "There has to be a better way?" You're not alone! Manually configuring each server can be tedious and time-consuming, especially when dealing with multiple instances. But fear not! This guide will walk you through automating the process, saving you time and headaches.

Why Automate Server Setup?

Automating server setup offers numerous benefits, making it a crucial skill for any sysadmin or developer. Imagine setting up dozens, or even hundreds, of servers manually. The thought alone is exhausting, right? Automation swoops in to save the day, offering:

  • Time Savings: This is the big one! Automating the process drastically reduces the time it takes to provision servers. Instead of spending hours on each server, you can have them up and running in minutes.
  • Consistency: Manual configuration is prone to errors. We're all human, after all! Automation ensures consistent configurations across all your servers, minimizing the risk of discrepancies and configuration drift. This consistency leads to a more stable and predictable environment.
  • Scalability: Need to scale up your infrastructure quickly? Automation makes it a breeze. You can easily provision new servers with the same configuration as existing ones, allowing you to handle increased demand without breaking a sweat. Think of it as cloning servers, but with all the setup done for you!
  • Reduced Errors: As mentioned earlier, manual configuration is error-prone. Automation scripts, when properly written and tested, significantly reduce the chances of human error, leading to a more reliable infrastructure. No more typos causing major outages!
  • Repeatability: Automation allows you to easily reproduce your infrastructure setup. This is crucial for disaster recovery, testing, and development environments. You can spin up identical environments whenever you need them, ensuring consistency across the board. It's like having a blueprint for your entire server setup.

In short, automating your server setup is a game-changer. It's not just about saving time; it's about building a more robust, scalable, and reliable infrastructure.

Tools of the Trade: VMware, Fedora, Ansible, and Terraform

Before we dive into the how-to, let's talk about the tools we'll be using. Each tool plays a specific role in our automation journey:

  • VMware Fusion: VMware Fusion is a virtualization platform that allows you to run virtual machines on your local computer. It's a popular choice for developers and sysadmins who need to test and develop applications in a virtualized environment. Think of it as a sandbox where you can experiment with different operating systems and configurations without affecting your main system. VMware Fusion provides a user-friendly interface for creating and managing virtual machines, making it a great starting point for learning about virtualization.
  • Fedora Server: Fedora Server is a Linux distribution known for its focus on free and open-source software and its cutting-edge features. It's a solid choice for server deployments, offering a stable and secure platform for running various applications. Fedora is community-driven, which means you'll find plenty of resources and support available online. It's also known for its frequent updates, ensuring you have access to the latest security patches and software versions. Using Fedora as our server OS gives us a reliable foundation for our automated setup.
  • Ansible: Ansible is a powerful automation tool that simplifies configuration management, application deployment, and task automation. It uses a simple, human-readable language (YAML) to define automation tasks, making it relatively easy to learn and use. Ansible is agentless, meaning it doesn't require any software to be installed on the target servers. It communicates with servers over SSH, making it secure and efficient. With Ansible, you can define the desired state of your servers, and Ansible will take care of bringing them into that state. This is the core of our automation strategy – ensuring our servers are configured exactly how we want them.
  • Terraform: Terraform is an infrastructure-as-code (IaC) tool that allows you to define and provision infrastructure using code. It supports various cloud providers and virtualization platforms, including VMware. With Terraform, you can describe your entire infrastructure – virtual machines, networks, storage, etc. – in a configuration file, and Terraform will handle the provisioning and management of these resources. Think of it as a blueprint for your entire infrastructure. Terraform allows you to version control your infrastructure, making it easy to track changes and roll back to previous configurations if needed. For our purposes, Terraform will help us create the virtual machines on VMware Fusion, setting the stage for Ansible to configure them.

These tools work together seamlessly to provide a comprehensive automation solution. Terraform creates the virtual machines, and Ansible configures them. It's a powerful combination that can significantly streamline your server setup process.

Step-by-Step Guide to Automating Fedora Server Setup on VMware Fusion

Okay, let's get our hands dirty! Here's a step-by-step guide to automating Fedora server setup on VMware Fusion using Terraform and Ansible.

1. Prerequisites

Before we start, make sure you have the following installed:

  • VMware Fusion: Download and install VMware Fusion on your macOS system.
  • Terraform: Download and install Terraform from the official website (https://www.terraform.io/downloads). Make sure to add the Terraform executable to your system's PATH.
  • Ansible: Install Ansible using pip: pip install ansible. You might need to use pip3 depending on your Python setup.
  • Fedora Server ISO: Download the Fedora Server ISO image from the Fedora website (https://getfedora.org/server/download/).

2. Setting up Terraform

First, we'll use Terraform to create the virtual machines on VMware Fusion.

  1. Create a Terraform Configuration File: Create a new directory for your Terraform project and create a file named main.tf inside it.

  2. Define the Provider: Add the following code to main.tf to configure the VMware vSphere provider:

terraform {
  required_providers {
    vsphere = {
      source  = "hashicorp/vsphere"
      version = "~> 2.0"
    }
  }
}

provider "vsphere" {
  user           = "your_vsphere_username" # Replace with your vSphere username
  password       = "your_vsphere_password" # Replace with your vSphere password
  vsphere_server = "your_vsphere_server_address" # Replace with your vSphere server address
  allow_unverified_ssl = true # Only for testing, not recommended for production
}

Remember to replace the placeholder values with your actual VMware Fusion credentials. The allow_unverified_ssl = true is for testing purposes and should not be used in production environments.

  1. Define the Virtual Machine: Add the following code to main.tf to define the Fedora Server virtual machine:
data "vsphere_datacenter" "dc" {
  name = "your_datacenter_name" # Replace with your Datacenter name
}

data "vsphere_datastore" "datastore" {
  name          = "your_datastore_name" # Replace with your Datastore name
  datacenter_id = data.vsphere_datacenter.dc.id
}

data "vsphere_network" "network" {
  name          = "your_network_name" # Replace with your Network name
  datacenter_id = data.vsphere_datacenter.dc.id
}

data "vsphere_compute_cluster" "cluster" {
  name          = "your_cluster_name" # Replace with your Cluster name
  datacenter_id = data.vsphere_datacenter.dc.id
}

resource "vsphere_virtual_machine" "fedora_server" {
  name             = "fedora-server-01"  # VM Name
  resource_pool_id = data.vsphere_compute_cluster.cluster.resource_pool_id
  datastore_id     = data.vsphere_datastore.datastore.id

  num_cpus = 2       # Number of CPUs
  memory   = 2048    # Memory in MB
  guest_id = "fedora33_64Guest" # Guest OS ID

  network_interface {
    network_id = data.vsphere_network.network.id
    adapter_type = "e1000"
  }

  disk {
    label            = "disk0"
    size             = 20    # Disk Size in GB
    eagerly_scrub    = false
    thin_provisioned = true
  }

  cdrom { #Attaching the Fedora ISO
    client_device = false # set true if you wish to provide iso path in path property instead of datastore_id/path
    datastore_id = data.vsphere_datastore.datastore.id # datastore id
    path = "[datastore1] ISOs/Fedora-Server-netinst-x86_64-39-1.5.iso" # datastore path of ISO
  }

 clone {
   template_uuid = data.vsphere_virtual_machine.template.id # We are creating template resource just to make terraform configuration easier.
   linked_clone = false
   ovf_deploy = false
 }

  # Customize VM configuration using customization block
 customization {
    linux_options {
      host_name = "fedora-server-01"
      domain    = "localdomain"
    }

    # network configuration, here we are setting up IP and Gateway
    network_interface {
      ipv4_address   = "192.168.1.100" # Ip of VM
      ipv4_netmask   = 24           # netmask
    }

    ipv4_gateway = "192.168.1.1"  # gateway
    dns_server_list = ["8.8.8.8", "8.8.4.4"] # dns server addresses

    # We have to make sure that this runs before Ansible.
    # We are also setting up the username and password in customization block for the VM using the below code block
    # Password should be of minimum length 8 chars
    # Customization script that gets executed at the end
    # User data is your cloud-init file content
    # As password has a minimum length of 8 chars, we cannot use password with less chars here
    # This is the method of setting password in Vmware Fusion through terraform customization
    # In this way we have to make sure to keep password minimum 8 chars long

    # This script will create a new user
    # user data can be also used as cloud init yaml configuration
    # But, here we are using bash to create the user
    # Use base64 to encode it
    # base64 -w 0 user_data.sh > user_data.b64
    # cat user_data.b64
    # decode base64 -d user_data.b64 > user_data
    # sh user_data
    # To use this make below changes
    # Edit the file user_data.sh
    # Encode the file with base64
    # Save the file content in user_data.b64 and provide the content to user_data property
    provisioner "file" {
    destination = "/tmp/customize_vm.sh"
    content     = <<EOF
#!/bin/bash

# Set root password
echo 'root:YourPassword' | chpasswd

# Create a new user and set the password
useradd automation #username
echo 'automation:YourPassword' | chpasswd

# Allow password authentication
sed -i 's/^PasswordAuthentication no/PasswordAuthentication yes/' /etc/ssh/sshd_config

# Restart SSH service
systemctl restart sshd

# Enable passwordless sudo for the automation user
echo "automation ALL=(ALL) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/automation
chmod 0440 /etc/sudoers.d/automation

EOF
    }
    provisioner "remote-exec" {
    inline = [
    "chmod +x /tmp/customize_vm.sh",
    "sudo /tmp/customize_vm.sh"
    ]
  }
  }  

}

data "vsphere_virtual_machine" "template" {
  name          = "Your_Template_Name"  # Your OS template to clone from if you have it
  datacenter_id = data.vsphere_datacenter.dc.id
}

Again, remember to replace the placeholder values (e.g., your_datacenter_name, your_datastore_name, your_network_name, your_cluster_name, Your_Template_Name,YourPassword , IP address) with the appropriate values for your VMware Fusion environment. If you don't have a template, comment out the clone block and look for a way to create it using the ISO instead. If you choose to clone, make sure the OS template you specified has cloud-init installed. This will prevent the need for a customization block.

  1. Initialize Terraform: Run terraform init in the directory containing main.tf to initialize the Terraform project and download the necessary providers.

  2. Apply the Configuration: Run terraform apply to create the virtual machine. Terraform will show you a plan of the changes it will make and ask for confirmation. Type yes to proceed.

3. Setting up Ansible

Now that we have a virtual machine running Fedora Server, we'll use Ansible to configure it.

  1. Create an Ansible Inventory File: Create a file named inventory in your project directory and add the following content:
[fedora_servers]
192.168.1.100 ansible_user=automation ansible_ssh_pass=YourPassword ansible_connection=ssh ansible_ssh_private_key_file=~/.ssh/id_rsa

Replace 192.168.1.100 with the IP address of your Fedora Server VM and YourPassword with the password you set during the customization script. Also, replace ~/.ssh/id_rsa with the path to your SSH private key if you're using key-based authentication. If you have enabled password authentication you can omit ansible_ssh_private_key_file=~/.ssh/id_rsa completely.

  1. Create an Ansible Playbook: Create a file named playbook.yml in your project directory and add the following content:
--- 
- hosts: fedora_servers
  become: true
  tasks:
    - name: Install updates
      yum: 
        name: '*'
        state: latest
    - name: Install common packages
      yum:
        name: 
          - vim
          - wget
          - git
        state: present
    - name: Start and enable firewalld
      service:
        name: firewalld
        state: started
        enabled: true
    - name: Allow SSH through the firewall
      firewalld:
        service: ssh
        permanent: true
        state: enabled
      notify:
        - Reload firewalld

  handlers:
    - name: Reload firewalld
      service:
        name: firewalld
        state: reloaded

This playbook performs the following tasks:

  • Installs updates on the server.
  • Installs common packages like vim, wget, and git.
  • Starts and enables the firewalld service.
  • Allows SSH traffic through the firewall.
  1. Run the Ansible Playbook: Run the following command in your project directory to execute the playbook:
ansible-playbook -i inventory playbook.yml

Ansible will connect to your Fedora Server VM and execute the tasks defined in the playbook. You'll see output indicating the progress of each task.

4. Automating Multiple Servers

Now that we have a working setup for a single server, let's scale it up to multiple servers. This is where the real power of automation shines!

  1. Modify the Terraform Configuration: In main.tf, modify the vsphere_virtual_machine resource to use a count parameter. This allows us to create multiple instances of the same VM.
resource "vsphere_virtual_machine" "fedora_server" {
  count = 3  # Number of servers to create
  name             = "fedora-server-${count.index + 1}" # VM Name
  resource_pool_id = data.vsphere_compute_cluster.cluster.resource_pool_id
  datastore_id     = data.vsphere_datastore.datastore.id

  num_cpus = 2       # Number of CPUs
  memory   = 2048    # Memory in MB
  guest_id = "fedora33_64Guest" # Guest OS ID

  network_interface {
    network_id = data.vsphere_network.network.id
    adapter_type = "e1000"
  }

  disk {
    label            = "disk0"
    size             = 20    # Disk Size in GB
    eagerly_scrub    = false
    thin_provisioned = true
  }

 clone {
   template_uuid = data.vsphere_virtual_machine.template.id # We are creating template resource just to make terraform configuration easier.
   linked_clone = false
   ovf_deploy = false
 }

  # Customize VM configuration using customization block
 customization {
    linux_options {
      host_name = "fedora-server-${count.index + 1}"
      domain    = "localdomain"
    }

    # network configuration, here we are setting up IP and Gateway
    network_interface {
      ipv4_address   = cidrsubnet("192.168.1.0/24", 8, count.index + 100) # Ip of VM
      ipv4_netmask   = 24           # netmask
    }

    ipv4_gateway = "192.168.1.1"  # gateway
    dns_server_list = ["8.8.8.8", "8.8.4.4"] # dns server addresses

    # We have to make sure that this runs before Ansible.
    # We are also setting up the username and password in customization block for the VM using the below code block
    # Password should be of minimum length 8 chars
    # Customization script that gets executed at the end
    # User data is your cloud-init file content
    # As password has a minimum length of 8 chars, we cannot use password with less chars here
    # This is the method of setting password in Vmware Fusion through terraform customization
    # In this way we have to make sure to keep password minimum 8 chars long

    # This script will create a new user
    # user data can be also used as cloud init yaml configuration
    # But, here we are using bash to create the user
    # Use base64 to encode it
    # base64 -w 0 user_data.sh > user_data.b64
    # cat user_data.b64
    # decode base64 -d user_data.b64 > user_data
    # sh user_data
    # To use this make below changes
    # Edit the file user_data.sh
    # Encode the file with base64
    # Save the file content in user_data.b64 and provide the content to user_data property
    provisioner "file" {
    destination = "/tmp/customize_vm.sh"
    content     = <<EOF
#!/bin/bash

# Set root password
echo 'root:YourPassword' | chpasswd

# Create a new user and set the password
useradd automation #username
echo 'automation:YourPassword' | chpasswd

# Allow password authentication
sed -i 's/^PasswordAuthentication no/PasswordAuthentication yes/' /etc/ssh/sshd_config

# Restart SSH service
systemctl restart sshd

# Enable passwordless sudo for the automation user
echo "automation ALL=(ALL) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/automation
chmod 0440 /etc/sudoers.d/automation

EOF
    }
    provisioner "remote-exec" {
    inline = [
    "chmod +x /tmp/customize_vm.sh",
    "sudo /tmp/customize_vm.sh"
    ]
  }
  }  

}

data "vsphere_virtual_machine" "template" {
  name          = "Your_Template_Name"  # Your OS template to clone from if you have it
  datacenter_id = data.vsphere_datacenter.dc.id
}

We've added count = 3 to create three servers. We've also modified the name and ipv4_address to include the count.index so that each server has a unique name and IP address.

  1. Apply the Terraform Configuration: Run terraform apply again to create the additional servers. Terraform will create the new VMs according to the updated configuration.

  2. Modify the Ansible Inventory: Update the inventory file to include the IP addresses of the new servers. You can use a range of IP addresses to simplify the configuration.

[fedora_servers]
192.168.1.100 ansible_user=automation ansible_ssh_pass=YourPassword ansible_connection=ssh
192.168.1.101 ansible_user=automation ansible_ssh_pass=YourPassword ansible_connection=ssh
192.168.1.102 ansible_user=automation ansible_ssh_pass=YourPassword ansible_connection=ssh

Or, if your IP addresses are sequential, you can use a range:

[fedora_servers]
192.168.1.[100:102] ansible_user=automation ansible_ssh_pass=YourPassword ansible_connection=ssh
  1. Run the Ansible Playbook: Run ansible-playbook -i inventory playbook.yml to configure all the servers. Ansible will now connect to each server and execute the tasks in the playbook.

Tips and Best Practices

  • Version Control: Use a version control system like Git to track changes to your Terraform and Ansible configurations. This allows you to easily revert to previous versions if something goes wrong and collaborate with others on your infrastructure code.
  • Modularization: Break down your Terraform and Ansible configurations into smaller, reusable modules. This makes your code easier to understand, maintain, and reuse across different projects.
  • Testing: Test your automation scripts thoroughly before deploying them to production. Use tools like Molecule to test Ansible roles and modules. This helps catch errors early and prevents unexpected issues in your infrastructure.
  • Security: Secure your infrastructure by using SSH key-based authentication, limiting access to your servers, and regularly updating your software. Follow security best practices for both Terraform and Ansible.
  • Idempotency: Ensure your Ansible playbooks are idempotent, meaning they can be run multiple times without changing the system state. This is crucial for ensuring consistency and preventing unexpected behavior.

Conclusion

Automating server setup on virtual machines is a valuable skill that can save you time, reduce errors, and improve the scalability of your infrastructure. By using tools like Terraform and Ansible, you can streamline the provisioning and configuration process, making it easier to manage multiple servers. So, ditch the manual setup, embrace automation, and make your life as a sysadmin or developer a whole lot easier! Happy automating!