GitLab Migration To Debian 12: Data Restoration Guide

Hey guys! Migrating GitLab to a new system can seem daunting, especially when you're dealing with external databases and precious data. But don't worry, this guide will walk you through the process of migrating your GitLab instance from an older system to a clean Debian 12 installation, ensuring you don't lose any of your valuable data. We'll cover everything from backing up your existing GitLab instance to restoring it on the new system, with a focus on handling external PostgreSQL databases. So, let's dive in and get your GitLab up and running smoothly on Debian 12!

Preparing for the Migration

Before we even think about touching the new system, let's make sure we have a solid plan for backing up your current GitLab instance. This is the most crucial step, as it ensures that you can recover your data if anything goes wrong during the migration. We'll be focusing on backing up the configuration, database, repositories, and shared files.

Backing Up Your Existing GitLab Instance

Backing up your GitLab instance is the first and most critical step in the migration process. It's like creating a safety net – if anything goes wrong during the migration, you'll have a complete copy of your data to fall back on. This process involves backing up several key components of your GitLab installation, including the configuration files, the database, the repositories, and any shared files. We'll go through each of these components step-by-step to ensure a comprehensive backup.

First, let's talk about the configuration files. These files contain all the settings and customizations you've made to your GitLab instance, such as user settings, authentication methods, and other configurations. You'll want to back these up to preserve your custom setup on the new system. The command gitlab-ctl backup-config is your friend here. This command will create a tar.gz archive containing your GitLab configuration, typically named gitlab-config.tar.gz. Make sure to store this file in a safe location, preferably off the server you're migrating from.

Next, we need to back up the database. Since you mentioned you're using an external database, this step is particularly important. The database holds all your issues, merge requests, users, and other crucial data. For external PostgreSQL databases, you'll need to use pg_dump, a command-line utility for backing up PostgreSQL databases. The command might look something like this: pg_dump -U <db_user> -h <db_host> -p <db_port> <db_name> > gitlab-db-backup.sql. Replace <db_user>, <db_host>, <db_port>, and <db_name> with your actual database credentials. This command will create a SQL dump file (gitlab-db-backup.sql) containing your database schema and data. Again, store this file securely.

Then comes the repositories. Your Git repositories are the heart of your GitLab instance, containing all your project code and history. We'll back these up using the gitlab-backup create command, which is part of the GitLab administration toolkit. This command will create a tar.gz archive containing all your Git repositories, typically named gitlab_backup.tar.gz. However, since we want to keep things organized and match your existing backup structure, we can extract the repositories from this archive and store them separately as gitlab-repos.tar.gz. This approach gives you more granular control over your backups and makes restoration easier.

Finally, let's not forget the shared files. These files include any attachments, avatars, and other files that are stored on the GitLab server. Backing these up ensures that you don't lose any of these important assets. You can usually find these files in the /var/opt/gitlab/gitlab-rails/uploads directory. Create a tar.gz archive of this directory and name it gitlab-shared.tar.gz. This will preserve all your shared files and ensure they're available on the new system.

By backing up these four key components – configuration, database, repositories, and shared files – you'll have a comprehensive backup of your GitLab instance. This will give you peace of mind during the migration process and ensure that you can restore your data if anything goes wrong. Remember, a little preparation goes a long way in ensuring a smooth and successful migration!

Verifying the Backups

Verifying your backups is a crucial step often overlooked, but it's as important as the backup process itself. Think of it as a safety check – you've packed your parachute, but you need to make sure it opens before you jump out of the plane. Verifying your backups ensures that the files you've created are complete, uncorrupted, and can be successfully restored. This step can save you a lot of headaches down the road, as it's much better to discover an issue with your backups before you start the migration process than after.

For the configuration backup (gitlab-config.tar.gz), a simple way to verify it is to try extracting its contents into a temporary directory. You can use the command tar -tzvf gitlab-config.tar.gz to list the files in the archive without extracting them. If you see a list of configuration files and directories, it's a good sign that the backup is intact. For a more thorough check, you can extract the archive using tar -xzvf gitlab-config.tar.gz -C /tmp/gitlab-config-check (or any other temporary directory) and then browse the extracted files to ensure they look as expected.

Verifying the database backup (gitlab-db-backup.sql) is a bit more involved but equally important. You can simulate a restore by creating a temporary database and attempting to import the SQL dump file. First, create a new database in your PostgreSQL server using createdb <temp_db_name> -U <db_user>. Then, import the backup using psql -U <db_user> -d <temp_db_name> -f gitlab-db-backup.sql. If the import completes without errors, it indicates that the backup is likely valid. After the import, you can connect to the temporary database and run some queries to verify that the data is present and consistent.

To verify the repositories backup (gitlab-repos.tar.gz), you can extract the archive into a temporary directory and then try to access the repositories using Git commands. Use tar -tzvf gitlab-repos.tar.gz to list the repositories in the archive. Then, extract the archive using tar -xzvf gitlab-repos.tar.gz -C /tmp/gitlab-repos-check. Navigate into one of the repository directories and run git log to check the commit history. If you see the expected commit history, it's a good sign that the repositories backup is valid.

Finally, for the shared files backup (gitlab-shared.tar.gz), you can use a similar approach to the configuration backup. List the files in the archive using tar -tzvf gitlab-shared.tar.gz and then extract the archive into a temporary directory using tar -xzvf gitlab-shared.tar.gz -C /tmp/gitlab-shared-check. Browse the extracted files to ensure that the attachments, avatars, and other shared files are present.

By taking the time to verify your backups, you're ensuring that you have a reliable copy of your data and that you can successfully restore it on the new system. This extra step can save you from a lot of potential issues and give you the confidence to proceed with the migration.

Setting Up the New Debian 12 System

Now that we've got our backups safely tucked away, it's time to roll up our sleeves and set up the new Debian 12 system. This involves installing Debian 12, configuring the necessary prerequisites, and installing GitLab itself. Think of it as building the foundation for your new GitLab home – a solid foundation ensures a smooth and stable migration.

Installing Debian 12

The first step, of course, is installing Debian 12. You can download the Debian 12 installation image from the official Debian website. During the installation process, you'll be asked to configure various settings, such as the hostname, network settings, and user accounts. When setting up the hostname, choose a name that is easily identifiable and follows your organization's naming conventions. For network settings, ensure that the system has a static IP address if it's going to be a long-term server. This will prevent issues with DNS and connectivity later on.

When creating user accounts, it's a good practice to create a dedicated user account for GitLab administration, rather than using the root account directly. This enhances security and makes it easier to manage permissions. You'll also be prompted to set up a root password – make sure to choose a strong password and store it securely.

During the installation, you'll also be asked to select the software packages to install. For a GitLab server, you'll typically want to install the standard system utilities, SSH server, and any other tools that you commonly use for server administration. You can also choose to install a desktop environment if you prefer a graphical interface, but for a server, a minimal installation without a desktop environment is often recommended for better performance and security.

Once the installation is complete, you'll have a fresh Debian 12 system ready to go. Before we move on to installing GitLab, there are a few prerequisites we need to take care of.

Installing Prerequisites

Before we can install GitLab, we need to make sure our Debian 12 system has all the necessary prerequisites. These are software packages and configurations that GitLab relies on to function correctly. Think of them as the supporting cast that makes the star of the show (GitLab) shine. Installing these prerequisites ensures that GitLab has everything it needs to run smoothly.

First up is PostgreSQL. Since you mentioned that you're using an external PostgreSQL database, you might think we can skip this step. However, GitLab still requires the PostgreSQL client libraries to be installed on the system, even if the database server is running elsewhere. To install the PostgreSQL client libraries, use the command sudo apt-get update && sudo apt-get install -y postgresql-client. This command will update the package lists and then install the PostgreSQL client package.

Next, we need to install other dependencies that GitLab requires, such as curl, openssh-server, ca-certificates, and perl. These packages provide essential functionalities for GitLab, such as network communication, secure shell access, and certificate validation. To install these dependencies, use the command sudo apt-get install -y curl openssh-server ca-certificates perl. This command will install all the necessary packages in one go.

After installing the dependencies, it's a good practice to configure SSH properly. Ensure that SSH is enabled and configured to allow secure access to the server. You might want to configure SSH keys for authentication instead of passwords, as this is more secure. You can also customize the SSH port if you want to add an extra layer of security.

Finally, make sure your firewall is configured correctly. Debian 12 comes with ufw (Uncomplicated Firewall) pre-installed. You'll want to enable the firewall and allow traffic on the necessary ports, such as SSH (port 22), HTTP (port 80), and HTTPS (port 443). You can use the commands sudo ufw enable, sudo ufw allow ssh, sudo ufw allow http, and sudo ufw allow https to configure the firewall.

By installing these prerequisites and configuring the system properly, you're setting the stage for a successful GitLab installation. With the foundation in place, we can now move on to installing GitLab itself.

Installing GitLab

With our Debian 12 system prepped and ready, it's time for the main event: installing GitLab. This is where we bring the GitLab application onto our new system. We'll be using the official GitLab package repository for Debian, which makes the installation process straightforward and ensures we get the latest version.

The first step is to add the GitLab package repository to our system's package sources. This tells apt (the package manager) where to find the GitLab packages. You can do this by running the following command:

curl -sS https://packages.gitlab.com/install/repositories/gitlab/gitlab-ee/script.deb.sh | sudo bash

This command downloads a script from GitLab's package repository and executes it using sudo. The script will add the GitLab package repository to your system's package sources and configure apt to trust the GitLab packages.

Next, we need to install the GitLab package. We'll be installing the Enterprise Edition (EE) version of GitLab, which includes all the features and functionalities. To install GitLab EE, use the following command:

sudo apt-get update && sudo apt-get install gitlab-ee

This command updates the package lists and then installs the gitlab-ee package. During the installation, you'll be prompted to configure various settings, such as the external URL for your GitLab instance. Make sure to set this to the correct URL, as it's crucial for accessing your GitLab instance later.

Once the installation is complete, GitLab will be running on your system. However, before we can start using it, we need to configure GitLab to use our external database. This is a critical step, as we want GitLab to connect to the database we backed up earlier. We'll cover this in detail in the next section.

By installing GitLab from the official package repository, you're ensuring that you have a clean and up-to-date installation. With GitLab installed, we can now focus on configuring it to use our external database and restoring our data.

Restoring GitLab Data

Alright, we've got GitLab installed on our shiny new Debian 12 system. Now comes the moment of truth: restoring your GitLab data. This is where we bring back all your projects, users, issues, and other important information from the backups we created earlier. It's like moving into a new house and unpacking all your belongings – you want everything to be in its place and ready to use.

Configuring GitLab to Use the External Database

Before we restore the database itself, we need to configure GitLab to use your external PostgreSQL database. This involves modifying the GitLab configuration file to point to your database server and provide the necessary credentials. Think of it as telling GitLab where to find its heart and soul – the database.

The main configuration file for GitLab is located at /etc/gitlab/gitlab.rb. You'll need to edit this file using a text editor such as nano or vim. Open the file using the command sudo nano /etc/gitlab/gitlab.rb.

Inside the configuration file, you'll find a section for database settings. You'll need to modify the following settings to point to your external database:

postgresql['enable'] = false
gitlab_rails['db_adapter'] = 'postgresql'
gitlab_rails['db_encoding'] = 'utf8'
gitlab_rails['db_database'] = '<db_name>'
gitlab_rails['db_username'] = '<db_user>'
gitlab_rails['db_password'] = '<db_password>'
gitlab_rails['db_host'] = '<db_host>'
gitlab_rails['db_port'] = <db_port>

Replace <db_name>, <db_user>, <db_password>, <db_host>, and <db_port> with your actual database credentials. Make sure to set postgresql['enable'] to false, as we're using an external database.

After making these changes, save the file and reconfigure GitLab using the command sudo gitlab-ctl reconfigure. This command applies the changes you've made to the configuration file and restarts the necessary GitLab services. It's like giving GitLab a new set of instructions to follow.

Once GitLab is reconfigured, it will be connected to your external database. Now we can move on to restoring the database backup itself.

Restoring the Database

With GitLab configured to use our external database, we can now restore the database backup we created earlier. This is where we bring back all your issues, merge requests, users, and other data that's stored in the database. Think of it as transplanting the heart of your old GitLab instance into the new one.

Before restoring the database, it's a good practice to stop GitLab to prevent any conflicts or data corruption. You can stop GitLab using the command sudo gitlab-ctl stop. This ensures that no processes are writing to the database while we're restoring it.

To restore the database, we'll use the psql command-line utility, just like we did when verifying the backup. The command might look something like this:

psql -U <db_user> -h <db_host> -p <db_port> -d <db_name> -f gitlab-db-backup.sql

Replace <db_user>, <db_host>, <db_port>, and <db_name> with your actual database credentials. This command will import the SQL dump file (gitlab-db-backup.sql) into your database.

Once the restore is complete, you can start GitLab using the command sudo gitlab-ctl start. This will bring GitLab back online and connect it to the restored database. It's like waking up the new GitLab instance with all its memories intact.

After starting GitLab, it's a good idea to check the logs for any errors or warnings. You can find the GitLab logs in the /var/log/gitlab directory. Look for any red flags that might indicate an issue with the database restore.

With the database restored, we're well on our way to a fully migrated GitLab instance. Now we just need to restore the other components: the configuration, repositories, and shared files.

Restoring Configuration, Repositories, and Shared Files

We've successfully restored the database, which is a huge step. Now, let's bring back the remaining pieces of the puzzle: the configuration, repositories, and shared files. Restoring these components ensures that your new GitLab instance is a complete replica of your old one, with all your settings, projects, and files intact.

First up is the configuration. We backed up the configuration files into gitlab-config.tar.gz. To restore these files, we simply need to extract the archive into the /etc/gitlab directory. However, before we do that, it's a good practice to back up the current configuration on the new system. This gives us a fallback in case anything goes wrong during the restore.

You can back up the current configuration by creating a tar.gz archive of the /etc/gitlab directory: sudo tar -czvf gitlab-config-backup.tar.gz /etc/gitlab. This will create a backup file named gitlab-config-backup.tar.gz in your current directory.

Now, we can restore the configuration by extracting the gitlab-config.tar.gz archive into the /etc/gitlab directory. Use the following command:

sudo tar -xzvf gitlab-config.tar.gz -C /etc/gitlab --strip-components=1

The --strip-components=1 option removes the top-level directory from the archive, ensuring that the files are extracted directly into /etc/gitlab. This is important to avoid creating an extra directory level.

After restoring the configuration, you'll need to reconfigure GitLab using the command sudo gitlab-ctl reconfigure. This applies the restored configuration settings and restarts the necessary GitLab services.

Next, let's restore the repositories. We backed up the repositories into gitlab-repos.tar.gz. To restore these, we need to extract the archive into the GitLab repositories directory, which is typically /var/opt/gitlab/git-data/repositories. Before we do that, let's stop GitLab again to prevent any conflicts: sudo gitlab-ctl stop.

Now, we can restore the repositories by extracting the gitlab-repos.tar.gz archive into the repositories directory. Use the following command:

sudo tar -xzvf gitlab-repos.tar.gz -C /var/opt/gitlab/git-data

This command extracts the repositories into the /var/opt/gitlab/git-data directory. GitLab will automatically discover the repositories in this directory.

Finally, let's restore the shared files. We backed up the shared files into gitlab-shared.tar.gz. To restore these, we need to extract the archive into the GitLab uploads directory, which is typically /var/opt/gitlab/gitlab-rails/uploads. As before, let's stop GitLab to prevent any conflicts: sudo gitlab-ctl stop.

Now, we can restore the shared files by extracting the gitlab-shared.tar.gz archive into the uploads directory. Use the following command:

sudo tar -xzvf gitlab-shared.tar.gz -C /var/opt/gitlab/gitlab-rails/uploads

This command extracts the shared files into the /var/opt/gitlab/gitlab-rails/uploads directory.

After restoring the repositories and shared files, we can start GitLab again: sudo gitlab-ctl start. This will bring GitLab back online with all your data and files restored.

By restoring the configuration, repositories, and shared files, we've completed the data restoration process. Your new GitLab instance should now be a complete replica of your old one.

Post-Migration Tasks

We've migrated GitLab to a new Debian 12 system and restored all our data. Woohoo! But hold your horses, we're not quite done yet. There are a few post-migration tasks we need to take care of to ensure everything is running smoothly and securely. Think of these as the finishing touches – they ensure that your new GitLab instance is not only functional but also optimized and secure.

Verifying GitLab Functionality

The first thing we need to do is verify that GitLab is functioning correctly. This involves checking various aspects of GitLab to ensure that everything is working as expected. It's like giving your new house a thorough inspection to make sure all the appliances are working and the lights are on.

Start by accessing your GitLab instance in a web browser. Use the external URL you configured during the installation process. If you can access the GitLab login page, it's a good sign that GitLab is up and running.

Next, log in using your existing credentials. If you can log in successfully, it means that the user authentication is working correctly. Try logging in with different user accounts to ensure that all users can access GitLab.

Once you're logged in, check your projects. Make sure that all your projects are listed and that you can access them. Try browsing the repositories, viewing issues, and creating merge requests. This will verify that the repositories and database are working correctly.

Also, check the shared files, such as avatars and attachments. Make sure that these files are displayed correctly. This will verify that the shared files restoration was successful.

If you encounter any issues during this verification process, check the GitLab logs for errors or warnings. The logs can provide valuable clues about what might be going wrong. You can find the GitLab logs in the /var/log/gitlab directory.

By thoroughly verifying GitLab functionality, you can catch any potential issues early and ensure that your new GitLab instance is working as expected.

Updating GitLab

After migrating to a new system, it's a good idea to update GitLab to the latest version. This ensures that you have the latest features, bug fixes, and security patches. Think of it as giving your new house a fresh coat of paint and installing new locks – it makes it look better and more secure.

To update GitLab, you can use the package manager (apt) since we installed GitLab from the official package repository. First, update the package lists using the command sudo apt-get update. Then, upgrade GitLab using the command sudo apt-get upgrade gitlab-ee. This will download and install the latest version of GitLab.

During the upgrade process, you might be prompted to resolve any configuration conflicts. This can happen if the configuration file has been changed since the last upgrade. Follow the prompts to resolve the conflicts and ensure that your configuration is up-to-date.

After the upgrade is complete, reconfigure GitLab using the command sudo gitlab-ctl reconfigure. This applies any configuration changes that were included in the upgrade and restarts the necessary GitLab services.

By updating GitLab to the latest version, you're ensuring that your new GitLab instance is secure, stable, and has all the latest features.

Securing GitLab

Security is always a top priority, especially after migrating to a new system. We need to secure our GitLab instance to protect our data and prevent unauthorized access. Think of this as installing a security system in your new house – it keeps your belongings safe and secure.

First, enable HTTPS for your GitLab instance. HTTPS encrypts the communication between your browser and the GitLab server, protecting your data from eavesdropping. GitLab provides a tool called letsencrypt that makes it easy to set up HTTPS using Let's Encrypt certificates. You can enable HTTPS by running the command sudo gitlab-ctl letsencrypt enable. Follow the prompts to configure Let's Encrypt and obtain a certificate for your domain.

Next, review your firewall settings. Make sure that only the necessary ports are open. Typically, you'll want to allow traffic on ports 22 (SSH), 80 (HTTP), and 443 (HTTPS). You can use the ufw command to manage your firewall rules.

Also, enable two-factor authentication (2FA) for all user accounts. 2FA adds an extra layer of security by requiring users to enter a code from their mobile device in addition to their password. This makes it much harder for attackers to gain access to your GitLab instance.

Finally, regularly update GitLab to patch any security vulnerabilities. We've already covered how to update GitLab, but it's important to make this a regular habit.

By taking these steps to secure your GitLab instance, you're protecting your data and ensuring that your new GitLab home is safe and secure.

Conclusion

So there you have it! We've successfully migrated your GitLab instance to a clean Debian 12 system, restored all your data, and secured the new installation. It might seem like a lot of steps, but by following this guide, you can ensure a smooth and successful migration. Remember, the key is to plan carefully, back up your data, and verify everything along the way. Now, go forth and enjoy your shiny new GitLab instance on Debian 12!