GitLab Runner on AWS

Hello, my fellow readers it’s been a long time that I am bringing you a new article. In this article I am going to talk about how to setup GitLab runner on AWS platform. First let me explain you what this GitLab is and GitLab runners.

What is Gitlab? 🤔

GitLab is indeed a widely recognized and popular platform in the IT industry, particularly among software development teams. Its comprehensive set of features for version control, collaboration, continuous integration/continuous deployment (CI/CD), issue tracking, and more. GitLab’s open-source nature also contributes to its popularity, as it allows for flexibility, customization, and community-driven development.

GitLab’s emphasis on DevOps practices and its ability to integrate with other tools and services further solidify its position as a key player in the software development ecosystem.

GitLab’s key features include:

• Git Repository Management: GitLab offers robust Git repository management with features such as branching, merging, and tagging to support collaborative development workflows.
• Issue Tracking: It provides built-in issue tracking capabilities to help teams manage tasks, bugs, and feature requests efficiently. Users can create, assign, and track issues directly within GitLab.
• Continuous Integration/Continuous Deployment (CI/CD): GitLab includes a powerful CI/CD pipeline system that automates the build, test, and deployment processes. It allows developers to define custom pipelines and automate the delivery of their applications.
• Code Review: GitLab facilitates code review processes with features like merge requests, inline commenting, and code discussions. This helps teams collaborate effectively and maintain code quality.
• Scalability and Flexibility: GitLab can be deployed on-premises or in the cloud, providing flexibility to organizations with different infrastructure requirements. It offers scalability to support small teams as well as large enterprises.

Overall, GitLab is a comprehensive DevOps platform that offers a wide range of features and capabilities to support the entire software development lifecycle, from version control and collaboration to continuous integration and deployment.

What is Gitlab Runner? 🤔

Image Source: GitLab CI/CD runner clean-up with pre-build scripts — DEV Community

GitLab Runner is an open-source project that is part of the GitLab ecosystem. It is a lightweight, highly scalable agent that runs CI/CD jobs. These jobs can include tasks like building, testing, and deploying code. GitLab Runner can be installed on various platforms including Linux, macOS, and Windows.

GitLab Runner components:

• Runner Executor: This component is responsible for running the jobs defined in the GitLab CI/CD configuration. It could be a shell executor, Docker executor, Kubernetes executor, etc.
• Runner: The Runner is the binary that you install on a machine to enable it to execute jobs. It communicates with the GitLab instance to receive jobs and send back the results.
• Coordinator: The GitLab instance acts as the coordinator, sending jobs to registered Runners and collecting the results.
• Runner Configuration File: This file specifies how the Runner should behave, including details like where to send the results, how to execute jobs, which tags to listen to, etc.
• Runner Registration Token: To connect a Runner to a GitLab instance, you need a registration token. This token is provided by GitLab and ensures that only authorized Runners can execute jobs for a specific project.

These components work together to automate the execution of CI/CD pipelines defined in the GitLab CI/CD configuration file, providing continuous integration and deployment capabilities for software development projects.

In the top article I have mentioned that GitLab runner can be install on any platforms. Now let’s configure a GitLab runner on AWS platform. Using AWS to create GitLab runners offers several advantages:

• Scalability: AWS provides elastic infrastructure, allowing you to easily scale up or down based on your project’s needs. You can configure GitLab runners to dynamically spin up instances when there is a workload and terminate them when not in use, optimizing resource utilization and costs.
• Reliability: AWS offers a highly reliable infrastructure with redundancy built-in across various services. This ensures that your GitLab runners have high availability, reducing the risk of downtime due to infrastructure failures.
• Integration: AWS provides integrations with other AWS services such as Amazon EC2, Amazon S3, Amazon ECR, and AWS Lambda. You can leverage these integrations to build robust CI/CD pipelines within GitLab, utilizing AWS services for artifact storage, container registry, serverless computing, and more.
• Managed Services: AWS offers managed services such as AWS Fargate and AWS Lambda, which abstract away the underlying infrastructure management tasks. You can use these services as GitLab runners to focus more on developing and deploying applications rather than managing infrastructure.

Overall, using AWS for creating GitLab runners provides flexibility, scalability, reliability, and integration capabilities to streamline your CI/CD workflows and improve the efficiency of your development processes.

Let’s create a GitLab Runner on AWS.

Setting up the AWS environment

• First you need to create an EC2 instance. You can create it using console or running following command in AWS CLI.

aws ec2 run-instances --image-id <your_image> -count 1 --instance-type t2.micro --key-name <your_keypair> --security-group-ids <your sg> --subnet-id <your subnet>

• After creating your EC2 instance make sure to it is up and running.

• Now connect to your EC2 instance, and switch to the root user.

• Now let’s install components for our instance, using run following commands

# Install git
yum install git -y

# Download GitLab Runner Binary
curl -L --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64

# Set permissions for GitLab Runner
chmod +x /usr/local/bin/gitlab-runner

# Create GitLab Runner user
useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash

# Install and run 
gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner
gitlab-runner start

Now we have config and install the runner into the EC2. Let’s config runner on GitLab.

• Go to your GitLab repository and click settings. Under CI/CD you can see Runners and click expand.

• Before creating private runner make sure to disable the public runner by clicking the toggle button. Then click New project runner.
• In the creation page select OS as Linux and add a tag.

• After click create runner, you will get following command. Make sure to copy it and save it.

• In the runner's page, you can see our created runner with a warning.

• Now we need configure our runner with EC2 instance. For that go to your EC2 instance and check and GitLab Runner service is up and running. You can run following command for that.

gitlab-runner status

• Now run the following command that you copied previously.

gitlab-runner register --url https://gitlab.com/  --registration-token glrt-FWyVJRcZrL4veCg5x_RU

• Now you will get following output.

• Now go to the GitLab runners page and you can see that our GitLab runner is connect with EC2 instance.

• Now let’s create a simple gitlab-ci yml to test our GitLab runner.

myjob:
    stage: build
    tags: 
        - gitlab-runner
    script:
        - echo "Amazon Web Service"
        - hostname

• Now let’s run our job and you can see the following output.

Conclusion

So, in this article we have create our own private GitLab runner on AWS environment. And we have talked about advantages of private GitLab runners.

In the next article I’ll show you how to integrate Terraform Cloud and AWS. If you need any help follow me on GitHub, LinkedIn or Twitter. looking forward to your suggestions

Thank you all…! Have a great day. don’t forget to leave a 👏✌️❤️