Gitlab Ci Pipelines Ci Cd And Devops For Beginners

Overview Preview 00:27

What is CI / CD? Preview 08:08

Alternative if you don't want to install new software Preview 00:31

Short introduction to Node.js Preview 01:51

Creating a new project Preview 05:07

Building the project locally Preview 02:34

Short introduction to images and Docker Preview 04:01

Building the project using Gitlab CI Preview 10:57

Adding a test stage Preview 06:23

Running jobs in parallel Preview 05:41

Running jobs in the background Preview 08:09

Debugging the error curl: (7) Failed to connect to localhost port 9000: Connect

While building CI/CD pipelines, things do not always work as expected. For this reason, it is very important to get familiar with the tools you are using and with understanding the console log, step by step. I cannot stress how important this skills is.

Deployment using surge.sh Preview 02:50

Using Environment variables for managing secrets Preview 03:33

Deploying the project using Gitlab CI Preview 04:59

How does Surge.sh know the environment variables? Preview 02:56

Many CLI tools look for existing predefined environment variables to use. This is why we have defined SURGE_LOGIN and SURGE_TOKEN as environment variables in GitLab CI.

Quiz #2

Post-deployment tests

This assignment guides you through adding a new stage of post-deployment tests.

Overview Preview 02:08

Predefined environment variables Preview 07:42

Pipeline triggers / Retrying failed jobs / Pipeline schedules Preview 02:50

Using caches to optimize the build speed Preview 10:01

Assignment: Improving build speed by using caches

In this assignment, we will look into optimizing our cache configuration and making our pipeline run faster.

Cache vs Artifacts Preview 01:52

Deployment Environments Preview 07:43

Defining variables Preview 05:34

Manual deployments / Manually triggering jobs Preview 06:02

Merge requests - Using branches Preview 05:25

Merge requests - What is a Merge Request? Preview 00:21

Merge requests - Configuring Gitlab Preview 01:49

Merge requests - Your first merge request Preview 06:19

Working with Merge Requests

This assignment will give you the possibility of practicing merge requests.

Dynamic environments Preview 06:41

Troubleshooting environment variables not being available Preview 00:22

Destroying environments (Clean-up after the Merge Request) Preview 09:05

before_script & after_script configuration Preview 03:23

Recap & conclusion Preview 01:58

Overview Preview 00:53

Understanding YAML Preview 06:40

- YAML is just a data serialization language which allows us to store different things. In our case, we use YAML to define a pipeline but YAML can be used for many other things (examples ???)

- to put it very simple, YAML can be used to define key - value pairs. For example name: John. But YAML can also store lists or objects.

- this is exactly what we have done so far, but without going too much into the details of what everything is

- while it may not look like, YAML is actually compatible to another format called JSON

- name: John

age: 23

food:

  - pizza

  - donuts

  - coke

friend:

    name: Joe

    age: 30

    food: null

- comments can be created with hashtags # foo

Disabling jobs Preview 01:06

Anchors Preview 05:20

Creating job templates Preview 08:44

Overview Preview 01:58

In this section, we increase the complexity of the application we are building and deploying. We will be building and testing a Java application, and we will deploy it to Amazon Web Services (AWS). I understand if you are not a Java developer or are not interested in using Java or AWS (Amazon Web Services). This section tries to provide you with a realistic example. I can assure it is totally fine just to watch and understand the underlying concepts which can apply to any other technologies and services.

The pipeline we will be building is more complex,  but the same GitLab CI principles will be used.

In this process, you will learn:

- build an advanced CI pipeline with code-style checks, unit tests, API tests, performance tests & security checks

- publish test results (both in HTML and XML format)

- learn about cloud services and how to use AWS for deployments

This means you will be exposed to new tools and technologies that you may not be familiar with. Overall the complexity of this project is much higher than before. As with many new things, a bit of patience is required when things do not work as expected.

As this course is focused on building pipelines with GitLab CI, I cannot give you a full introduction to all the tools and technologies used. I will provide you with links to articles and other video tutorials.

If you want to follow along, that is fantastic. As always, I am here in case you need help or get stuck.

Please note that some lectures may have a Troubleshooting document in the resources folder and can help you fix some common issues.

Are you just as excited to get started? Let's go!

Introduction to the Java application Preview 04:14

This is a simple Java application that represents a simple car fleet management solution. The tool that you see here is an IDE called IntelliJ. I understand that if you are not familiar with IntelliJ, getting this to run on your computer may be challenging. If you can't get it to work, don't worry. This is nice to have but not needed to build the pipeline. Also, check the Resources for some tutorials that can help you get started.

I have already done the programming work, but together we will be building the CI/CD pipeline.

Feel free to clone this repository so that we all have a common basis to get started.

If you won't want to install and run the application locally, no problem. You can work on the .gitlab-ci.yml file without any issues.

What this application does is to expose an API that allows you to add, view, and remove cars from a database.

An API is a program that does not have a graphical interface, like a website, for example. But that API can use used by a front-end application to display the data in a browser.

Calling an API with Postman Preview 02:44

As the Java application does not have a UI, we will use a software development tool called Postman, which is free to download and install. With Postman you can import the Postman collection and start interacting with the application.

Continuous Integration (CI) pipeline overview Preview 00:46

We are ready to start building the pipeline for this project and the first step is the CI pipeline.

If you remember, the CI pipeline typically has a few stages: build, code quality, test, packaging the application for later use.

The purpose of the CI pipeline is to ensure that the artifact that we are building corresponds to our quality criteria and is releasable.

Build stage: Building a Java application locally Preview 03:19

Let's start building the CI pipeline with the build stage. Even if you are not a Java developer or have no relation to Java, most programs go through a build stage.

The build process will take the source code and transform it into something that can be executed on a computer. We call this process compilation. In this case, the build process will translate source code into Java bytecode that can be executed on the Java Virtual Machine (JVM). The output is a jar file (which is an archive) that contains this code.

To run the build process locally, I will use a tool called Gradle, which is just a build tool.

Build stage: Building a Java application with Gitlab CI Preview 03:02

The next step is to build the Java project, but this time using GitLab. We will do the following:

- define new pipeline file: .gitlab-ci.yml and add it to Git

- add build stage & job

- publish artifact

Test stage: Adding a smoke test Preview 06:27

In this lecture, we quickly discuss what is a smoke test, what a possible smoke test could be for our application and will add a new test stage to the GitLab CI pipeline.

CI/CD pipeline recap Preview 02:10

Now we have an artifact (or a package of software) and we are ready for deployment. There are two opposite directions in which we can go: deploy on your own infrastructure (aka server that we control and manage also called an in-house server) or deploy using a cloud provider, like Amazon AWS, Google Cloud, Microsoft Azure, and many others.

The advantage of using a cloud provider is that you only rent the infrastructure for the time you are using it. Using a cloud provider, you can focus on actually building and maintaining the application and forget about the hardware and scalability issues.

Which option makes more sense, it is up to you. For some of the reasons mentioned above, cloud services have risen greatly in popularity in the last years.

The following lectures will show to use Amazon AWS to deploy a Java application.

Brief introduction to Amazon Web Services (AWS) Preview 03:49

Amazon Web Services or simply AWS is a cloud platform offering over 170 services available in data centers located all over the world. Such services include virtual servers, managed databases, file storage, content delivery, and many others.

While this section focused on AWS, the principles presented here largely apply to any other providers.

This lecture discusses:

- how to create a new account

- how to to setup billing for AWS

Notice about unexpected costs while using AWS Preview 00:58

Serverless computing with AWS Elastic Beanstalk Preview 01:28

This lecture contains a short Introduction to the serverless architecture and AWS Elastic Beanstalk. Even if you use a cloud provider like AWS, you can still rent a virtual machine that has a dedicated CPU, memory and disk.

If you use a virtual server, this means that you still need to handle software updates, back-ups, monitoring, and any other aspects that ensure your application is running as expected.

AWS Elastic Beanstalk is a way to deploy an application but let AWS handle the hardware and software needed to run it. It is probably one of the easiest ways to deploy an application in the cloud.

How to deploy to AWS (manual upload) Preview 04:01

How to deploy to AWS from GitLab CI Preview 01:56

The goal is to automate the deployment process and to eliminate any manual work. Fortunately, AWS offers a tool called AWS CLI which allows us to deploy to AWS directly from GitLab CI.

Getting started with AWS S3 Preview 02:11

AWS S3 is the main storage service for the entire AWS platform. In the following lecture we will use S3 to upload the artifact (the Java jar file).

GitLab Group settings Preview 01:02

The GitLab group functionality is a way to organize similar projects into groups. Having a group allows you to configure environment variables that are available in multiple projects.

How to upload a file to AWS S3 from GitLab CI Preview 07:52

Using the S3 copy command it is possible to upload a file (S3 object) from GitLab CI to AWS S3.

How to deploy a Java application to AWS Elastic Beanstalk using the AWS CLI Preview 07:43

Deploying to AWS EB involves creating a new application version referencing the artifact from S3 and updating the production environment with the latest application version.

Assignment Preview 00:22

Assignment.

Assignment solution Preview 03:18

Assignment solution.

Create an application version Preview 03:56

The Java application allows us to insert the application version in one of the info endpoints. This allows us to check which version is currently deployed, without opening the AWS management console.

Verify the application version after deployment Preview 05:32

We need to ensure that the right application version was deployed. The way to approach this is to look at the info endpoint which will tell us the current application version. Having this check is mandatory to avoid any confusion regarding what was deployed.

Revisiting the CI pipeline Preview 00:51

Ensuring coding standards with tools for codestyle checking with PMD Preview 04:11

Most projects want to have a consistent code style and to follow some conventions and best practices. Often automated tools are used to assist with this process.

These tools typically do static code analysis, as the inspection performed without actually running the code. This approach is in contrast with a dynamic code analysis, which will actually run the code in order to perform the inspection.

One simple one that can be used for Java projects is PMD. PMD can help find unused variables, problematic code blocks and overall to enforce generally accepted best practices.

PMD already has a large set of predefined rules but will also allow you to configure or add new rules, as needed.

Assignment - Add code quality stage with PMD Preview 00:25

In this assignment, you are required to add a new job that runs the PMD tool. This is supposed to detect any issues with the code.

Assignment solution - Add code quality stage with PMD Preview 03:10

This is the solution that shows how to add a new GitLab CI job that runs the PMD tool and generates a report.

Quick introduction to unit testing in CI pipelines Preview 04:32

In this lecture, we quickly go over that a unit test is. In a nutshell, unit tests are responsible for testing only single units of code, typically one class. The execution time is very fast and gives instant feedback.

For Java projects, JUnit is the most popular framework for writing unit tests.

Unit test stage: Run JUnit tests with GitLab CI Preview 03:56

In this video, we will take a look at how to run unit tests (JUnit) in GitLab CI, how to generate JUnit reports, how to publish and expect them after the execution.

How to structure a CI/CD pipeline in GitLab CI? Preview 02:01

When structuring a pipeline, you need to take into account multiple factors. In this lecture, we will quick explore some possibilities.

API test stage: Run Postman API tests in GitLab CI Preview 10:47

For this application, API tests are the highest level of tests we can execute. We will use Postman to write and execute some simple API tests.

GitLab Pages (for publishing HTML reports or dashboards) Preview 03:45

GitLab Pages is a great addition to any CI/CD pipeline. GitLab Pages allows you to publish HTML websites directly from a repository. With some HTML and CSS skills, this great option for creating dashboards. This also allows you to publish HTML reports.

Ask the instructor Preview 00:10

Not the end Preview 00:23

Bonus lecture Preview 00:25