← back

Create a custom CDN - Content Delivery Network

Using Terraform, Ansible, Azure, Nginx and geoDNS created a distributed server network for content delivery.

Although this architecture can be used for creating anything which need to be deployed as `per user location` basis.

For example

  • Content Delivery Network
  • Global HTTPS Cache using Varnish
  • Distributed Deployments
  • And so on...

In this very blog, we are just going to see CDN

CDN - Content Delivery Network

cdn example

CDN are server used for delivering content - images, videos, audios, texts and blog for naming few. These are located on every location where there is significant human population is present.

In the example image above, I have shown how can Company (or even we) place server at different location and create a logic to distribute request depending on client location to nearest server.

nearest server example

What logic ?

It can be geoDNS, geoDistance, Weighted Shuffle and Anycast.

Go and read about them, Here is TL;DR

  • geoDNS : Every server has unique IP, we distribute request on the basis os IP, ASN, Region, Continent, Country.

    See example image
    GeoDNS example
  • geoDistance : Every server has unique IP, we distribute request on the basis nearest server, by calculating shortest distance.

  • Weighted Shuffle : Random order with weights - Mainly for preventing DDoS

  • Anycast: All the server has same IP, request goes to nearest server using BGP Protocol.

    See example image
    Anycast example

Here we means the DNS Server.

Anycast is hard to achieve, since involve working with hardware and network layer and publishing same IP for all server. Weighed Shuffle is not want we want. So we are going to use geoDNS and geoDistance.

That was a brief about CDNs, now lets talk about creating one.

Creating a Custom CDN Network.

We are gonna use ...

If you don't want to do these steps exactly similar, then you can do it your way. The core idea same.

  1. Azure for creating Virtual Machines at 3 different locations

    To do this, we are using Terraform - An Infrastructure as Code platform

  2. Nginx for Static contents serving

    To do this we will use Ansible

  3. Gcore for DNS with Free geoDNS service.

    Other DNS service providers, like Cloudflare does not have geoDNS for Free - Its only for Enterprise users.

  4. Finally upload Contents to our Content Delivery Network

    To do this we also use Ansible

Creating Virtual Machines - Our Servers

The task is to create multiple server (virtual machine) at different locations, and get unique IP Address of all.

I'm going to create 7 servers at these locations:

  • "Central India"
  • "Central US"
  • "West Europe"

See how to do it with Terraform

The source code for this terraform infra setup will be found here

We created our Virtual Machine map

variable "vm_map" {
    type = map(object({
        name     = string
        location = string
        size     = string
    default = {
        "vm1" = {
            name = "centralindia"
            location = "Central India"
            size = "Standard_B1s"
        "vm2" = {
            name = "centralus"
            location = "Central US"
            size = "Standard_B1s"
        "vm3" = {
            name = "westeurope"
            location = "West Europe"
            size = "Standard_B1s"

And the main file, which created all other resources is

terraform {
    required_providers {
        azurerm = {
            source = "hashicorp/azurerm"
            version = "~> 3.0.2"
    required_version = ">= 1.8.5"
provider "azurerm" {
    features {
        resource_group {
            prevent_deletion_if_contains_resources = false
resource "azurerm_resource_group" "custom-cdn" {
    name = "custom-cdn-ResourceGroup"
    location = "Central India" # location has no effect
    # since resouce group is just a container for other resource
resource "azurerm_virtual_network" "custom-cdn" {
    for_each = var.vm_map
    name = "${each.value.name}-VNET"
    location = each.value.location
    address_space = [ "" ]
    resource_group_name = azurerm_resource_group.custom-cdn.name
resource "azurerm_subnet" "custom-cdn" {
    for_each = var.vm_map
    name = "${each.value.name}-Subnet"
    resource_group_name = azurerm_resource_group.custom-cdn.name
    virtual_network_name = azurerm_virtual_network.custom-cdn[each.key].name  
    address_prefixes = [ "" ]
resource "azurerm_public_ip" "custom-cdn" {
    for_each = var.vm_map
    name = "${ each.value.name }-PublicIp"
    location = each.value.location
    resource_group_name = azurerm_resource_group.custom-cdn.name
    allocation_method = "Static"
resource "azurerm_network_interface" "custom-cdn" {
    for_each = var.vm_map
    name = "${ each.value.name }-NIC"
    location = each.value.location
    resource_group_name = azurerm_resource_group.custom-cdn.name
    ip_configuration {
        name = "${each.value.name}-public"
        subnet_id = azurerm_subnet.custom-cdn[each.key].id
        public_ip_address_id = azurerm_public_ip.custom-cdn[each.key].id
        private_ip_address_allocation = "Dynamic"
resource "azurerm_virtual_machine" "custom-cdn" {
    for_each = var.vm_map
    name = "${ each.value.name }-VM"
    location = each.value.location
    resource_group_name = azurerm_resource_group.custom-cdn.name
    network_interface_ids = [ azurerm_network_interface.custom-cdn[each.key].id ]
    vm_size = each.value.size
    storage_image_reference {
        publisher = "Canonical"
        offer     = "0001-com-ubuntu-server-jammy"
        sku       = "22_04-lts-gen2"
        version   = "latest"
    storage_os_disk {
        name              = "${ each.value.name }-OsDisk"
        caching           = "ReadWrite"
        create_option     = "FromImage"
        managed_disk_type = "Standard_LRS"
    os_profile {
        computer_name  = each.value.name
        admin_username = "custom-cdn"
        admin_password = "Password1234!"
    os_profile_linux_config {
        disable_password_authentication = false
output "custom_cdn_public_ip" {
    value = {
        for vm in azurerm_public_ip.custom-cdn : vm.name => vm.ip_address

See the admin password is Password1234! - Change this to more secure one.

terraform apply

We will get all the IP Address.

# This will also print the output
terraform output

After creating the Virtual Machines, we will have all the IP address as.

custom_cdn_public_ip = {
  "centralindia-PublicIp" = ""
  "centralus-PublicIp" = ""
  "westeurope-PublicIp" = ""

Serving Static Contents using Nginx

The task is to login to each virtual machine and setup nginx and start serving static content - files and folders.

See how to do it with Ansible

The source code for this ansible is in infra folder here

After we got all the IP Address of all VMs. Create a inventory.ini file for storing all this data, which ansible take as input.

central-india location=central-india ansible_host=[HOST IP] ansible_user=custom-cdn ansible_ssh_pass=Password1234!
central-us location=central-us ansible_host=[HOST IP] ansible_user=custom-cdn ansible_ssh_pass=Password1234!
west-europe location=west-europe ansible_host=[HOST IP] ansible_user=custom-cdn ansible_ssh_pass=Password1234!

Replace [HOST IP] with server IP address.

To install and setup nginx, create a nginx.conf in the same directory.

server { 
    listen 80;
    server_name {{ domain_name }};
    add_header X-Server-Location {{ location }};
    autoindex on;
    root /home/custom-cdn/contents;

and now create an Ansible Playbook file.

- name: Install NGINX on all VMs
    hosts: azure_vms
    become: yes
        domain_name: cdn.kunals.me
        - name: Update apt cache
            update_cache: yes
        - name: Create a new contents directory
            path: /home/custom-cdn/contents
            state: directory
            mode: "0755"
        - name: Install NGINX
            name: nginx
            state: present
        - name: Create NGINX configuration for new domain
            src: nginx.conf
            dest: /etc/nginx/sites-available/{{ domain_name }}.conf
            - Restart NGINX
        - name: Enable new NGINX site
            src: /etc/nginx/sites-available/{{ domain_name }}.conf
            dest: /etc/nginx/sites-enabled/{{ domain_name }}.conf
            state: link
            - Restart NGINX
        - name: Restart NGINX
                name: nginx
                state: restarted

and finally in the same directory. Create Ansible configuration file, ansible.cfg

host_key_checking = false

After these files, we are ready to run the playbook.

make sure you have sshpass installed on your system. Ansible require it.

Run the playbook

ansible-playbook -i inventory.ini setup_nginx.yml

This will install nginx on all server, create nginx.conf file in right place and restart nginx.

See how to do it Manually


To Login we use SSH

ssh [email protected]
# custom-cdn is each machines username
# replace to actual server ip

Create a folder where all the contents reside.

mkdir contents
# pwd
# /home/custom-cdn/contents

Install nginx and make sure its running.

Open the IP address in the browser you must see nginx page

Edit the nginx.conf to add current user.

edit file /etc/nginx/nginx.conf

- user www-data;
+ user custom-cdn;

Restart nginx

sudo systemctl restart nginx

Lets write a configuration at /etc/nginx/sites-enabled/

Create a new file

sudo vim cdn.kunals.me
# choose any domain you want

And write this

server {
    listen 80;
    serve_name cdn.kunals.me;
    add_header X-Server-Location centalindia; # place your sever location name here
    autoindex on;
    root /home/custom-cdn/contents;

Reload nginx

sudo nginx -s reload

This will make nginx server static files and folder in contents directory of each machine.

Which means on hitting cdn.kunals.me the files and folders in contents folder will be served.

Setting DNS

Now lets point cdn.kunals.me to each IP address of our servers. You heard it right, one domain name will point to multiple IP address using request distribution logic.

Our DNS of choice is Gcore DNS, reason?. It has free geoDNS support. Go ahead and create your account on Gcore.

Open the Gcore's DNS page

Click on Add Zone

Add zone demo image

Add your domain nameserver to Gcore.

Add domain name

After your domain name is Active, lets create some A records.

Make sure you have Interface mode set to Advance

Interface mode

Click on Add a Record Set

new records set

Add an A with name cdn.kunals.me ( replace it with your domain). And click on Geo Distance to select geoDistance preset.

Fill each IP address in the records.

add a record

After filling all the 3 records, click on Create.

Now we should able to hit cdn.kunals.me and get the empty directory served by nginx.

Uploading content to our network.

Anything that is present in contents directory will be served. Our task is to put items in this directory across all servers.

See how to do it with Ansible

Create a new ansible playbook called scp.yml

- name: Transfer files using scp
    hosts: azure_vms
        file_path: null # Path to the file you want to transfer
        - name: SCP file to VM
            cmd: sshpass -p {{ ansible_ssh_pass }} scp {{ file_path }} {{ ansible_user }}@{{ ansible_host }}:/home/custom-cdn/contents
        delegate_to: localhost

when running this playbook specify the file_path variable you want to send.

ansible-playbook -i inventory.ini scp.yml -e "file_path=/home/kunal/hello.txt"  

This way we will store any file in contents folder of every server.

When doing it manually, for EVERY SERVER

We can use scp - Secure Copy. An SSH based file transfer utility.

scp new_file.txt [email protected]:/home/custom-cdn/contents
# Replace to actual server IP


We are able to store content to our content network, any we can access it at cdn.kunals.me

Bonus - Add a SSL Certificate to get HTTPS

See how to do it with Ansible

Create a new ansible playbook, called install_ssl_cert.yml

- name: Setup SSL Certificates
hosts: azure_vms
become: yes
    domain_name: cdn.kunals.me
    admin_email: [email protected]
    - name: Update apt cache
        update_cache: yes
    - name: Ensure Snapd is installed
        name: snapd
        state: present
    - name: Install certbot 
        cmd: snap install --classic certbot
    - name: Install Certificates
        cmd: certbot --nginx -m {{ admin_email }} -d {{ domain_name }} --agree-tos --non-interactive

And run this playbook as

ansible-playbook -i inventory.ini  install_ssl_cert.yml

When doing manually, FOR EACH SERVER

Install the certbot

Run this command to issue certificates.

sudo certbot --nginx


Thank for reading it. For any suggestions email me - kunal [at] kunalsin9h [dot] com