This guide will walk you through the steps to set up your Raspberry Pi as a CUPS (Common Unix Printing System) server, allowing seamless printing from your laptops, desktops, and even smartphones. Let's dive in!

What You'll Need:

• Raspberry Pi (any model will work, but a stable internet connection is key)

• Epson L130 inkjet printer

• USB cable to connect the printer to the Raspberry Pi

• MicroSD card with Raspberry Pi OS installed

• Wi-Fi connection for your Raspberry Pi

• A computer on the same network to configure the printer

Step-by-Step Guide:

1. Update Your System: First things first, let's ensure your Raspberry Pi's software packages are up to date. Open a terminal on your Raspberry Pi and run the following command:

    sudo apt update
   

   This command refreshes the package lists, ensuring you have access to the latest versions.

2. Install CUPS and Printer Drivers: Now, we'll install CUPS, the printing system we'll be using, and the Gutenprint drivers, which provide support for a wide range of printers, including the Epson L130. Execute this command:

    sudo apt install cups printer-driver-gutenprint
   

   Confirm the installation when prompted by pressing Y and hitting Enter.

3. Add Your User to the Printing Group: To manage the printer effectively, you'll need to add your user account (in this case, "minhaz") to the lpadmin group. This grants you administrative privileges for the printing system. Run the following command, replacing "minhaz" with your actual username if it's different:

    sudo usermod -a -G lpadmin minhaz
   

   You might need to log out and log back in for this change to take effect.

4. Allow Remote Access to CUPS: By default, CUPS only allows access from the local machine. To enable printing from other devices on your network, you need to allow remote access. Use the following command:

    sudo cupsctl --remote-any
   

   This command configures CUPS to accept print jobs from any IP address on your network. Be mindful of your network security when enabling this.

5. Restart the CUPS Service: To apply the changes you've made, restart the CUPS service:

    sudo systemctl restart cups
   

   This ensures that CUPS reloads its configuration with the new settings.

Configuring the Printer through the Web Interface:

Now that your Raspberry Pi is running the CUPS server, you can configure the Epson L130 through a web browser on another computer on the same network.

1. Find Your Raspberry Pi's IP Address: Open a terminal on your Raspberry Pi and run:

    hostname -I
   

   This will display the IP address of your Raspberry Pi.

2. Access the CUPS Web Interface: Open a web browser on your computer and enter the IP address of your Raspberry Pi followed by port 631. For example, if your Raspberry Pi's IP address is 192.168.1.100, you would enter http://192.168.1.100:631 in your browser's address bar.

3. Add Your Printer:

   • You might see a security warning; proceed to the website.

   • Click on the "Administration" tab.

   • Under the "Printers" section, click "Add Printer".

   • You might be prompted for a username and password. Use the username you used to log in to your Raspberry Pi and its corresponding password.

   • CUPS will search for connected printers. You should see your Epson L130 listed. Select it and click "Continue".

   • On the next screen, you can give your printer a descriptive name (e.g., "Epson Network Printer"), add a location, and description. Click "Continue".

   • In the "Make" list, select "Epson". Click "Continue".

   • In the "Model" list, look for "Epson Stylus Series" or something similar. In the "Driver" options, try selecting the driver that best matches your L130 model. You might need to experiment with different drivers if the first one doesn't work perfectly. The "Gutenprint v5.3.4" drivers are often a good choice, so look for an Epson L-series driver within that list if available, or try a generic Epson Stylus driver. Click "Add Printer".

   • You might be asked to set default options for the printer. Configure them as needed and click "Set Default Options".

Printing from Other Devices:

That's it! Your Epson L130 is now a network printer. To print from other devices on your network:

• Windows: Go to "Settings" > "Devices" > "Printers & scanners" > "Add a printer or scanner". It should automatically detect your network printer. If not, you can manually add it using its IP address (the Raspberry Pi's IP address) and specifying it as an IPP printer.

• macOS: Go to "System Preferences" > "Printers & Scanners" > click the "+" button. Select "IP" and enter the Raspberry Pi's IP address in the "Address" field, using ipp as the protocol and printers/your_printer_name (the name you gave the printer in CUPS) as the "Queue".

• Linux: The process varies depending on your distribution, but generally involves adding a new printer and selecting the IPP protocol, then entering the Raspberry Pi's IP address and the CUPS printer queue name.

Enjoy Wireless Printing!

You've successfully transformed your Raspberry Pi into a network print server for your Epson L130. Now you can enjoy the convenience of wireless printing from all your devices without the hassle of tangled USB cables. This simple yet powerful setup can significantly improve your printing workflow and make sharing your printer a breeze. Happy printing!

• You have a large team with more than 5 people

• You do not use any Config/Secret Management Tool

• Lack of team collaboration

Anyway, that's not the point. All you need is to run the following query in the Logging service to find out which users made changes to which configmap or secret.

protoPayload.@type = "type.googleapis.com/google.cloud.audit.AuditLog" AND protoPayload.serviceName = "k8s.io"
resource.type="k8s_cluster"
protoPayload.authenticationInfo.principalEmail !~ "system" AND protoPayload.authenticationInfo.principalEmail !~ "gserviceaccount"
protoPayload.methodName="io.k8s.core.v1.configmaps.update" OR protoPayload.methodName="io.k8s.core.v1.secrets.update"

1. Login to GCP and navigate to Logging

2. Set a proper timeline from the date-time picker (last X hour or last Y days)

3. Open up the Query Editor and paste the following code snippet

protoPayload."@type"="type.googleapis.com/google.cloud.audit.AuditLog"
resource.type="service_account"
protoPayload.methodName="google.iam.admin.v1.DeleteServiceAccount"

Voila! Look for the principalEmail field in the output, which will show the name of the person (or bot) who made the change.

First, enable ADB on your Android TV unit.

To Activate Developer Options: Navigate to Settings > About. Tap Build number seven times to enable developer mode.

Once done, find your TV's IP address and connect to it using the adb command line tool from your personal PC or Mac.

adb connect 192.168.254.101
adb shell

Finally uninstall the following packages one by one.

pm uninstall --user 0 azan.android.av.sharp.co.jp
pm uninstall --user 0 com.amazon.amazonvideo.livingroom
pm uninstall --user 0 com.google.android.play.games
pm uninstall --user 0 com.mediatek.wwtv.mediaplayer
pm uninstall --user 0 com.mediatek.wwtv.tvcenter
pm uninstall --user 0 com.mstar.android.tv.disclaimercustomization
pm uninstall --user 0 fusion.android.tv.demo
pm uninstall --user 0 jp.co.sharp.av.android.emanual
pm uninstall --user 0 jp.co.sharp.av.android.epopdemo
pm uninstall --user 0 rcota.android.av.sharp.co.jp.rcota

For best performance, disable animation scales to 0.

To Change Animation Scales: In Settings, go to System > Developer options. Under the Drawing or Animation section, locate:

• Window animation scale

• Transition animation scale

• Animator duration scale

Set all of them to 0 or 0.5x for a blazing fast performance.

Prerequisites

You will need the following items.

• Python3 with selenium module

• Chrome Browser and Chrome Driver [Download]

Script

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
import json
import sys
import time

chrome_options = Options()
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--headless")

driver = webdriver.Chrome('./chromedriver', options=chrome_options)

# load webpage
driver.get("https://example.com")

# the id of the button to be clicked
submit = driver.find_element_by_id("btn-id")
submit.click()
time.sleep(3)

# button click action
confirm = driver.find_element_by_id("confirm-button")
confirm.click()

print("Done")

Create Secret

First, you need to create a permanent Access Token from your GitLab repository or group that will have access to all the child repositories and the container registry. This token will be used by kubelet to pull images from the GitLab-managed container registry. The secret creation will look like the following.

kubectl create secret docker-registry gitlab-token-auth \
   --docker-server=https://registry.gitlab.com \
   --docker-username=kubelet \
   --docker-password=1234zxcv0987

CI/CD Pipeline

If you trigger a GitLab workflow inside GitLab-hosted runners, the workflow will have the privilege to push container images into the same code repository. Built-in variables like CI_REGISTRY_USER, CI_REGISTRY_PASSWORD, and CI_REGISTRY_IMAGE will be automatically populated during the pipeline run. Here is the code snippet that needs to be added to push the newly built image into Gitlab's container registry.

push:
  image: docker:24
  services:
    - docker:24-dind
  stage: push
  script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker build . -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA

Deployment

Once the image is pushed into Gitlab registry, you can use the previosly created Secret to pull the image into Kubernetes and run it. You will need to patch your deployment to include the imagePullSecrets.

spec:
  metadata:
    spec:
      containers:
      - name: app
        image: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
      imagePullSecrets:
      - name: gitlab-token-auth

Here is the CloudWatch query that filters logs with level = error and aggregates them by the count of occurrences.

fields @timestamp, @message
| filter level = "error"
| stats count(*) by @log

If you want an automated alert every time level = error appears, you can turn it into a CloudWatch Metric Alarm. Use the following command to create such an alarm.

aws cloudwatch put-metric-alarm --cli-input-json file://alarm.json

And here is the alarm.json file that contains all the required information.

{
    "logGroupName": "prod-backend/docker/api",
    "filterName": "api-error",
    "filterPattern": "{ $.level = \"error\" }",
    "metricTransformations": [
        {
            "metricName": "api-error",
            "metricNamespace": "api",
            "metricValue": "1",
            "unit": "count"
        }
    ]
}

Finally, you can connect the api-error metric to an SNS topic to get notified every time an error occurs in the log group.

ConfigMap

First, you need to create a ConfigMap that will be used as the core GitLab file /etc/gitlab/gitlab.rb. I will disable many internal tools and modules to make the core GitLab UI function as a Git server.

Please note that, you will also require a persistent PostgreSQL database connection parameters.

apiVersion: v1
kind: ConfigMap
metadata:
  name: gitlab-config
data:
  EXTERNAL_URL: https://gitlab.example.com
  GITLAB_OMNIBUS_CONFIG: |
    # disable heavy features
    postgresql['enable'] = false
    registry['enable'] = false
    redis['enable'] = false
    nginx['enable'] = false
    logrotate['enable'] = false
    gitlab_kas['enable'] = false
    monitoring_role['enable'] = false
    prometheus['enable'] = false
    alertmanager['enable'] = false
    node_exporter['enable'] = false
    redis_exporter['enable'] = false
    postgres_exporter['enable'] = false
    gitlab_exporter['enable'] = false
    prometheus_monitoring['enable'] = false
    grafana['enable'] = false

    # web    
    gitlab_workhorse['listen_network'] = "tcp"
    gitlab_workhorse['listen_addr'] = "0.0.0.0:8181"

    # timezone
    gitlab_rails['time_zone'] = 'Asia/Dhaka'

    # database
    gitlab_rails['db_database'] = 'db_gitlab'
    gitlab_rails['db_username'] = 'user_gitlab'
    gitlab_rails['db_password'] = '0987abcd1234wxyz'
    gitlab_rails['db_host'] = '10.10.10.1'
    gitlab_rails['db_port'] = 5432

Deployment

Now we will create a deployment that will use the ConfigMap mentioned above. As you can see, I have added three extra volumes to persistently store the etc, log, and data directories. This is very important; otherwise, you will lose all your data every time you restart the pod. You are welcome to use any other volume type like hostPath, glusterfs, nfs, etc.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: gitlab
  name: gitlab
spec:
  selector:
    matchLabels:
      app: gitlab
  template:
    metadata:
      labels:
        app: gitlab
    spec:
      containers:
      - envFrom:
        - configMapRef:
            name: gitlab-config
        image: gitlab/gitlab-ce
        name: app
        ports:
        - containerPort: 8181
          name: http
        volumeMounts:
        - mountPath: /etc/gitlab
          name: etc
        - mountPath: /var/opt/gitlab
          name: data
        - mountPath: /var/log/gitlab
          name: log
      volumes:
      - name: etc
        hostPath:
          path: /etc
      - name: data
        hostPath:
          path: /data
      - name: log
        hostPath:
          path: /var/log

Service

Once the pods are up and running, we need to expose the Deployment as a Service. You will notice that we have exposed port 8181 of the containers to forward web traffic to Gitlab workhorse.

apiVersion: v1
kind: Service
metadata:
  labels:
    app: gitlab
  name: gitlab
spec:
  ports:
    - name: http
      port: 8181
      targetPort: 8181
  selector:
    app: gitlab

Ingress

Finally, you need to bind a domain with the ingress.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: gitlab
spec:
  rules:
    - host: gitlab.example.com
      http:
        paths:
          - path: /
            backend:
              serviceName: gitlab
              servicePort: http
  tls:
    - hosts:
        - gitlab.example.com
      secretName: example-com-ssl

Here's an example snippet that downloads a file from a secure SFTP server using custom authentication.

SFTP_HOST=sftp://172.18.1.100
SFTP_AUTH=user:12345678
SFTP_FILE=/data/files/$(date --date="1 day ago" +%Y%m%d)

curl -v -u $SFTP_AUTH $SFTP_HOST/$SFTP_FILE -o $SFTP_FILE 2>&1

MySQL

docker run --rm -d \
    -p 3306:3306 \
    -e MYSQL_ROOT_PASSWORD=12345678 \
    mariadb

PostgreSQL

docker run --rm -d \
    -p 5432:5432 \
    -e POSTGRES_PASSWORD=12345678 \
    -e POSTGRES_USER=postgres \
    -e POSTGRES_DB=postgres \
    postgres

MongoDB

docker run --rm -d \
    -p 27017:27017 \
    -e MONGO_INITDB_ROOT_USERNAME=root \
    -e MONGO_INITDB_ROOT_PASSWORD=12345678 \
    mongodb/mongodb-community-server:6

Redis

docker run --rm -d \
    -p 6379:6379 \
    -e ALLOW_EMPTY_PASSWORD=no \
    -e REDIS_PASSWORD=12345678 \
    bitnami/redis

etcd

docker run --rm -d \
    -p 2379:2379 \
    -e ALLOW_NONE_AUTHENTICATION=yes \
    bitnami/etcd

RabbitMQ

docker run --rm -d \
    -p 5672:5672 \
    -p 15672:15672 \
    -e RABBITMQ_DEFAULT_USER=admin \
    -e RABBITMQ_DEFAULT_PASS=12345678 \
    rabbitmq:3-management

Well, here is the trick. You can do the following.

1. Create a Kubernetes-native CronJob using any Docker image that includes kubectl

2. Run the pod with a service account that has the Role/ClusterRole to perform the desired actions

First, choose a namespace where the CronJob will be created and executed. Then, create a service account in that namespace.

kubectl create ns my-namespace
kubectl create sa my-cronjob -n my-namespace

Next, associate some RBAC rules with the service account. You can use system roles (like cluster-admin or view) but try not to allow excessive permission.

kubectl create clusterrolebinding --clusterrole edit \
    --serviceaccount=my-namespace:my-cronjob

Finally, create a CronJob definition that utilizes the previously mentioned ServiceAccount. Make sure that the spec.schedule is accurate in the UTC timezone. Additionally, you should include an exit 0 at the end of the command; otherwise, Kubernetes will regard the job as failed.

Here's an example:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: my-cronjob
  namespace: my-namespace
spec:
  schedule: "0 0 * * SUN"
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: Never
          containers:
          - command:
            - /bin/bash
            - -c
            - |
              kubectl -n prod rollout restart deploy
              kubectl -n stage rollout restart deploy
              kubectl -n dev rollout restart deploy
              exit 0
            image: bitnami/kubectl
            imagePullPolicy: IfNotPresent
            name: job
          serviceAccount: my-cronjob

First, you need to create a bucket in the same region as the CloudWatch Log Group.

aws s3api create-bucket --bucket app-logs --create-bucket-configuration LocationConstraint=us-west-2

Next, you must modify the bucket policy to ensure the CloudWatch Log Exporter can write to it. Here is the policy document:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "s3:GetBucketAcl",
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::app-logs",
      "Principal": {
        "Service": "logs.us-west-2.amazonaws.com"
      }
    },
    {
      "Action": "s3:PutObject",
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::app-logs/*",
      "Principal": {
        "Service": "logs.us-west-2.amazonaws.com"
      }
    }
  ]
}

Use the following command to apply the policy to the bucket.

aws s3api put-bucket-policy --bucket app-logs --policy file://policy.json

Next, initiate an export job that will transfer all log streams from a specific log group into the previously created S3 bucket. You also need to specify the range in Unix timestamp format.

aws logs create-export-task --task-name "app-logs-group-1" \
    --log-group-name "prod/app-logs" \
    --from 1704045600000 --to 1704132000 \
    --destination "app-logs" --destination-prefix "prefix1"

The command above will produce a task ID. You can query the task ID to check whether the export job has been completed.

aws logs describe-export-tasks --task-id d6f1d52c-2783-4145-9668-4f5cc5579f41

Once complete, you can simply download the bucket content to your local machine and analyze it.

aws s3 sync s3://app-logs ./logs

The Ingress objects are namespace-scoped. What happens if your applications are deployed in multiple namespaces? By default, the AWS Load Balancer Controller will create a separate ALB for each ingress object. Even if there are multiple Ingress objects in the same namespace, you will end up having an individual ALB for each of them. And that's perfectly fine.

However, if you have 50 or 100 namespaces, and your application needs to be deployed in multiple namespaces, you will incur higher costs because each ALB will charge you for the number of hours they remain active, even if you do not use them.

There is a simple trick to consolidate all Ingress objects into a single ALB, where all those routes will be combined into ALB rule groups. All you need to do is annotate each Ingress with the following labels.

alb.ingress.kubernetes.io/group.name: myapp

For all matching group.name labels in all namespaces, the AWS Load Balancer Controller will create a single ALB with multiple rules inside it. Here is a detailed example:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/backend-protocol: HTTP
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-west-1:987654321:certificate/37c33c94-e3be-41cc-9ec8-fd16b2873bd4
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
    alb.ingress.kubernetes.io/load-balancer-name: myapp
    alb.ingress.kubernetes.io/group.name: myapp
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/ssl-redirect: "443"
    alb.ingress.kubernetes.io/success-codes: "200"
    alb.ingress.kubernetes.io/target-type: instance
    kubernetes.io/ingress.class: alb
  name: myapp-dev
  namespace: myapp-dev
spec:
  rules:
  - host: myapp-dev.example.com
    http:
      paths:
      - backend:
          service:
            name: app
            port:
              number: 80
        path: /
        pathType: Prefix
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/backend-protocol: HTTP
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-west-1:987654321:certificate/37c33c94-e3be-41cc-9ec8-fd16b2873bd4
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
    alb.ingress.kubernetes.io/load-balancer-name: myapp
    alb.ingress.kubernetes.io/group.name: myapp
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/ssl-redirect: "443"
    alb.ingress.kubernetes.io/success-codes: "200"
    alb.ingress.kubernetes.io/target-type: instance
    kubernetes.io/ingress.class: alb
  name: myapp-stage
  namespace: myapp-prstageod
spec:
  rules:
  - host: myapp-stage.example.com
    http:
      paths:
      - backend:
          service:
            name: app
            port:
              number: 80
        path: /
        pathType: Prefix
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/backend-protocol: HTTP
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-west-1:987654321:certificate/37c33c94-e3be-41cc-9ec8-fd16b2873bd4
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
    alb.ingress.kubernetes.io/load-balancer-name: myapp
    alb.ingress.kubernetes.io/group.name: myapp
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/ssl-redirect: "443"
    alb.ingress.kubernetes.io/success-codes: "200"
    alb.ingress.kubernetes.io/target-type: instance
    kubernetes.io/ingress.class: alb
  name: myapp-prod
  namespace: myapp-prod
spec:
  rules:
  - host: myapp.example.com
    http:
      paths:
      - backend:
          service:
            name: app
            port:
              number: 80
        path: /
        pathType: Prefix

You can find all available annotations here: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.6/guide/ingress/annotations/

However, there are certain limitations of Traefik on cloud platforms. It can handle the TLS/SSL layer only through Kubernetes Secret or Let's Encrypt, meaning it cannot leverage AWS ACM for resolving SSL certificates. This limitation stems from AWS ACM itself, as it can only be used with resources like ALB and CloudFront.

However, you may want to have both 1) SSL termination by ACM and 2) support for Traefik-specific features (Middleware, Circuit Breaker, etc.) all in one place. Fortunately, I have tested a system that allows you to continue using an AWS ACM-provided SSL certificate while handling ingress routing with Traefik. Here is the high-level architecture of the proposed system:

To set up this system, you'll first need an EKS cluster with the AWS Load Balancer Controller add-on. You can bootstrap the cluster using eksctl.

Once the cluster is ready, deploy Traefik in web mode only, disabling the websecure entrypoint.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: traefik
spec:
  selector:
    matchLabels:
      app: traefik
  template: 
    metadata:
      labels:
        app: traefik
    spec:
      serviceAccountName: traefik
      containers:
      - image: docker.io/traefik:v3.0
        name: traefik
        ports:
        - name: "web"
          containerPort: 80
        args:
          - "--entrypoints.web.address=:80/tcp"
          - "--api.dashboard=false"
          - "--providers.kubernetescrd"

Next, create a service for the Traefik deployment/daemonset and set its type to NodePort (ALB requires the service to be of NodePort type for its backend).

apiVersion: v1
kind: Service
metadata:
  name: traefik
spec:
  ports:
  - name: web
    nodePort: 30080
    port: 80
    protocol: TCP
    targetPort: web
  selector:
    app: traefik
  type: NodePort

Next, create an ALB Ingress with the appropriate annotations for ACM UUID, Name, and Class.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/backend-protocol: HTTP
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:ap-southeast-1:0000000000:certificate/cb6cf41e-a6f4-4fd3-9aa5-44503f317420
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
    alb.ingress.kubernetes.io/load-balancer-name: my-lb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/ssl-redirect: "443"
    alb.ingress.kubernetes.io/success-codes: 200-404
    alb.ingress.kubernetes.io/target-type: instance
  name: traefik
spec:
  ingressClassName: alb
  rules:
  - http:
      paths:
      - backend:
          service:
            name: traefik
            port:
              name: web
        path: /*
        pathType: ImplementationSpecific

Once applied, an ALB will be created. You can then route any HTTPS traffic through the ALB, and ultimately, it will be routed using Traefik within your Kubernetes Cluster. A sample IngressRoute object would look like this:

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: my-apps
spec:
  entryPoints:
  - web
  routes:
  - kind: Rule
    match: Host(`a.example.com`)
    services:
    - name: app-a
      port: 80
  - kind: Rule
    match: Host(`b.example.com`)
    services:
    - name: app-b
      port: 80

helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update

Redis Single Node

helm install redis bitnami/redis \
    --set architecture=standalone \
    --set auth.password=STRONG-PASSWORD-12345678 \
    --set fullnameOverride=redis \
    --set master.persistence.enabled=false \
    --set master.disableCommands=false

Redis Sentinel

helm install redis bitnami/redis \
    --set auth.enabled=true \
    --set auth.sentinel=true \
    --set auth.password=STRONG-PASSWORD-12345678 \
    --set fullnameOverride=redis \
    --set master.persistence.enabled=false \
    --set master.disableCommands=false \
    --set sentinel.enabled=true \
    --set sentinel.masterSet=mymaster \
    --set replica.replicaCount=3 \
    --set replica.persistence.enabled=false

Redis Cluster

helm install redis bitnami/redis \
    --set architecture=replication \
    --set fullnameOverride=redis \
    --set auth.password=STRONG-PASSWORD-12345678 \
    --set master.count=3 \
    --set replica.replicaCount=3 \
    --set master.persistence.enabled=false \
    --set replica.persistence.enabled=false \
    --set serviceAccount.create=true \
    --set podSecurityPolicy.create=false \
    --set master.podSecurityContext.enabled=false \
    --set replica.podSecurityContext.enabled=false \
    --set master.podAntiAffinityPreset=hard

ElasticSearch Single Node

helm install elasticsearch bitnami/elasticsearch \
    --set master.replicas=1 \
    --set fullnameOverride=elasticsearch \
    --set master.persistence.enabled=false

ElasticSearch Cluster

helm install elasticsearch bitnami/elasticsearch \
    --set master.replicas=3 \
    --set data.replicaCount=3 \
    --set coordinating.replicaCount=3 \
    --set fullnameOverride=elasticsearch \
    --set master.persistence.enabled=false \
    --set data.persistence.enabled=false \
    --set coordinating.persistence.enabled=false

Kafka and Zookeeper

helm install kafka bitnami/kafka \
    --set fullnameOverride=kafka \
    --set podSecurityContext.enabled=false \
    --set containerSecurityContext.enabled=false \
    --set serviceAccount.create=false \
    --set autoDiscovery.enabled=false \
    --set persistence.enabled=false \
    --set zookeeper.persistence.enabled=false

RabbitMQ Single Node

helm install rabbitmq bitnami/rabbitmq \
    --set auth.username=rabbitmq \
    --set auth.password=STRONG-PASSWORD-12345678 \
    --set clustering.enabled=false \
    --set podSecurityContext.enabled=false \
    --set serviceAccount.create=false \
    --set rbac.create=false \
    --set persistence.selector=volume=rabbitmq

RabbitMQ Cluster

helm install rabbitmq-rdms bitnami/rabbitmq \
    --set fullnameOverride=rabbitmq \
    --set auth.username=rdms-rabbitmq-user \
    --set auth.password=STRONG-PASSWORD-12345678 \
    --set clustering.enabled=true \
    --set replicaCount=3 \
    --set serviceAccount.create=true \
    --set podSecurityContext.enabled=false \
    --set containerSecurityContext.enabled=false \
    --set extraPlugins='rabbitmq_shovel rabbitmq_shovel_management'

Traefik DaemonSet

helm repo add traefik https://traefik.github.io/charts
helm install traefik traefik/traefik \
    --set deployment.kind=DaemonSet \
    --set env[0].name=TZ \
    --set env[0].value=Asia/Dhaka \
    --set ingressRoute.dashboard.enabled=false \
    --set ports.web.port=80 \
    --set ports.websecure.port=443 \
    --set service.type=LoadBalancer

Even it is possible to use SSL certificates generated by Let's Encrypt (privkey.pem and fullchain.pem from /etc/letsencrypt/live direcotory) by creating a TLS secret from these files.

But what if you want to generate certificates using Traefik itself? Luckily Traefik has full support for all Let's Encrypt challenges (http, https and dns based verification). Let's jump in.

First, ensure that Traefik has connectivity to the internet. Without internet connectivity, Traefik will not be able to send certificate renewal requests to ACME servers.

Second, if you have multiple replicas of Traefik running in the cluster (DaemonSet or Deployment) then you can reduce the replicas to 1. This will lower the chance of failed verification attempts and being restricted to issue certificates for a week.

Third, Traefik stores the keys and certificates in a JSON file named acme.json. This must persist when Traefik restarts. Otherwise, Traefik will try to re-issue the certificate and your domain could be blacklisted for a while.

To enable ACME based resolver, add the following parameters email, storage and challenge as Traefik command line parameters. You are free to choose any name. I used letsencrypt in this case.

...
- args:
  - --log.level=DEBUG
  - --entrypoints.web.address=:80
  - --entrypoints.websecure.address=:443
  - --certificatesresolvers.letsencrypt.acme.email=me@example.com
  - --certificatesresolvers.letsencrypt.acme.storage=/etc/traefik/acme/acme.json
  - --certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=web
  - --serversTransport.insecureSkipVerify=true
  image: traefik:v2.9
  name: traefik
...

Then let's mount the path /etc/traefik/acme as a persistence volume. You can use hostPath, nfs, glusterfs or anything compatible with Kubernetes. But make sure that the mount path is shared across all Traefik pods running in the cluster.

  volumeMounts:
  - name: acme
    mountPath: /etc/traefik/acme
volumes:
- name: acme
  nfs:
    path: /data/traefik-system/acme
    server: 192.168.100.200

Once Traefik pods are in Running state, let's create an IngressRoute object that uses letsencrypt resolver as SSL store. Here is one example.

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: my-app
spec:
  entryPoints:
  - websecure
  routes:
  - kind: Rule
    match: Host(`my-app.com`)
    services:
    - name: my-app
      port: 80
  tls:
    certResolver: letsencrypt

Once this resource is applied, Traefik will try to issue a certificate using ACME API and disable all other ingress hosts for a few seconds. Upon a successful issue, the acme.json file will be generated.

Please note that wildcard certificates can only be generated using DNS-based verifications. So you have to use the sans sections in the tls block of IngressRoute carefully.

I hope the post helps you all.

These package managers often download files over an unencrypted HTTP connection and verify the binaries using a checksum or GPG signature. However, in some cases, it is possible to bypass the filtering by enforcing HTTPS so these tools use an encrypted connection to fetch and download artifacts.

Composer

composer config --global disable-tls
composer config --global secure-http

NPM

npm config set registry https://registry.npmjs.org/
npm config set ssl-strict=false

Apt

sed -i s/http/https/g /etc/apt/sources.list

AWS SecretManager requires the secrets to be in decoded format. However, Kubernetes Secrets are encoded in base64 and require conversion. We can use jq to do this for us.

Once the secrets are decoded, we can pass this key-value pair to AWS CLI to create a Secret Manager object.

kubectl get secret app-config -o jsonpath='{.data}' | jq -r 'reduce to_entries[] as {$key, $value} (null; .[$key] = ($value|@base64d))' > secret.json

aws secretsmanager create-secret --name app-config --secret-string file://secret.json

AWS Secret Manager to Kubernetes Secret

AWS CLI can fetch decoded secrets from Secret Manager. However, kubectl requires the secrets to be in env format if there are plenty of them. Once again, we can use jq to map them in env like and later this env file can be used to create the Kubernetes Secret.

aws secretsmanager get-secret-value --secret-id app-config | jq -r '.SecretString | fromjson | to_entries[] | "(.key)=(.value)"' > secret.env

kubectl create secret generic app-config --from-env-file=secret.env

Secrets from Vault can be imported into the Kubernetes cluster using csi-secrets-store and hashicorp/vault provider. But the CRD configuration seems a bit complex for noobs like me. So I ended up using ESO which seems easiest.

First, you need to install ESO with Helm inside the cluster. I am using eso-system as the namespace but feel free to choose another.

helm repo add external-secrets https://charts.external-secrets.io
helm install external-secrets external-secrets/external-secrets -n eso-system --create-namespace

Once the release is installed, 3 pods should appear and wait until they all are in both Running and Healthy state.

NAME                                                READY   STATUS
external-secrets-cert-controller-655d7bb477-kp2ck   1/1     Running
external-secrets-f9bc79d45-xk9lm                    1/1     Running
external-secrets-webhook-5b4779cbf8-mspsl           1/1     Running

To sync Secrets from Vault, we need to authorize the ESO service account to make valid calls to Vault APIs. I used a custom Token with a policy that allows only read access to the Vault KV engine. We need to pass the Token to ESO using Kubernetes secret.

kubectl create secret generic vault-token -n eso-system --from-literal=token=hvs.9lownuRz5bZO221dgkKXG5hQB

Once the Vault Token is ready, we can create a Vault Secret Store and tell ESO to establish a connection with it.

However, you can create multiple SecretStore in multiple namespaces. In that case, each ExternalSecret object will look for SecretStore in the same namespace.

In my case, I used one ClusterSecretStore and referenced multiple ExternalSecret in multiple namespaces to a single Store, to reduce the complexity.

Let's create our first ClusterSecretStore. Apply the following using kubectl apply -f

apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
  name: vault-backend
spec:
  provider:
    vault:
      server: "https://vault.your.host:8200"
      path: "kv"
      version: "v1"
      auth:
        tokenSecretRef:
          name: "vault-token"
          namespace: eso-system
          key: "token"

Now check if the Secret Store is ready to be used. The field Ready=True is seen if the Vault Token is correct.

$ kubectl get clustersecretstores 
NAME            AGE   STATUS   READY
vault-backend   12h   Valid    True

When the Secret Store is ready, it's time to put some values in Vault. You can use Vault UI. For my case, I used the CLI to enable Vault v1 KV engine and add some values there.

$ vault kv put kv/webapp-prod SPRING_DATASOURCE_USER=john SPRING_DATASOURCE_PASSWORD=1234qwer
$ vault kv get kv/webapp-prod
=============== Data ===============
Key                           Value
---                           -----
SPRING_DATASOURCE_PASSWORD    1234qwer
SPRING_DATASOURCE_USER        john

Now let's create an ExternalSecret object inside the cluster which will tell SecretStore to sync the above-mentioned secrets from Vault.

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: webapp-secret
  namespace: webapp-prod
spec:
  refreshInterval: 30s
  secretStoreRef:
    name: vault-backend
    kind: ClusterSecretStore
  target:
    name: webapp-secret
    creationPolicy: Owner
  dataFrom:
  - extract:
      key: kv/webapp-prod

I have used the extract option which will fetch all KV pairs from Vault. Also, I used the Kubernetes namespace as the Vault KV Path to easily map multiple secrets into multiple namespaces.

Anyway, let's check if the ExternalSecret syncing is successful.

$ kubectl get externalsecret -n webapp-prod
NAME            STORE           REFRESH   STATUS         READY
webapp-secret   vault-backend   30s       SecretSynced   True

The state SecretSynced indicates that ESO has successfully fetched the secrets from Vault as per the instruction. Let's check if the Kubernetes native Secret object is created.

$ kubectl get secret webapp-secret -n webapp-prod -o yaml | head -4
apiVersion: v1
data:
  SPRING_DATASOURCE_HOST: MTI3LjAuMC4x
  SPRING_DATASOURCE_PASSWORD: MTIzNHF3ZXI=

If you base64 -d them, you will notice that the values are the same as they are in Vault. Feel free to add more KV pairs in the same path and the new values should appear in Kubernetes Secret as well,

export ETCDCTL_API=3

timestamp=`date +%Y%m%d-%H%M%S`

etcdctl --endpoints 127.0.0.1:2379 \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  snapshot save /data/etcd-backup/snapshot-$timestamp.db

However, sometimes you may find the backup size has exceeded several hundred megabytes, which is substantial for a simple cluster containing 10-20 active namespaces and hundreds of pods. This occurs due to the internal fragmentation of etcd database files. The backup size (and disk space used in /var/lib/etcd) can be reduced by sending a fragmentation signal to all the etcd members. The following command should be sufficient to accomplish this:

export ETCDCTL_API=3

etcdctl --endpoints 127.0.0.1:2379 \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  defrag --cluster=true

Learn more about etcd defragmentation here.