🎬 Movie Bets

• ✅ Concurrency and Resiliency Patterns in Saga Transactions for Spring Boot Microservices
• ✅ Part 5: Reliable Long-Running Saga Processes, using Kafka and JPA Transactions with Outbox Pattern
• ✅ Long-running Saga Processes, with dynamic recursive process steps, using Kafka and JPA Transactions with Outbox Pattern
• ✅ Event Collaboration and Concurrency Control in Saga Transactions, using Kafka Consumer Groups, Partitions adn Transactions
• ✅ Splitter/Aggregator Pattern for starting and aggregating Long-running Saga Process Jobs, using Kafka and JPA Transactions with Outbox Table

📖 Concurrency and Resiliency Patterns in Saga Transactions for Spring Boot Microservices

• ✅ Cloud-Native Spring Boot Development and Production Environment with Spring Boot, Kafka, Kubernetes, Skaffold and Terraform
• ✅ Reliable Asynchronous Long-Running Saga Processes with Event-Driven Kafka Transactions and Outbox Pattern
• ✅ At-least once processing guarantee and Idempotency Key for Event Deduplication in Saga Transactions
• ✅ Event-Driven Choreography Saga with Spring Kafka Listeners and Kafka Transactions
• ✅ Outbox Pattern with JPA ACID Transactions
• ✅ Synchronizing Kafka and Database Transactions with Kafka and JPA Transaction Manager
• ✅ Handling Concurrency and Parallelization with Kafka Consumer Groups and Partitions
• ✅ Sequential processing guarantee for messages with the same key in Kafka
• ✅ Parallel processing for messages in different Kafka Partitions
• ✅ Handling Resiliency with Kafka Consumers and Retryable Exceptions
• ✅ Retryable Exceptions and Dead-Letter Queue Error Channel with Spring Kafka
• ✅ E2E Concurrency Testing Framework with Completable Futures using Spring Cloud OpenFeign

📘 Cloud-Native Startup Template

• 📖 This Cloud-Native Full-Stack Developer Template provides fully functional Development and Production Environment
• 📖 Next.js and React UI
• 📖 Event-Driven Microservices with Spring Boot, Kafka, PostgreSQL and Outbox Pattern
• 📖 Swagger UI Gateway with Keycloak Authorization
• 📖 Kafka UI
• 📖 Kafka Cluster with Kubernetes Strimzi Operator
• 📖 E2E Testing Service with Spring Cloud OpenFeign REST Client
• 📖 Local Kubernetes Development Environment with Skaffold
• 📖 Production Kubernetes Development Environment with Skaffold
• 📖 Github Actions CI/CD GitOps pipeline
• 📖 Azure Terraform Infrastructure with AKS Kubernetes Cluster and Private Container Registry

📘 Full Technology Stack:

• ✅ React UI
• ✅ Next.js React Framework with Keycloak Authorization
• ✅ Event-Driven Microservices with Spring Boot and Kafka
• ✅ Spring Kafka Event Listeners
• ✅ Synchronized Kafka and JPA Transactions for Outbox Pattern
• ✅ Safe Idempotent Retry Transactions with Idempotency Key and At-Least Once Delivery Guarantee
• ✅ Lookup Tables for Safe Idempotent Retries using Idempotency Key and PostgreSQL Database
• ✅ PostgreSQL Database
• ✅ Keycloak OAuth2 Authorization Server
• ✅ Spring Cloud Gateway with Keycloak Authorization
• ✅ Spring Cloud OpenFeign
• ✅ Swagger UI with Keycloak Authorization
• ✅ Kafka UI
• ✅ Terraform
• ✅ Kubernetes
• ✅ Github Actions
• ✅ Github Secrets and envsubst Environment Variables parser
• ✅ Kubernetes Secrets and Configmap Variables
• ✅ Local Kubernetes Development Environment with Skaffold
• ✅ Production Kubernetes Development Environment with Skaffold
• ✅ Custom Kubernetes Manfiests Generation for Local and Production Environments with sh scripts
• ✅ Custom Skaffold Manifests Generation for Local and Production Environments with sh scripts
• ✅ Hot reload of Next.js React Javascript code for Local and Production Environments with Skaffold
• ✅ Hot reload of Spring Boot Java code for Local and Production Environments with Skaffold
• ✅ Hot reload of Docker Containers for Local and Production Environments with Skaffold

📖 Links

• Concurrency and Resiliency Patterns in Saga Transactions for Spring Boot Microservices, Parts 1-4
• Apache Kafka for Event-Driven Spring Boot Microservices Udemy Course
• Demo Saga Pattern, Outbox Pattern using Spring Boot, Debezium, Kafka, Kafka Connect
• Stock Tracking Platform with Outbox Pattern, Kafka Event Streaming, Debezium CDC Connector and PostgreSQL
• Video Streaming Platform with Debezium CDC Kafka Connector, Kafka Event Streaming, Minio File Storage and FFmpeg Video Processing
• E2E Testing Pipeline for Spring Boot Microservices using OpenFeign Client and Github Actions
• Microservices with Node JS and React Udemy Course

📚 Step-by-Step Setup Guide

📖 Prepare your environment (skip if you only need local deployment)

1️⃣ Prepare GitHub Account

• Make sure you have your own Github Account

---

2️⃣ Create Your GitHub Repository

• Clone this repository and copy the source code to your new repository

---

3️⃣ Prepare Azure Account

• Make sure you have your own Azure Account with enough permissions (Sign Up for a Free Trial, if you don't have one)

---

4️⃣ Customize Source Code & GitHub Workflows

• Edit ".github/workflows/deploy-*.yaml" files: replace "master" with the name of your main branch (you can change default main branch name in github repository settings)
• Edit "k8s/prod/ingress-srv.yaml" file: replace "skycomposer.net" with the name of your registered domain (see Step 5 and section Azure Production Environment for more details)

---

5️⃣ Register Your Domain

• You need a registered domain to provide TLS connection with trusted Certificate Authority.
• For more details on setting up TLS on AKS Ingress with LetsEncrypt see this article: https://medium.com/@jainchirag8001/tls-on-aks-ingress-with-letsencrypt-f42d65725a3
• This article will show you how to configure TLS on AKS with LetsEncrypt for any registered domain, including AWS Route 53.
• Make sure that you know how to create Hosted Zone and Record A for your domain provider.
• For more details, see the section Azure Production Environment

---

⚙️ Local Kubernetes Environment Setup with Skaffold:

1. Create local Kubernetes Cluster. If you have Docker Desktop, just go to Settings -> Kubernetes -> Enable Kubernetes -> Apply & Restart

2. Switch context to local Kubernetes Cluster. If you have Docker Desktop, just go to Kubernetes Context and select "docker-desktop"

3. create .env.local file in env folder and provide the following parameters:

CONTAINER_REGISTRY="moviebets.azurecr.io"
DOCKER_FILE_NAME="Dockerfile"
DOCKER_PUSH="false"
VERSION="latest"
BASE_URL="http://ingress-nginx-controller.ingress-nginx.svc.cluster.local"
API_BASE_URL="http://ingress-nginx-controller.ingress-nginx.svc.cluster.local/api"
KEYCLOAK_BASE_URL="http://localhost/keycloak"

4. Build:

mvn clean install -DskipTests

This command will prepare java libraries for deployment

4. Deploy:

sh skaffold-local.sh

This script will build docker images and start local kubernetes environment with hot reloading of your code changes

6. Open http://localhost and test the app.

• Make sure that Keycloak Login and Register works, and you are able to Create Movie Event and place bet on it.
• You can use credentials: admin / admin for user with admin privileges and credentials: user / user for regular user
• Any new user, registered with Keycloak, is automatically assigned regular user privileges
• Optionally, create 2 test accounts by clicking Register on the Keycloak Login Page, create movie event and place bet with one account and do the same with another account.
• If the winner exists at the move event close time, then the movie event is closed, and the bets start to settle by recursively triggering kafka events during settlement, until all settlement jobs are finished.
• Note: If the winner doesn't exist, then the movie event is configured to be closed later, after configurable amount of time (6ß seconds by default)
• The winner exists only if the movie has more votes than another movie. If the votes are equal, the movie event is extended for configurable duration, until there is a winner.

🎉 Congratulations!

• You successfuly tested Movie Bets App locally!

Azure Production Environment (with Terraform & Skaffold)

1. Create terraform.auto.tfvars file in infra folder and provide following parameters:

kubernetes_version= "1.32.2"
app_name = "{provide_your_own_globally_unique_name}"
location = "westeurope" (use any other azure location, for example, "germanywestcentral", if you have any issues with "westeurope")

2. Replace moviebets with your own globally unique name (see files container-registry.tf, kubernetes-cluster.tf and resource-group.tf)

3. Login to Azure Cloud with az login CLI

4. CD to infra folder

5. Run terraform initand terraform apply --auto-approve

6. After the script is successfully finished, run the following command:

az aks get-credentials --resource-group {your_app_name} --name {your_app_name}

• Make sure that your context is switched from local Kubernetes Cluster to Azure Kubernetes Cluster. If you have Docker Desktop, just open Kubernetes Context and make sure that the name of the context corresponds to your Azure Kubernetes Cluster

6. run kubectl get pods and make sure that kubectl works correctly and returns 0 resources

7. Login to Azure Container registry with the following command:

docker login {login_server}

• You can find docker login server, username and password in Azure Cloud (go to Container Registry -> Settings -> Access Keys)

8. In env folder create .env.prod file and set the following environment variables:

CONTAINER_REGISTRY="moviebets.azurecr.io"  (provide your own globally unique container registry)
DOCKER_FILE_NAME="Dockerfile"
DOCKER_PUSH="true"
VERSION="latest"
BASE_URL="https://skycomposer.net" (provide your own domain name, see `Step-05` and notes below for more details)
API_BASE_URL="https://skycomposer.net/api"
KEYCLOAK_BASE_URL="https://skycomposer.net/keycloak"

• Make sure you set your own values for CONTAINER_REGISTRY, BASE_URL, API_BASE_URL and KEYCLOAK_BASE_URL (replace skycomposer.net with the name of your domain)

9. Register your domain and enable TLS on AKS Ingress with Lestencrypt: https://medium.com/@jainchirag8001/tls-on-aks-ingress-with-letsencrypt-f42d65725a3

• Make sure you provide your email for CA cluster issuer in k8s/cert/cluser-issuer.yml Kubernetes resource (see more details in the article)
• You can find production Ingress Kubernetes Resource in k8s/prod/ingress-srv.yaml. This resource will be applied with sh skaffold-prod.sh script. Make sure that you replaced skycomposer.net with your registered domain name
• When you run sh skaffold-prod.sh, you will also install ingress controller and letsencrypt kubernetes resources, described in the article, see the details in scripts/prod/prepare-k8s.sh file

10. run sh skaffold-prod.sh

• This script will build docker images, push them to azure container registry and deploy images to production kubernetes cluster with hot reloading of your code changes

11. Run kubectl get pods and make sure that all containers are RUNNING

12. Edit ingress cm-acme-http-solver, as described in the article: https://medium.com/@jainchirag8001/tls-on-aks-ingress-with-letsencrypt-f42d65725a3

13. Verify that certificate tls-secret is working (Ready = True, see the article for more details)

14. Open https url with your registered domain in your Browser and make sure that Keycloak Login and Register works, you are able to Create a Movie Event and place a bet on it

• Optionally, register 2 new users, create movie event with one account and place bet with another account
• The only difference between skaffold dev and skaffold run is that skaffold dev allows hot reloading of your code changes on production! Try to make any code change with your IDE and you will immediately see this change on production!
• If you run sh skaffold-local.sh or sh skaffold-prod.sh you will see logs in real-time. After closing the cli window, all kubernetes resources will be destroyed! Therefore, in order to deploy final changes to production, replace skaffold dev with skaffold run in the skaffold-prod.sh script. You will not have hot reloading with skaffold run anymore, but kubernetes resources will not be destroyed after you close cli window.

🎉 Congratulations!

• You successfully tested Moive Bets App in production!

Github Actions Deployment Pipeline Setup

1. Create the following Github Secrets (Go to Your Repository -> Settings -> Secrets and Variables -> Actions -> New Repository Secret):

CONTAINER_REGISTRY=... (Azure Container Registry)
KUBE_CONFIG=.. (Base64 encoded  ~/.kube/config file contents)
REGISTRY_UN=... (Azure Container Registry Username)
REGISTRY_PW=... (Azure Container Registry Password)

• You can find values for CONTAINER_REGISTRY, REGISTRY_UN and REGISTRY_PW in Azure Cloud (go to Container Registry -> Settings -> Access Keys)
• You can get the value of KUBE_CONFIG with this command cat ~/.kube/config | base64 (make sure you switched context to Azure Production Kubernetes Cluster before running this command!)

2. Make any code changes (for example change SkyComposer to SkyComposer 2 in moviebets-ui/components/header.js file)

3. Push changes with git add ., git commit -m "test changes" and git push origin

4. Go to "Your repository -> Actions" and make sure that the Deployment Pipeline is automatically started and successfully finished

• This pipeline will build changed docker image, push it to container registry and deploy changed image with new version to kubernetes cluster

5. Open https link for your registered domain in your Browser and make sure that you can see SkyComposer 2 title on the top left

🎉 Congratulations!

• You successfully tested Movie Bets App code changes with Github Actions Deployment Pipeline!