Streamlined AKS {Kubernetes} Log Management with Vector and Azure Blob Storage

Streamlined AKS {Kubernetes} Log Management with Vector and Azure Blob Storage

💡Highlights of this project:

👨‍💻 Provisioned an AKS cluster using Azure portal or Azure CLI, enabling deployment and management of containerized applications in a highly available and scalable environment.

👩‍💻 Deployed an Nginx deployment in a test namespace of the AKS cluster, allowing testing of containerized applications and validating cluster functionality.

🤞 Created an Azure blob storage account and container named "akslogs" to store application logs securely and reliably, facilitating log analysis and troubleshooting.

💬 Configured Vector, a high-performance log collector, to access the Azure blob storage container and stream application logs using a configuration file called "vector-config.yaml", ensuring proper storage and analysis of logs.

📝 note:-Vector is a modern, open-source, and high-performance tool that simplifies the collection, processing, and routing of logs and metrics.

🚀Deploying and managing containerized applications using AKS, managing log storage using Azure blob storage, and utilizing Vector to collect and stream application logs for analysis.

✔ Please check out my GitHub for more information and the code related to this project!

▶ GitHub Link
🔗 Romeshdg/Streaming-AKS-Kubernetes-Logs-to-Azure-Blob-Storage-using-Vector (github.com)

Azure CLI commands

az group create --name MyResourceGroup --location central India

az aks create -g MyResourceGroup -n myAKSCluster --node-count 1 --generate-ssh-keys

Step 2: Log in to the cluster and create a deployment

Step 3: Create Azure blob storage to store the logs

az storage account create --name storageromesh2002 --resource-group MyResourceGroup --location central India --sku Standard_ZRS --encryption-services blob

az storage container create --account-name storageromesh2002 --name akslogs \

Step4: Storage Account Connection String

overview:

Step 5: Prepare vector-config.yaml

Step 6: Install Vector using Helm

helm repo add vector https://helm.vector.dev helm repo update helm upgrade --install vector vector/vector -f vector-config.yaml -n test-ns --version 0.17.1

Step 7: The moment of truth!

After a few minutes see the logs streaming to the “akslogs” container in your Azure Blob Storage.

✅ Overall, this project can help organizations effectively manage and leverage their AKS logs for various use cases, such as troubleshooting, performance optimization, security, compliance, and business intelligence.

📍 Thanks for reading, and happy learning! :) ✌

✨Let's continue to learn and grow together.

follow me on LinkedIn for more updates and insights. Also, don't forget to like, share, and comment on this post to spread the word and help others in their Learning journey.

Did you find this article valuable?

Support Romesh Dharamgudi by becoming a sponsor. Any amount is appreciated!