This post continues from previous post.
EKS Fargate : django app deployment
I will describe how to configure logging on eks fargate and implement continuous deployment using githubaction.
From previous walkthrough, we have setup a cluster and deployed a django app. Common EKS practice for logging is to have daemonset running each node and collect data from each nodes. However fargate limits daemonset for running since they are serverless in nature. AWS is working on better solution but currently they recommend writing to filesystem instead of stdout or stderr.
How to capture application logs when using Amazon EKS on AWS Fargate | Amazon Web Services
For pods running on Fargate, you need to use the sidecar pattern. You can run a Fluentd (or Fluent Bit) sidecar container to capture logs produced by your applications. This option requires that the application writes logs to filesystem instead of stdout or stderr. A consequence of this approach is that you will not be able use kubectl logs to view container logs. To make logs appear in kubectl logs, you can write application logs to both stdout and filesystem simultaneously. In the tutorial below, I am using tee write to file and stdout.
you need to configure django logger with FileHandler. I have introduced new env FILE_LOGGER to toggle FileHandler on pod deployment.
eksctl create iamserviceaccount \\
--name sample-log-sa \\
--namespace api\\
--cluster eks-sample \\
--attach-policy-arn arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy \\
--approve
kubectl get serviceaccount -n api
Flunetd is configured to send logs to cloudwatch with configured log_group_name and stream_name
We will write to temporary volume for each pod and rotate them. Fluentd will pickup log from designated folder location (configured in path) to send them to cloudwatch