For those that have been tracking the development, AWS WAF has proven to be a very easy to use web application firewall service. In fact, it’s been a little too easy. In a matter of moments, you can launch an AWS WAF ruleset at CloudFront or at an application load balancer. There were even some very helpful quick starts from AWS that leveraged Lambda and CloudFront or load balancer request logging to identify rate based threats and auto blacklist, apply honey pot defense mechanisms against bots and scrapers, as well as subscribing to a list of known bad actor IP addresses and have that list blacklisted hourly. All very useful functionality for someone looking to defend a web application against the wiles of the internet. What did you have to trade for all this turn-key goodness? The one thing that everyone who operated a WAF needs, visibility.
In true AWS fashion, they have heard the customer. At the end of August, they launched AWS WAF logging. For each web request, AWS WAF logs now provide raw HTTP/S headers along with information on which AWS WAF rules are triggered. This is useful for troubleshooting custom WAF rules and Managed Rules for AWS WAF. These logs will be made available via Amazon Kinesis Data Firehose in the JSON format.
This provides incredible value for those who need visibility into request headers for requests that traverse AWS WAF. It also maps at a log level the specific request that triggered the AWS WAF rule for archiving. This was a feature once reserved for those fast enough to login to see the request in the console in the narrow window that it was made available there. Now, you can use any old log parsing tool your heart desires to gain visibility in to what the AWS WAF is seeing. This is helpful not only for archiving historic threats, but also for establishing baselines and norms as you develop a logging strategy using AWS WAF.
For now, there is a dependency on Kinesis Firehose. I am not knocking Firehose at all. It is a robust way to ingest stream data such as logs. However, as of this writing, the firehose offering specifically is not yet on the FedRamp road map which may preclude some Government agency adoption. For those who are ready to adopt a pipeline for log analytics such as AWS WAF Firehose, the future is here.
To get started, you simply create an instance of Kinesis data firehose in the appropriate AWS account. From here, you can also choose the target for your log stream, be it S3, ElasticSearch, or RedShift. You can also leverage third party log aggregation and analytics tools for advanced monitoring of AWS WAF request traffic. You’ll notice in the AWS WAF service Web ACL there is a new tab at the top where you can enable logs for the Web ACL. When configuring, customers also have the option of redacting fields from web requests that they do not want to be logged. Like most AWS service like this, the service itself is free! You only pay for the resources that are generated as a part of using it, such as S3 storage costs for the logs.
Give it a shot!-Jeff Carson, Solution Architect