What are EMR Flow - New Flow Logs implementation Prerequisites?

• Prisma Cloud Enterprise (SaaS)
• CSPM

Who should look to upgrade to S3- Hourly partition? 

• Customers currently set up on the S3- 24-hour partition.

• Customers who use Cloudwatch to send in the flow logs

• Customers currently sending in a large volume of logs into Prima Cloud and see a lag in their flow logs ingestion..

What does this Flow log implementation for AWS offer?

• Better ingestion performance than the existing S3 Flow logs implementation, thereby solving the lag problem.

• Address False positives with Internet exposure-related network policies.

Why Should the customer create a new Flow log in AWS and not continue to use an existing one?

Flow log is immutable, the new flow requires two additional things in the Flow log -

• Partition logs every hour

  • Partitioning logs per hour makes Prisma Cloud make fewer calls to the customer's S3 bucket, reducing costs for the customer.

  • Fewer calls mean better Ingestion performance.

• Additional fields

  • Additional fields like TCP-flags and flow-direction are used in the heuristic used in the Internet exposure calculation in Network policies. That means lesser False Positives for these Network policies.

 

Steps to Follow 

1. Create a Flow log with the following specifics
   1. Flow logs format requires all required and connection direction related fields.

Sample Flow logs format :
${account-id} ${action} ${interface-id} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${protocol} ${packets} ${bytes} ${start} ${end} ${log-status} ${region} ${version} ${tcp-flags} ${flow-direction} ${traffic-path} ${vpc-id} ${subnet-id} ${instance-id} ${pkt-srcaddr} ${pkt-dstaddr} ${pkt-src-aws-service} ${pkt-dst-aws-service} 

1. Required fields

• account-id

• action

• interface-id

• srcaddr

• dstaddr

• srcport

• dstport

• protocol

• packets

• bytes

• start

• end

• log-status

• region

• version

• tcp-flags 

• flow-direction

• traffic-path

• vpc-id (extra metadata for resource tagging)

• subnet-id (extra metadata for resource tagging)

• instance-id (extra metadata for resource tagging)

• pkt-srcaddr (preserves original src IP)

• pkt-dstaddr (preserves original dst IP)

• pkt-src-aws-service (future use)
  pkt-dst-aws-service (future use)

 

3. Partition logs by time : Every 1 hour (60 minutes)

.

4. Currently only text format is supported for ingestion

Please note that no change is required on the Prisma side as long as the S3 Bucket does not change.

 

• There is an issue where flow logs for AWS can create false positives because we are reading in AWS v2 flow log format.
• Some symptoms may include:
  • False positive - Instance exposed to internet