[Prisma Cloud Compute Edition] ECS Fargate Tasks Terminate with Exit Code 255 When Handling High Traffic
725
Created On 07/31/24 08:21 AM - Last Modified 02/19/26 16:54 PM
Symptom
- You have defenders installed on your ECS Fargate tasks
- Those defended tasks, when handling higher traffic, exit with code 255
- Same deployment type but with lower traffic doesn't seem to report the same error
- Without the defender, the tasks work fine even with higher traffic
Environment
- Prisma Cloud Compute Edition
- AWS ECS Fargate
Cause
- The maximum command line argument size is 128kb of data in most Linux systems (depending on the memory page size). Ref here - https://elixir.bootlin.com/linux/v4.15/source/include/uapi/linux/binfmts.h
- When the argument is larger than this size the defender fails to read the argument string and aborts the execution (it cannot find the NULL character within the size limit):
failed to fetch execve arguments: failed to read argument: failed to find string in <ArgString>
- So, the issue here is that the argument command line is too long and hits the limit of 128kb, and the defender fails to process the execution as designed
- Since, the defender is limited for 128kb only and limitation cannot be changed in the defender, the task fails
Resolution
- One possible workaround here can be to pass the encoded JSON by another file, so the argument string can be a file path to that JSON, making it much shorter.
- Or look for another solution for shorter arguments.
- Upgrade to Defender version 32.06 or later which is fixed to remove this128kb limitation for argument size as modern Linux systems can support more than 2MB arguments size as documented here: https://man7.org/linux/man-pages/man2/execve.2.html