Prisma Cloud Compute Registry scanning shows '429 Too Many Request (HAP429)'

• Initiating the Registry Scanning shows below error: 

failed to retrieve repository TESTORG* tags, error failed to query https://registry-1.docker.io/v2/TESTORG*/tags/list, error failed to query https://registry-1.docker.io/v2/TESTORG*/tags/list, Too Many Requests Too Many Requests (HAP429)

• Prisma Cloud Compute
  • Self-Hosted
  • SaaS
• Docker Hub Registry

• 429 indicates too many requests are sent in a given amount of time (rate-limiting).
• When you issue a pull request and you are over the limit for your account type, Docker Hub will return a 429 response code when the manifest is requested.
• Docker Hub limits the number of Docker image downloads (“pulls”) based on the account type of the user pulling the image.
• Pull rate limits are based on individual IP addresses.
• For anonymous users, the rate limit is set to 100 pulls per 6 hours per IP address.
• For authenticated users, it is 200 pulls per 6-hour period. There are no limits for users with a paid Docker subscription.

• Set Cap to "1" to scan the most recent Registry images. The default value of 5 will scan the most recent 5 images. Setting this field to 0 will scan all images.
• Increase the scan frequency from Manage > System > Scan. By default, Prisma Cloud scans the Registry every 24 hours. Increase it to 72 hours or more as per the scenario. 
• Increase the Rate Limit at the Docker. 

The Defender Process to scan the Registry image

• The image is pulled by the defender to perform the scan.
• Once the scanning is completed, the image will be removed. 
• The Defender logs show the same:

  • Analyzing the image

  • Pulling the image

  • Scanning the image

  • Removing the image

DEBU 2022-03-18T21:49:35.559 scanner.go:145 Analyzing image testimage/test latest:sha256:3a11abed3c123456789023d23b7d0e828c939a4284512345678902138aa3f76a

DEBU 2022-03-18T21:49:35.559 scanner.go:161 Pulling registry image id: sha256:3a11abed3c123456789023d23b7d0e828c939a4284512345678902138aa3f76a tag: test.io/testimage/test:latest

DEBU 2022-03-18T21:49:54.610 scanner.go:167 Pull image tag:test.io/testimage/test:latest id: sha256:3a11abed3c123456789023d23b7d0e828c939a4284512345678902138aa3f76a completed after 19.051067993 seconds

DEBU 2022-03-18T21:49:54.618 scanner.go:100 Image scanning started sha256:3a11abed3c123456789023d23b7d0e828c939a4284512345678902138aa3f76a

DEBU 2022-03-18T21:49:54.619 containerized_runner.go:69 Containerized scan started sha256:3a11abed3c123456789023d23b7d0e828c939a4284512345678902138aa3f76a

DEBU 2022-03-18T21:50:54.066 containerized_runner.go:270 Containerized scan completed sha256:3a11abed3c123456789023d23b7d0e828c939a4284512345678902138aa3f76a 59.45 [sec]

DEBU 2022-03-18T21:50:54.103 augment_static.go:218 sha256:3a11abed3c123456789023d23b7d0e828c939a4284512345678902138aa3f76a [augment] 59.49 [sec]

DEBU 2022-03-18T21:50:54.105 scanner.go:218 Image test.io/testimage/test:latest sha256:3a11abed3c123456789023d23b7d0e828c939a4284512345678902138aa3f76a [scanning] 59.49 [sec]

DEBU 2022-03-18T21:50:54.105 scanner.go:221 Scan for image tag:test.io/testimage/test:latest id: sha256:3a11abed3c123456789023d23b7d0e828c939a4284512345678902138aa3f76a completed after 78.545885826 seconds

DEBU 2022-03-18T21:50:54.105 scanner.go:409 Removing image sha256:3a11abed3c123456789023d23b7d0e828c939a4284512345678902138aa3f76a

How to check Rate Limit by pulling the image

• When you issue a pull request and you are over the limit for your account type, Docker Hub will return a 429 response code with the following body when the manifest is requested:

You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limits

How can I check my current rate?

• Refer to Docker Download rate limit.
• Valid manifest API requests to Hub will usually include the following rate limit headers in the response:

ratelimit-limit    
ratelimit-remaining

• These headers will be returned on both GET and HEAD requests. Note that using GET emulates a real pull and will count towards the limit; using HEAD will not, so we will use it in this example. To check your limits, you will need curl, grep, and jq installed.
• ​​​​To get a token anonymously (if you are pulling anonymously):

$ TOKEN=$(curl "https://auth.docker.io/token?service=registry.docker.io&scope=repository:ratelimitpreview/test:pull" | jq -r .token)

• To get a token with a user account (if you are authenticating your pulls) - don’t forget to insert your username and password in the following command:

$ TOKEN=$(curl --user 'username:password' "https://auth.docker.io/token?service=registry.docker.io&scope=repository:ratelimitpreview/test:pull" | jq -r .token)

• Then to get the headers showing your limits, run the following:

$ curl --head -H "Authorization: Bearer $TOKEN" https://registry-1.docker.io/v2/ratelimitpreview/test/manifests/latest

• Which should return headers including these:

ratelimit-limit: 100;w=21600
ratelimit-remaining: 76;w=21600

• This means my limit is 100 pulls per 21600 seconds (6 hours), and I have 76 pulls remaining.