La conexión gRPC a iot.services-edge.paloaltonetworks está rota, error: se superó la fecha límite de contexto

La conexión gRPC a iot.services-edge.paloaltonetworks está rota, error: se superó la fecha límite de contexto

34165
Created On 05/18/21 15:51 PM - Last Modified 06/10/25 22:59 PM


Symptom


No se puede establecer la conexión gRPC a iot.services-edge.paloaltonetworks.com y se agota el tiempo de espera.

También se generan registros del sistema de alta gravedad que indican problemas de conexión.
2021/04/26 09:49:58 high     iot            grpc-co 0  gRPC connection to iot.services-edge.paloaltonetworks.com:443 is broken, error: context deadline exceeded time: 2021-04-26 09:49:58

Las estadísticas de IoT pueden darnos información histórica sobre el estado de la conexión. Además, desde la misma salida podemos verificar el estado del certificado del dispositivo. Alternativamente, podemos usar el comando cli "show device-certificate status"
admin@PA-VM-100-1> show iot icd statistics all

Summary of ICD gRPC client [tcp//iot.services-edge.paloaltonetworks.com:443]:
number of connection reset:       0
number of connection failed:      12480
number of connection established: 0
number of connection attempts:    12481 
number of connection released:    12479 
number of connection selected:    0
number of selections failed:      287959
number of bytes sent:             0
number of bytes received:         0
Last gRPC connection Attempt:     2021-04-29 17:12:12 +0200 CEST
Last successful gRPC connection:  1970-01-01 01:00:00 +0100 CET 


Summary of gRPC connections [configured source IP: ]:
ICD device cert status: Installed
        Validity: 
                Notbefore: 2021-04-23 12:26:29 +0000 UTC 
                Notafter: 2021-07-22 12:26:29 +0000 UTC
EnforcerURL: enforcer.iot.services-edge.paloaltonetworks.com:443

max gRPC connections: 1, max alive time: 900, max bytes sent: 0
[0]gRPC conn[192.168.10.32:56326 -> 35.223.164.209:443], state false, selected 0, sent 0, received 0, close @1970-01-01 01:00:00 +0100 CET, backup true
grpc stats: wire 0, app: 0, num: 0
Error code: context deadline exceeded


Unknown IP Query LRU statistics:
number of entries          : 0
number of expired entries  : 0
number of queries to cloud : 0
number of queries ignored  : 8822081
number of queries answered : 0


Verdict LRU statistics:
number of verdicts            : 0
number of verdicts ignored    : 0
number of verdicts pushed out : 0
GETALL duration               : nil


Summary of connections to dataplane[slots: 1, dps: 1]:
s1dp1: address 127.1.1.2, online true, conn state true


Current iot bookmark: not received yet
Last verdict request:             1970-01-01 01:00:00 +0100 CET


Summary of ICD Redis  [unix:/opt/pancfg/cache/iotd/redis_iotd.sock]:
number of verdicts enqueued:          0
number of verdicts discarded:         0
number of unknown verdict types:      0
number of unknown verdicts failed:    0
number of verdicts dropped:           0
number of failed logging updates:     0
number of verdicts persisted:         0
number of verdicts unchanged:         0
number of bad content errors:  0
number of Redis conn attempts:        1
number of successful conn:            1
Last good Redis connection:        2021-04-26 09:07:56 +0200 CEST
Last failed Redis connection:      Never
Last verdict writing time:         Never


Summary of ICD device verdict to Iotd daemon:
number of verdicts enqueued:      0
number of verdicts discarded:     0
number of verdicts sent:          0
number of verdicts unsent:        0
Last verdict sent to Iotd:        1970-01-01 01:00:00 +0100 CET


Summary of ICD device verdict to DP:
number of verdicts enqueued:      0
number of verdicts discarded:     0
number of verdicts sent:          0
number of verdicts unsent:        0
Last verdict sent to DP:          1970-01-01 01:00:00 +0100 CET

Current Time:  2021-04-29 17:12:20.526017905 +0200 CEST m=+288265.567775167

Si comprobamos el archivo icd.log (>less mp-log icd.log) podemos ver un error en el protocolo de enlace:
{"level":"info","time":"2021-04-26T09:49:36.917084697+02:00","message":"Count of OCSP urls: 1"}
{"level":"info","time":"2021-04-26T09:49:36.928677307+02:00","message":"Got OCSP response verify OK from http://r3.o.lencr.org"}
{"level":"info","time":"2021-04-26T09:49:36.928705544+02:00","message":"Check OCSP CN=iot.services-edge.paloaltonetworks.com took 22.890651ms"}
{"level":"info","time":"2021-04-26T09:49:36.928744605+02:00","message":"Certificate is valid: CN=iot.services-edge.paloaltonetworks.com"}
{"level":"info","time":"2021-04-26T09:49:36.929032089+02:00","message":"Cert expiration check. Subject: CN=R3,O=Let's Encrypt,C=US NotAfter: 2021-09-29 19:21:40 +0000 UTC NotBefore: 2020-10-07 19:21:40 +0000 UTC"}
{"level":"error","time":"2021-04-26T09:49:36.940562214+02:00","message":"CRL HTTP Get failed: status code = Ƿ"}
{"level":"info","time":"2021-04-26T09:49:36.940602182+02:00","message":"downloadCRLWithURL crlURL=http://crl.identrust.com/DSTROOTCAX3CRL.crl took 11.537066ms"}
{"level":"info","time":"2021-04-26T09:49:36.940619216+02:00","message":"Fetch CRL from http://crl.identrust.com/DSTROOTCAX3CRL.crl successfully before timeout."}
{"level":"info","time":"2021-04-26T09:49:36.940629875+02:00","message":"Check CRL crlURL=http://crl.identrust.com/DSTROOTCAX3CRL.crl took 11.58108ms"}
{"level":"error","time":"2021-04-26T09:49:36.940638759+02:00","message":"Downloading CRL from http://crl.identrust.com/DSTROOTCAX3CRL.crl"}
{"level":"warn","log":"grpc","time":"2021-04-26T09:49:36.940721237+02:00","message":"grpc: addrConn.createTransport failed to connect to {iot.services-edge.paloaltonetworks.com:443 0  <nil>}. Err :connection error: desc = \"transport: authentication handshake failed: Downloading CRL from http://crl.identrust.com/DSTROOTCAX3CRL.crl\". Reconnecting..."}
{"level":"info","time":"2021-04-26T09:49:37.587058192+02:00","message":"Cert expiration check. Subject: CN=iot.services-edge.paloaltonetworks.com NotAfter: 2021-06-27 05:53:01 +0000 UTC NotBefore: 2021-03-29 05:53:01 +0000 UTC"}

 

Environment

Cause


Los registros anteriores son una indicación de fallas de conexión a crl http://crl.identrust.com/

Resolution


Para resolver este problema, asegúrese de que no hay ningún dispositivo en la ruta que bloquee las conexiones a http://crl.identrust.com/ o http://r3.o.lencr.org.

Additional Information


La conexión a iot.services-edge.paloaltonetworks también puede fallar si no se puede conectar a OCSP la dirección del firewall servidor http://r3.o.lencr.org.

Desde icd.log podemos ver:
{"level":"info","time":"2021-04-02T14:14:15.612020435+11:00","message":"Count of OCSP urls: 1"}
{"level":"info","time":"2021-04-02T14:14:15.659498115+11:00","message":"Check OCSP CN=iot.services-edge.paloaltonetworks.com took 115.017831ms"}
{"level":"error","time":"2021-04-02T14:14:15.659665712+11:00","message":"Send OCSP request failed: http://r3.o.lencr.org error: OCSP HTTP response error: status code:503"}
{"level":"warn","log":"grpc","time":"2021-04-02T14:14:15.66013365+11:00","message":"grpc: addrConn.createTransport failed to connect to {iot.services-edge.paloaltonetworks.com:443 0  <nil>}. Err :connection error: desc = \"transport: authentication handshake failed: Send OCSP request failed: http://r3.o.lencr.org error: OCSP HTTP response error: status code:503\". Reconnecting..."}
{"level":"info","time":"2021-04-02T14:14:15.934510834+11:00","message":"Cert expiration check. Subject: CN=iot.services-edge.paloaltonetworks.com NotAfter: 2021-04-11 22:51:38 +0000 UTC NotBefore: 2021-01-11 22:51:38 +0000 UTC"}

 
Other users also viewed:
Actions
  • Print
  • Copy Link

    https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA14u0000001VP8CAM&lang=es&refURL=http%3A%2F%2Fknowledgebase.paloaltonetworks.com%2FKCSArticleDetail

Choose Language