XML API Issue With Passwords Containing Special Characters
106498
Created On 09/25/18 20:36 PM - Last Modified 04/21/20 00:46 AM
Symptom
This XML file does not appear to have any style information associated with it. The document tree is shown below. <response status="error" code="400"> <result> <msg>Missing value for parameter "password".</msg> </result> </response>
https://198.51.100.1/api/?type=keygen&user=apiuser&password=api#user (ran in browser)
This XML file does not appear to have any style information associated with it. The document tree is shown below. <response status="error" code="403"> <result> <msg>Invalid Credential</msg> </result> </response>
Cause
In all the cases the password contains special characters. The API key generation will fail when the password contains special characters such as # and &. This is not a PAN-OS specific issue. This is due to the way browsers and cURL handle special characters. This is because these are reserved characters used as general or sub delimiters.
RFC defining list of delimiters: https://tools.ietf.org/html/rfc3986#section-2.2
RFC: https://tools.ietf.org/html/rfc3986#section-3.5
In section 3.5: A fragment identifier component is indicated by the presence of a number sign "#" character and terminated by the end of the URL. Now, look at cURL official documents: https://curl.haxx.se/.
Curl supports fragments fine when a URL is passed to it, but the fragment part is never actually sent over the wire, so it doesn't make a difference to cURL's operations whether it is present or not. Hence, the "#" is never sent across the wire, resulting in such behavior.
When these are used without URL encoding, you can see that the browser does not actually send the entire URL (password in this case). In the below examples, HTTP was enabled for demonstration purposes, and the ease of packet captures show the HTTP payload.
Example 1: username: user2 and password: user&2
Browser:
Packet capture taken at the client sending the GET request:
If you look at the password section, the browser truncates the password at "user."
Example 2: username: user3 and password user#3
Browser:
Packet capture taken at the client sending the GET request:
If you look at the password section, the browser truncates the password at "user." Hence, the issue is due to the interpretation of special characters by browsers and cURL.
Resolution
Here is a quick reference of reserved character set:
reserved = gen-delims / sub-delims
gen-delims = ":" / "/" / "?" / "#" / "[" / "]" / "@"
sub-delims = "!" / "$" / "&" / "'" / "(" / ")"
/ "*" / "+" / "," / ";" / "="
If the data for a URI component would conflict with a reserved character's purpose as a delimiter, then the conflicting data must be URI encoded. In this case, if the password contains reserved characters, they have to be encoded and then used as input.
For instance, if the password contains special characters such as "#" and "&," use the URL encode %23 and %26 respectively.
Example API Request:
https://198.51.100.1/api/?type=keygen&user=apiuser&password=api%23user
Additional Information
For additional information, here are some articles for reference:
https://docs.paloaltonetworks.com/pan-os/8-1/pan-os-panorama-api/get-started-with-the-pan-os-xml-api/get-your-api-key
For easy encode and decode of URIs:
https://www.url-encode-decode.com/
For the list of all special character encoding for HTML:
https://www.w3schools.com/tags/ref_urlencode.asp