Difference in CSV export lines between Firewall and Panorama
Why is it that when I use the command >scp export log traffic query start-time equal <time stamp> end-time equal <time stamp> to <location> on a firewall, I can get a CSV file that has more than 1 million lines, but when the command is ran on a Panorama I only get a maximum amount of 65535 lines?
The distributed nature of Panorama and PA-7000 platforms makes that a log query will cause several sources to be accessed and potentially terrabytes of data needed to be sifted through to accommodate for the export which could cause performance degradation, as the management plane will be taxed, and network congestion in distributed collector environments. This is why the log export capability is set to a 65535 lines limitation by default for these platforms. The total number of exported lines can be increased to 1 million by setting the max-log-count parameter.
This limitation is not imposed on firewall platforms as they store their logs on a single disk with limited storage capacity, making a large query less resource intensive. Log export on a firewall system is limited to 4 billion lines.
If log needs to be routinely exported off of Panorama, consider Configure Log Forwarding from Panorama to External Destinations