Thanks for the feedback on the patch. I'll make the style corrections and push an updated version.

Hi @osresearch, sorry for the delay in getting back to you on this! I just finished looking into performance testing this patch and found that there seems to be a significant enough regression that I am hesitant to merge this in its current state. I would like to think for a while longer about how to progress this PR, as I do think this would be a nice change to have! Some options to consider are:

• Making a similar change to a different ef_vi test app
• Locking this behind a build option
• Duplicating eflatency to have eflatency_memops (for example) which incorporates this change

Although I haven't thought for long enough to decide which one of these would be most appropriate.

Thanks for doing the performance testing on the patch, @jfeather-amd . Can you describe where the slowdowns seem to be? In the non-vlan, non-echo, non-validating case (the default), my latency deltas were in the noise on the X2 and X3 cards, so I'm very curious about your methodology so that I can replicate the results for my future testing.

I've re-run tests on the X3 cards with better isolation and pinning the eflatency task to a single CPU; the results show no change in the min, 50%, 95% and 99% numbers, although there is an unexpected increase in the mean of about 50ns. This is caused by the unconditional memset() and checksum_udp_pkt() on the send side, although these occur outside of the ci_frc64_get() timing loop and which I had assumed would not affect the timing. Adding if(cfg_validating)... around the packet rewriting removes this effect.

However, this performance regression appears to be an issue with the way mean is computed -- it is the total time for all packets (delta between the two gettimeofday() calls), not the mean of the measured times (rdtsc ticks). I wonder if the mean should be computed as the average of the actual times instead. It is unexpected to me that the first column of results doesn't match the data used for the other columns. I've submitted #240 to compute the mean from the timings array instead of the wall clock time.

Hello @osresearch, sorry for not replying sooner. We are busy!

The purposes why we use eflatency include performance comparisons between various OpenOnload versions on different hardware configurations. If we change the methodology behind eflatency reports, we won't be able to compare them between the released OpenOnload versions without the change and newer ones with the change. ("Comparing apples and oranges".) It might not be a technical problem, but it is still significant work to update the ecosystem to accommodate this change gracefully without confusing the users (e.g. "Why did the performance unexpectedly become better/worse?"), and we are not motivated to do it now.

Would you like to contact our customer support team (and sales team) so we can better understand your motivation and usage cases and figure out what we can do about them? You would need to email support-nic@amd.com.