AIX support for hardware enhanced high precision network time synchronization

Request Abstract
Pursuant to maximum precision network time sync, the RFE requests capabilities in POWER/AIX be enhanced to include hardware timekeeping and time synchronization, and the ability to audit compliance. This may include, but is not limited to, changes to the kernel, network driver, PTP client.




Regulatory requirement: MiFID II clock synchronization and timestamping


References:
https://www.esma.europa.eu/sites/default/files/library/2015/11/2015-esma-1464_annex_i_-_draft_rts_and_its_on_mifid_ii_and_mifir.pdf pp. 506-507


Commentary on the regulation:
https://www.bloomberg.com/trading-solutions/blog/mifid-ii-clock-synchronization-advice-dont-drop-the-mic-part-1/


Requirements and Wish list
- Precision MUST be within 100us of UTC.
- Precision is desired to be within 10us of UTC.


Use case: One use case is a low-latency distributed workflow for foreign exchange that traverses several systems. Minimum cycle time for the workflow is in the ballpark of 200us. The application team relies on timestamps at various checkpoints in the workflow to assess performance and issues on a per transaction basis.


The current environment uses software NTP. Concern was raised about skew in timestamps, citing incidents where the start-to-finish durations are posting negative. Tests confirm that the volatility in default software NTP synchronization between servers can be well more than the 100us ballpark needed for sane transaction measures, and are seen to exceed 1ms. NTP doesn't guarantee higher precision. Furthermore, this situation would be non-compliant under upcoming MiFID II regulation.




Prior work
Software PTP was provided in DCR #MR031214247, and that is being evaluated as a potential solution with higher precision.


However, it is not clear that the required or desired precision will be maintained effectively in software. Compliance needs to be proven.


IBM recently confirmed the current state of the latest AIX Mellanox driver and PTP client is that neither supports hardware time synchronization.




Assumptions
- Server: POWER 7+ (minimum p780+), POWER 8 (S824, E850, E880)
- Network card: Mellanox ConnectX-3 PRO 10Gb dual-port
- OS: AIX 7.1 TL3 SP3 or higher




Requested enhancement (above, see Request Abstract)


Partner references
- FSMLabs TimeKeeper: (http://www.mellanox.com/related-docs/whitepapers/WP_Highly_Accurate_Time_Synchronization.pdf)
- Mellanox: https://community.mellanox.com/docs/DOC-2404