Unable to Transfer or Conference PSTN Calls with Cloud Connector Edition (CCE)

This is a real strange one. I recently had a customer who could not transfer or conference PSTN calls using a CCE (they were able to before). This was from a mixture of Skype for Business clients, Polycom VVX handsets and Polycom CX3000 conference phones. When attempting a transfer on the Skype for Business client I got “Cannot complete the transfer”, when attempting an escalation to a conference I got “An error occurred”. When attempting to merge calls from a CX3000 I got “invalid sign-in address [email protected]”.

My first port-of-call for escalation to a conference was to check the “mediation user” was setup correctly in SfB Online - if you haven’t done this check here under “Configure online hybrid Mediation Server Settings”. I ran “Get-CsHybridMediationServer” and it was already setup correctly.

As SfB Online looked OK, I then looked at the CCE itself. I noticed that it hadn’t yet been upgraded to CCE 2.0.0 and was still running on 1.4.2. I changed the maintenance schedule so that the upgrade would happen sooner, but still the same issue on CCE 2.0.0!

OK, lets check the CloudConnector.ini file for any mistakes - I noticed REFER was being handled by the Gateway (we all love REFER to gateway). I ran an AudioCodes syslog and could see no REFER messages coming its way. In any case, I rebuilt the CCE to let Mediation handle all REFER messages, still the same (only this time I couldn’t see any RE-INVITE in the AudioCodes syslog either).

Now, this is where it’s gets interesting. I started a CCE trace (I’ve written how to do this here) and opened it up in Snooper. Filtered the trace to the particular call in question and looked for any clues. Straight away I noticed an error - “429 Provide Referrer Identity”. Looking at the ms-diagnostics for clues it mentioned:

ms-diagnostics: 1020;reason="Identity of the referrer could not be verified with the ms-identity parameter";ErrorType="The identity of the referrer could not be verified with the ms-identity information";Referrer="[email protected]";HRESULT="0xC3E93EE0(SIP_E_CRYPT_REFERRER_DATE_SKEWED)";signer="sipfed.online.lync.com";source="ap.contoso.com"
ms-split-domain-info: ms-traffic-type=SplitIntra
$$end_record

So it looks like the referrer (the user attempting the transfer) couldn’t be verified.

And then it clicked “SIP_E_CRYPT_REFERRER_DATE_SKEWED” has to be down to the time being off somewhere. I checked the CCE host and noticed it was 30 seconds ahead of the NTP “Internet Time” (time.windows.com) on my PC - the server was unable to sync the time and had crept in to the future. I set this back to two minutes behind “Internet Time”, rebooted the CCE and what do you know, call transfers and escalation to conferences work!

What is strange is that technically the skew was now further apart (I’m guessing you can’t go too far back), but if you are even slightly in the “future” it will fail - this would make sense when encryption is involved. This would also explain why it worked happily and then didn’t. Moral of the story, make sure your CCE is able to sync with time servers!