I've been working on a custom Security Support Provider (SSP) package for authenticating DCOM calls from clients which do not belong to a Windows domain. I've not done any work in this area before so this has been an intensive introduction to SSP and also to some of the more complex details of Windows security (Keith Brown's book Programming Windows Security has been very helpful).
My starting point in the learning process was the SAMPSSP sample provider from Microsoft which is mentioned in the MSDN docs. I couldn't find it on the MSDN website but instead found a copy here.
The biggest showstopper with this sample code when used with DCOM is that the implementation of ImpersonateSecurityContext simply returns a success code without doing any impersonation. This results in CoCreateInstanceEx returning the ubiquitous E_OUTOFMEMORY. I'm afraid to say I wasted an appreciable amount of time on this, matters being made more confusing by the fact that co-creating an object using NTLM authentication followed by a call to CoSetProxyBlanket using the custom SSP worked fine.
One thing that surprised me is that even if RPC_C_AUTHN_LEVEL_CONNECT is specified at both ends, which I understood to mean that only the initial verification of credentials is signed to ensure integrity, there is still a call to MakeSignature and VerifySignature for both the request and response of every call on an interface.
Some other points worth mentioning are:
- The client and server must be on different machines. If not, the custom SSP is ignored in the CoCreateInstanceEx call and CoSetProxyBlanket fails.
- Microsoft.com has an informative Word document on SSPI.
- To install your SSP append the name of SSP's dll to the SecurityProviders value of this key: HKLM\SYSTEM\CurrentControlSet\Control\SecurityProviders, for example:
msapsspc.dll, schannel.dll, digest.dll, msnsspc.dll, mysample.dll
- To install your SSP for DCOM add a new value to this key: HKLM\SOFTWARE\Microsoft\Rpc\SecurityService. The name of the value is the RPC ID of your SSP (the id used in CoCreateInstanceEx and CoSetProxyBlanket) and the text is the name of your dll, e.g. 123: REQ_SZ "mysample.dll".