The HTTP request is unauthorized with client authentication scheme 'Ntlm' The authentication header received from the server was 'NTLM'

I know there's a lot of questions on SO similar to this, but I couldn't find one for this particular issue.

A couple of points, first:

I have no control over our Sharepoint server. I cannot tweak any IIS settings.
I believe our IIS server version is IIS 7.0.
Our Sharepoint Server is anticipating requests via NTLM.
Our Sharepoint Server is on the same domain as my client computer.
I am using .NET Framework 3.5, Visual Studio 2008

I am trying to write a simple console app to manipulate Sharepoint data using Sharepoint Web Services. I have added the Service Reference, and the following is my app.config:

<system.serviceModel>
    <bindings>
        <basicHttpBinding>
            <binding name="ListsSoap" closeTimeout="00:01:00" openTimeout="00:01:00"
                receiveTimeout="00:10:00" sendTimeout="00:01:00" allowCookies="false"
                bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard"
                maxBufferSize="65536" maxBufferPoolSize="524288" maxReceivedMessageSize="65536"
                messageEncoding="Text" textEncoding="utf-8" transferMode="Buffered"
                useDefaultWebProxy="true">
                <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
                    maxBytesPerRead="4096" maxNameTableCharCount="16384" />
                <security mode="Transport">
                    <transport clientCredentialType="Ntlm" proxyCredentialType="Ntlm" />
                </security>
            </binding>
        </basicHttpBinding>
    </bindings>
    <client>
        <endpoint address="https://subdomain.companysite.com/subsite/_vti_bin/Lists.asmx"
            binding="basicHttpBinding" bindingConfiguration="ListsSoap"
            contract="ServiceReference1.ListsSoap" name="ListsSoap" />
    </client>
</system.serviceModel>

This is my code:

static void Main(string[] args)
{
    using (var client = new ListsSoapClient())
    {
        client.ClientCredentials.Windows.ClientCredential = new NetworkCredential("username", "password", "domain");
        client.GetListCollection();
    }
}

When I call GetListCollection(), the following MessageSecurityException gets thrown:

The HTTP request is unauthorized with client authentication scheme 'Ntlm'.
The authentication header received from the server was 'NTLM'.

With an inner WebException:

"The remote server returned an error: (401) Unauthorized."

I've tried various bindings and various code tweaks to try to authenticate properly, but to no avail. I'll list those below.

I've tried the following steps:

Using a native Win32 Impersonator before creating the client

using (new Impersonator.Impersonator("username", "password", "domain"))
using (var client = new ListsSoapClient())
{
    client.ClientCredentials.Windows.ClientCredential = new NetworkCredential("dpincas", "password", "domain");
    client.GetListCollection();
}

This produced the same error message.

Setting TokenImpersonationLevel for my client credentials

using (var client = new ListsSoapClient())
{
    client.ClientCredentials.Windows.AllowedImpersonationLevel = TokenImpersonationLevel.Impersonation;
    client.GetListCollection();
}

This produced the same error message.

Using security mode=TransportCredentialOnly

<security mode="TransportCredentialOnly">
    <transport clientCredentialType="Ntlm" />
</security>

This resulted in a different error message:

The provided URI scheme 'https' is invalid; expected 'http'.
Parameter name: via

However, I need to use https, so I cannot change my URI scheme.

I've tried some other combinations that I can't remember, but I'll post them when I do. I'm really at wits end here. I see a lot of links on Google that say "switch to Kerberos", but my server seems to only be accepting NTLM, not "Negotiate" (as it would say if it was looking for Kerberos), so that is unfortunately not an option.

Any help out there, folks?

Solution

After a lot of trial and error, followed by a stagnant period while I waited for an opportunity to speak with our server guys, I finally had a chance to discuss the problem with them and asked them if they wouldn't mind switching our Sharepoint authentication over to Kerberos.

To my surprise, they said this wouldn't be a problem and was in fact easy to do. They enabled Kerberos and I modified my app.config as follows:

<security mode="Transport">
    <transport clientCredentialType="Windows" />
</security>

For reference, my full serviceModel entry in my app.config looks like this:

<system.serviceModel>
    <bindings>
        <basicHttpBinding>
            <binding name="TestServerReference" closeTimeout="00:01:00" openTimeout="00:01:00"
             receiveTimeout="00:10:00" sendTimeout="00:01:00" allowCookies="false"
             bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard"
             maxBufferSize="2000000" maxBufferPoolSize="2000000" maxReceivedMessageSize="2000000"
             messageEncoding="Text" textEncoding="utf-8" transferMode="Buffered"
             useDefaultWebProxy="true">
                <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
                 maxBytesPerRead="4096" maxNameTableCharCount="16384" />
                <security mode="Transport">
                    <transport clientCredentialType="Windows" />
                </security>
            </binding>
        </basicHttpBinding>
    </bindings>
    <client>
        <endpoint address="https://path/to/site/_vti_bin/Lists.asmx"
         binding="basicHttpBinding" bindingConfiguration="TestServerReference"
         contract="TestServerReference.ListsSoap" name="TestServerReference" />
    </client>
</system.serviceModel>

After this, everything worked like a charm. I can now (finally!) utilize Sharepoint Web Services. So, if anyone else out there can't get their Sharepoint Web Services to work with NTLM, see if you can convince the sysadmins to switch over to Kerberos.