SP2010 April CU stops incoming emails to a list or calendar

If you have followed my earlier post “Setting Up Incoming Mail to a List or Calendar” then you will most likely have a working incoming email configuration for your SharePoint 2010 environment. However, the laws that be at Microsoft have inadvertently released a bug in the April (2012) CU that will affect the incoming mail process.

The symptoms of this bug are that your email request will have been successfully sent through the Exchange process and arrived at your drop folder on your SharePoint web front end servers in the form of an ‘.EML’ file ready to be picked up by the timer job called ‘Microsoft SharePoint Foundation Incoming mail’. However, since the April CU was applied the .EML file will remain in the drop folder and is not picked up by the timer job. This is a frustrating bug and if you were to analyse the ULS logs a little deeper you are likely to get entries similar to the screenshot below and in particular the EventID of 6871.

The error is down to the Incoming Mail timer job somehow now being dependent on your site collection having a quota set. If you leave the site collection with the default quota of ‘0’ then the emails will fail to be picked up by SharePoint. However, this is not always consistent as I have a working SharePoint farm that is on the same farm level version with no quotas set for the site collection and incoming mail is working perfectly (very weird). So if you are unlucky to encounter this issue, then help is at hand as there are two options that you can run with which are detailed below:

Option 1: Configure your site collection with a site quota to your required specification. Once you make the changes the incoming emails that are sitting in your Drop folder will be picked up the next time the timer job has run.

Go to Central Administration > Application Management > Configure quotas and locks > Select the relevant site collection from the drop down menu > on the Site Quota Information section, set the limit to your desired number (e.g 10000 MB). Note: This be the new storage limit of your site collection so make sure have catered for future growth of the site collection.

Option 2: Download and apply the Microsoft hotfix KB2598348 which includes the fix for this bug. One thing to take into consideration for this resolution is that it will likely to bring your farm version level to July 2012 so you will need to test your new farm version in your development or staging environment to ensure it is stable for your site collections.

European SharePoint Conference Copenhagen 2013 – 5 Layers of Security

 

Day three of the conference and I attended a comprehensive session on SharePoint security by Michael Noel. Now I wouldn’t normally heap praise on a session like this as security does tend to have the yawn factor and I for one struggle to keep awake on the subject of SP security so yes I did bring match sticks to keep my eyes open just in case..

Michael Noel comes from an infrastructure background so the session turned out to be quite interesting. A couple of key takeaways from the session were certainly to utilise the ‘Always On’ feature of SQL 2012 and the Transparent Data Encryption (TDE) features to encrypt your database backups. TDE is a very good feature available in both SQL 2008 and SQL 2012.

The key points from the session are detailed below in the five layers of SharePoint Security:

  • Layer 1 – Infrastructure Security: Use Kerberos instead of NTLM for numerous benefits like less hops for authentication. The search service account and content access account should be different as this will stop users seeing content they shouldn’t normally be allowed to see.
  • Later 2 – Data Security: Use Transparent Data Encryption (TDE) to encrypt the database. Note that the temp database will also be encrypted so you will need a separate SQL instance if only some of your content databases are required to be encrypted. If you use RBS then you can use bitlocker to encrypt the files on the file server. However, the data in memory is not encrypted.
  • Layer 3 – Transport Security: External or internal certificates are recommended if your SharePoint site is external facing. Be aware that there is a 20% overhead on your web servers when using certs. It is best practice to load balance Central Admin and use SSL. The traffic between your web servers and SQL is unencrypted so to encrypt this transport layer you will need to use IPSEC as it encrypts all packets between servers.
  • Layer 4 – Internet & SharePoint: SharePoint is not designed to be Internet facing without a degree of protection, so forefront unified access gateway (UAG) along with an ISO/Proxy server and your firewalls would need to be in place.
  • Layer 5 – Rights Management: AD RMS is a form of digital rights management (DRM) that is used to restrict activities on files.

I hope you found this interesting and you have some takeaways that you might use in your environment or even consider as an option in the future.

European SharePoint Conference Copenhagen 2013 – Forensics for IT Pro’s

On the Tuesday of the SharePoint conference I attended an interesting session by Jason Kaczor on ‘Forensics for IT pro’s and administrators’. This session had a beware sign stamped all over it for developers as the session was mainly aimed at giving IT pro’s the knowledge and tools to check custom code (thoroughly!!) before it is deployed.

In a majority of cases issues with SharePoint is commonly caused by customisations (custom code). The issues that you are most likely to discover from bad custom code is memory leaks. Below are some bullet points on the tips and recommendations that I found useful from the session:

  • Only accept .wsp files (cabinet file) to deploy to your environment. Reject .exe, .msi and bat files.
  • If you have to deploy a msi (eg from a vendor) then unpack the file using msiexec or 7-zip and inspect the files. Deploy to your test environment first and look out for a licence file.
  • The wsp should contain a manifest.xml, which will list the solutions and features.
  • The solutions will have no version numbers, but the features do have version numbers as it is used to update/rollback a solution. Ensure the version number is an increment on the previous version.
  • If the wsp has no dll’s then the deployment will generally be safe. If you have multiple dll’s in your deployment then this is a potential risk to your environment.
  • Run SPDisposeCheck against the compiled code (DLL) which checks for potential memory leaks in the code.
  • If you have custom code already deployed to the GAC in your farm then you can use tools like windiff or winmerge to extract the structure of the files to a safe place for future reference.
  • The solution called ‘Lapointe.SharePoint2010.Automation.wsp’ by Gary Lapointe is a must have tool for any SharePoint environment. The solution runs a full audit of your SharePoint farm detailing all custom code deployed to it.

There are some very useful Static Analysis tools available for you to use to help troubleshoot and analyse custom code which I have listed below for you:

  • Fx Cop 10 – Performs static code analysis of .NET code.
  • Gendarme – Extensible rule-based tool to fine problems in .Net applications.
  • Cat.net – Is a binary code analysis tool for finding security vulnerabilities
  • Dependency Walker – Finds dll’s dependencies
  • Perfmon – Use is to find memory leaks.
  • iLSpy – Open source .Net assembly browser and decompiler.

European SharePoint Conference Copenhagen 2013 – Enterprise Search

I have finally got round to blogging about the European SharePoint conference. I would have liked to have given the excuse that it has taken this long due my hands still needing time to defrost from the Scandinavian chill but work and life have been on fast forward til now. Ok, enough of me rabbiting on, here is some feedback from the conference and in particular Enterprise Search.

As you may well know, SharePoint 2013 now fully integrates FAST Search for SharePoint into the main product as a service application (so no separate install). Below are some key points about SP2013 search from the Enterprise Search workshop I attended that was run by Agnes Molnar.

  • Fast Search now fully integrated as a service application
  • Deep refiners are not switched on by default, they have to be enabled.
  • A new hover button is available in your search results (very nice feature)
  • Document previews are only available for documents held within SharePoint.
  • Document previews not available for PDFs.
  • Managed Properties are now opened from the ‘Search Schema’ in your search administration.
  • ‘Results Sources’ now replaces ‘Search Scopes’ and ‘Federated Sources’ in search administration.
  • You can now create ‘Result Sources’ from managed properties.
  • A new feature called ‘Continuous Crawling’ can enable you to crawl your content sources continually. However, this is for SharePoint content sources only.
  • The Continuous Crawler component requires resources of at least 6-8 processors.
  • You can now delegate search administration to designated users.
  • People Search is now fully integrated into the Search application using the Fast search capabilities unlike in SP2010 where it had to use SharePoint Search only to crawl people data.
  • Better query rules, one query request returns multiple result sets.
  • Document parsing is different than 2010, the crawl component crawls every file in the content source regardless of the document extension. I believe powershell can be used to exclude certain document extensions if needed. This will mean that your ndex will be larger in SharePoint 2013, worth looking out for.

I hope this sheds a little light into the SP 2013 Search Application. Some companies will be some way off migrating to SharePoint 2013 but the more information we are aware of before migrating then the more prepared we will be.

I will have another blog on troubleshooting and the performance of SharePoint Search from the conference which will follow soon.

Making SharePoint Search Available in Windows Explorer

In SharePoint 2010 there is a nice little feature in a search site called a ‘search connector’ that enables you to run search queries from Windows Explorer. The search connector is displayed in your favourites drop down. Also, you can have more that one search connector if you have a number of different search sites configured in your SharePoint environment and name them accordingly e.g Intranet Search, Team Site Search etc.

Below are the details of how to set up a search connector:

1) Run a search query from your search site (Any OOTB search site) and click on the third icon below which will be at the top of your search results and slightly to the right.

2) Select ‘Add’ to the Add Search Connector prompt’.

Your Search icon will now appear in your Favourites menu in Windows Explorer. The icon can be renamed accordingly.

Setting up Incoming Mail to a List or Calendar in SharePoint 2010

In SharePoint 2010 there is the capability to enable incoming mail directly to a list or a calendar. This is a very useful feature for your SharePoint 2010 environment as it can enable your users to send documents directly to a list and send meeting requests to a shared SharePoint calendar. The benefits of sending meeting requests to a shared SharePoint calendar is that your users may have a requirement to show the meetings that are taking place for a certain project or office depending on how your site is being used.

Before I go through the steps in how to set this up, you need to be aware that the shared Calendar in SharePoint is not stored in Exchange. It is stored in SharePoint along with are any attachments that are sent to the shared Calendar.

Please note that I am not going to include the steps of creating the Directory Management Service which will enable SharePoint to be able to write to Active Directory. I will be covering creating the SMTP Service on the SharePoint servers, configuring the SMTP Send Connector in Exchange, configuring Incoming Mail in Central Administration and enabling incoming mail in your list/calendar. This will include ensuring that your mail address will be using a friendly name e.g ‘@mail.contoso.com’. If you wish to find out more about how to set up Directory Management Service with SharePoint then I can recommend SharePoint George’s blog.

Step 1: Configuring the SMTP Service on the SharePoint Server(s)

The first step is to set up the SMTP Service. This needs to be set up on your SharePoint servers that you have designated to handle the mail (eg web front end servers).

SharePoint 2010 requires the SMTP Service to be running, so you need to go to Features in Server Manager and install the SMTP Service on your Windows Server 2008 R2 server.

Once installed you will get the results screen to show a successful install.

Now open the IIS 6.0 Management Tools from Administrative Tools and configure the SMTP Service.

Go to the properties of the ‘SMTP Virtual Server’, click on the General tab and enable logging for troubleshooting. Then click on the access tab and enable “Anonymous access”. On the same tab, go to “Relay” and select the settings in the screenshot below.

Now go to messages and set the relevant settings for your organisation, an example is in the screenshot below. Then go to the Delivery tab and you can make the settings as you desire. I have gone for the default settings.

On the Security tab add the service account this is running the SharePoint 2010 timer service. This is the SharePoint service account that will be accessing the SMTP service to collect the mail. Ensure the same service account has read/write permissions to the mailroot folder (C:\InetPub\Mailroot). This will enable mail to be deleted once the timer service has collected the mail item.

Your SMTP Service will now look like the image below.

The domain name is automatically set as the ‘servername.domain’. The domain name is what is specified in the incoming email settings in Central Administration, so we need to ensure this is a friendly name for the users. In IIS 6.0 change the domain name to a friendly name e.g ‘mail.contoso.com’. This is important as the same name will need to be specified as the name of the Address Space in the SMTP Send Connector and the Incoming Email settings in Central Administration. So it needs to be a functional and user friendly name. Nb: The Domain Name does not need to be registered in DNS as a host record or MX record as Exchange will handle the address space.

Step 2: Configuring the SMTP connector to send mail to SharePoint.

The next stage is to enable mail to be routed to your SharePoint farm from Exchange. This is configured in the Exchange Management Console using the SMTP Send Connector. The SMTP Send Connector effectively routes mail to the relevant SharePoint Servers hosting the SMTP service (configured in step 1). Before you create the SMTP Send Connector, in DNS create an MX record for each of your SharePoint servers hosting the SMTP Service. The MX records needs to have the same naming convention for each SharePoint Server. An example is below:

MX Record ‘sharepointmail.contoso.com’ mapped to ‘sharepointwfe01.contoso.com’, with cost of ’10’

MX Record ‘sharepointmail.contoso.com’ mapped to ‘sharepointwfe02.contoso.com’, with cost of ’10’.

In Exchange Management Console, create a SMTP Send Connector with a useful name e.g ‘SharePoint 2010 Incoming Mail’.

In the address space, specify the address space name that you wish to use. E.g ‘mail.contoso.com’ with a cost of ’50’.

In the Network Settings, specify to use ‘Route Mail through the following smart hosts’ and add the name of the smart host which is the name of the MX Records. E.g ‘sharepointmail.contoso.com’.

In the Source Server section, add the relevant Mail servers that you wish to associate to the Send Connector.

Nb: If you have two or more servers and want to send mail to one server first, then specify the server1 with a cost of 10 and the server2 with a cost of 20. Mail will go to server1 first and then to server2 second if server1 is unavailable. This will need experience of the Exchange and AD as the cost is associated to the MX Record.

Step 3: Configuring SharePoint to use Incoming Mail

The penultimate stage is to configure SharePoint with the relevant incoming email settings and enable incoming mail on your document library or calendar. To do this, go to Central Administration, System Settings and Configure Incoming Mail. Turn the feature on and specify’ Advanced’ rather than ‘automatic’.

Specify ‘No’ for the Directory Management Service

Specify the mail server display name as ‘mail.contoso.com’. This is the same name as the SMTP Send Connector and the Domain Name in your SMTP Service so it is important to keep the naming convention consistent. This will be the name space for all your email address that you create for your document library or calendar.

Specify the drop folder as ‘C:\inetpub\mailroot\drop’. This is the folder where Exchange will drop the email\calendar item. The item will then be picked up from the SharePoint 2010 Timer Service and inserted into the relevant document library or calendar.

Step 4: Enable Incoming Mail on your Calendar or Document Library

The final stage is to now enable incoming mail on your calendar or document library. This can be done by going to the ‘List Settings’ of your doc library\calendar and under communications select ‘Incoming Email Settings’ and in the incoming email section type the name of your email address. For this example I have used ‘calendar@mail.contoso.com’. This will ensure that all meeting requests that are sent to this address will be added to the calendar.

If you would like to monitor the mail progress then I would recommend monitoring the drop folder for the first couple of test emails\appointments to ensure the item is dropped in that directory and collected by the timer service. Please note that the timer job ‘Microsoft SharePoint Foundation Incoming E-Mail’ checks the folder every couple of minutes to collect mail.

FBA Users automatically being made inactive from a Site Collection

An issue that had been causing no end of trouble in our MOSS 2007 extranet environment had been when FBA users appear to have been mysteriously deleted from a Site collection. This had been happening with no administration interference from the SharePoint site administrators and at times outside of when the Profile import synchronisation timer job had been run. This proved to be a challenge to further investigate and identify the cause of the issue.

The first step was to identify what was going on under the covers of SharePoint (i.e.in the content database). On a new occurrence of the issue I identified the users FBA login and ran the SQL query below against the userinfo table of the content database. The userinfo table contains the information on the SharePoint profiles that have been created in the database. A SharePoint profile is created the first time a user logs on to the site. To show the information for this instance I will use the name ‘d.newton’ as an example FBA login name.

select * from [ContentDatabaseName].[dbo].[UserInfo] where tp_login like ‘%d.newton%’

The SQL query above returned more than one user with the same tp_login name and more importantly the column tp_lsActive was set to ‘0’ for all the ‘d.newton’ logins. This effectively means that the user profile for ‘d.newton’ is inactive in SharePoint and no access is granted to the site collection. In addition, the column tp_deleted was populated with the same value as the users profile number. After comparing the duplicated tp_login names, I found that the names appeared to be the same but when you copy the login name from SQL to notepad I found that the duplicate tp_login names had a whitespace at the end. An example is below:

Tp_id = 551    tp_login = ‘membershipprovider:d.newton’    

Tp_id = 552    tp_login = ‘membershipprovider:d.newton ‘    (Single white space)

Tp_id = 553    tp_login= ‘membershipprovider:d.newton ‘    (double white space)

So how, when and why are the duplicated profiles created?? After running a number of custom reports I found that the duplicated profiles were being created by the SharePoint system account and after further testing against the custom extranet login.aspx it was discovered that the login page would accept a username with a whitespace at the end of the username ‘d.newton’ (Note: the password will need to be the same as the original username’s password). So what is happening in the userinfo table when you add the username with the whitespace? The original FBA user is made inactive with the tp_IsActive column being set to ‘0’ and a new profile is created with the same tp_login name with a whitespace and a new profile id but with no access to the relevant SharePoint site. From the user’s experience, they will not be able to see the relevant site. When the user logs in with the incorrect login, SharePoint detects that there is conflict in the users tp_login name where it is required to be distinct, so SharePoint automatically makes the original SharePoint profile inactive to resolve the conflict.

Armed with this new information, we now have the knowledge that the profile 551 with the username ‘d.newton’ with no whitespace is the correct profile and we can make this user active again from within site permissions of the relevant site.

It is also useful to know that we can now replicate the issue, which is good news! However, we need to find a resolution to stop this from re-occurring. The first task I would recommend undertaking is to change the extranet’s forms loginurl to the out of the box Microsoft login.aspx and re-test the same issue. It is very likely that you will find that the Microsoft’s default login.aspx page doesn’t have the same behaviour. The reason for this is that the Microsoft login.aspx page trims whitespace at the end of the username, which means that a username conflict or duplication never occurs. The steps to change the forms login page back to the OOTB login.aspx is detailed below:

1. On your web front end server, open the web.config of your extranet site and navigate to the ‘authentication mode’ section.

2. Change the forms loginurl to “/_layouts/login.aspx” (Details below)

<authentication mode=”Forms”>

<forms loginUrl=”/_layouts/Login.aspx” />

</authentication>

In our environment it turned out to be the custom login.aspx page that was causing the issue. Our custom login page was referencing code that was not trimming the username correctly and thus allowing the duplication of profiles to be created in the userinfo table. What is annoying is that the userinfo table accepts whitespaces in the tp_login column! If you encounter the same issue as us then I would recommend either resolving your code or create a new login.aspx page based on the Microsoft OOTB login.aspx.

Removing the Duplicated Users

As we now had a number of duplicated SharePoint profiles in the userinfo table, it would be useful if the profile synchronisation timer job would clean up the duplicates or inactive users. However, I have been advised by Microsoft that the profile synchronisation timer job will only delete Active Directory inactive users and not FBA inactive users. The only supported way to remove an FBA user from the userinfo table is via powershell and the article by Niklas Goude explains it very well.