Document ID not being generated

Problem

In a scenario where we have many document Content Types, we may want to enable Document ID - aka "Permanent Links" - which is a feature in SharePoint that assigns a specific ID for each document. That ID never changes even if the document is renamed or moved to another location.

In this scenario, the Document ID field was being generated for some libraries/content types, but not others, which is a surprisingly common problem.

Background

There are several stages we need to go through to enable use this mechanism.

Stage 1

We enable the feature in the Site Collection Features.

Stage 2

Under Site Collection Administration > Document ID Settings, we need to enable the assignment and pick a prefix.

The IDs use the format:
PREFIX-LISTID-LISTITEMID (ref)

Where:
PREFIX: the preconfigured prefix for the site collection, which can be any set of numbers and letters between 4-12 characters
LISTID: the property docid_msft_hier_listid of the list's rootfolder object where the document was initially uploaded to
LISTITEMID: the ID of the list item when it was first created

Stage 3

The next step is the most prone to confusion and difficulties.
In each web application there is a timer job called:

Document ID enable/disable job
Work item that propagates content type changes across all sites when the Document ID feature is reconfigured.

When this job is executed (either manually or automatically once per day), it will schedule a task.
That's keyword here is schedule, as in delay for later.
You would think it performed right away, but that is not the case.
Internal code shows that if the current site is "too big" the task will be done asynchronously some other time. And any site with more that 1 subsite is already too big for the purposes of this job.

When this job does run (there are some naughty ways to force it by messing with server time but they are not advised), a few things will happen.

The Document content type will be updated to contain the Document ID and Document ID Url fields.
The Document Set content type will be updated to contain the Document ID and Document ID Url fields.
The content type changes will be pushed to all child content types.

This can be followed through the ULS Viewer:

Document ID Enable: Starting Document CT Update; site='http://...'
Document ID Enable: Starting Document Set CT Update; site='http://...'

This is where most problems will appear, e.g.:

• The document ids don't generate after the job runs
• We are having trouble checking if the job ran, is running or will run
• Something went wrong when propagating the content types

We will have a look at issues and solutions later, but, in short, if everything succeeds, 4 event receivers will be added to each library to account for document events (adding, updating, checking in, checking out). The event receivers are added for each content type that exists in the library which is descendant of Document or Document Set.
This, plus the fact that we can simply drop a document and see if the field Document ID is populated, can tell us if everything is ok.

Stage 4

The last stage accounts for documents that already exist in libraries, and therefore will not be subject to the event receivers, but still need Document IDs.
Another timer job, now Document ID assignment job, is also scheduled for daily schedule and can be manually started, and this time run straight away without waiting times.

Document ID assignment job
Work item that assigns Document ID to all items in the site collection.

This should setup Document IDs for all documents in all libraries, and may even assign Document IDs for libraries which failed in step 3 due to outstanding issues (even receiver assignment != automated process assignment).

Analyzing problems and identifying solutions

So your document ids don't generate?
Let's try to fix that.

In an ideal setup, what happens is that 4 event receivers are added to each library, per content type, which generates a Document ID for each document that is uploaded or changed. So, we might have 4, 8, 12, etc. depending on how many content types we have in the list.

In some cases, the generation doesn't take place, but we might still see a Document ID, and that is because the previous Document ID value, already stored in the physical document (e.g. .docx) is taken. This of course gives us a wrong sense that the assignment is working, when it isn't. So be aware of false positives.

Event Receivers

While these could be programatically added to the library as a workaround solution, it doesn't fix the underlying problem.

ID	50774d11-b617-43c6-adcd-8fd32826b66d
NAME	Document ID Generator
ASSEMBLY	Microsoft.Office.DocumentManagement, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c
CLASS	Microsoft.Office.DocumentManagement.Internal.DocIdHandler
SEQUENCE	1000
SYNCHRONIZATION	Synchronous
EVENTTYPE	ItemAdded
ID	a7ab2cca-a3fd-47b0-ac6c-9c45d8bb0ad7
NAME	Document ID Generator
ASSEMBLY	Microsoft.Office.DocumentManagement, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c
CLASS	Microsoft.Office.DocumentManagement.Internal.DocIdHandler
SEQUENCE	1001
SYNCHRONIZATION	Synchronous
EVENTTYPE	ItemUpdated
ID	9936f9c8-ff85-4c23-8a03-21363e52f457
NAME	Document ID Generator
ASSEMBLY	Microsoft.Office.DocumentManagement, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c
CLASS	Microsoft.Office.DocumentManagement.Internal.DocIdHandler
SEQUENCE	1002
SYNCHRONIZATION	Synchronous
EVENTTYPE	ItemCheckedIn
ID	26ed21c2-7f59-449c-8b50-dbe764384cd9
NAME	Document ID Generator
ASSEMBLY	Microsoft.Office.DocumentManagement, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c
CLASS	Microsoft.Office.DocumentManagement.Internal.DocIdHandler
SEQUENCE	1003
SYNCHRONIZATION	Synchronous
EVENTTYPE	ItemUncheckedOut

Forcing the job

Setting aside a few less than usual issues, most of the times the problem will lie in the event receivers side. The enable timer job doesn't run successfully, causing the event receivers to not be added and preventing the generation for newly added documents.

The first thing you'll want to do is ensure the job runs again as soon as possible.
The best you can do here is schedule a new job and after that you must wait for about 1 hour depending on internal system configurations (and as mentioned, only changed by messing with server time).
If you open the Document ID settings and save without changing anything, you won't schedule a new job.
Therefore, you need to make a change and save, even if you disable, save and then enable and save the assignment.
Ideally, you want to see the a message on top:
"Configuration of the Document ID feature is scheduled to be completed by an automated process."

In the logs, you will also see a scheduled job being registered:

Document ID/IsSiteTooBig: Site 'http://...' - Too many webs, 31 > 1 (really!?)
Document ID Enable: ScheduleWorkitem for 'http://...': enable=True, schedule=True, overwrite=False

Read only or problematic content types

Since this operating involves updating the ootb Document content type and pushing all the changes to all content types, if there are read-only content types at the web or list level, or they are problematic, this can block the process.
My suggestion is to run a query and identify any custom web or content types that are read-only without any especial need to be so, and make them non read-only.
Generally speaking, content types should be updatable. For example, a problem I had was a missing document template for a content type. Even though everything seemed fine, when the content type was saved, an exception would occur.

New libraries or existing libraries?

A simple yet effective test that I suggest is to create a brand new document library and add the problematic content type to it.
Odds are, assuming it's a site content type, it received the updates from the document content type, and will therefore generate ids. If it does, it likely means that the existing list content type on the problematic list has a problem, or failed to receiver the update. If it does not work, maybe you have a more broad issue.

Logs logs logs

Although not always to the point, the ULS logs can be the best (or only) way to understand a bit more about that is happening.
Filtering by Category contains "Document ID" will ensure you only get exceptions related to the Document ID server, whether it's enough enabling or assigning IDs.

Conclusions

Obviously the Document ID feature is one of those mechanisms in SharePoint that look and sound simple in theory but can be a real headache, especially if we have large systems with many content types and solutions.
With this post I hope to go a bit in depth on what this feature is, what does it do in the background, and, more importantly, what are the most common issues and troubleshoot paths.

References

Web Application Document ID

Using Document Templates with Lists and Content Types in SharePoint

Internal details of Document ID feature activation or why Document ID is not added to content type after feature activation – Part 1

Internal details of Document ID feature activation or why Document ID is not added to content type after feature activation – Part 2

Document ID not being set in Libraries