Monday, September 7, 2020

Breaking down document locking in SharePoint

This is a topic I've been meaning to discuss in a while.

We open, change and close documents almost every day, and sometimes we collaborate with other people at the same time on those documents.

SharePoint brings all co-authoring features we need, straight out-of-the-box, but sometimes, there are problems.

While things are starting to look better on the cloud side, the concept of document locking is still very much up-to-date and understanding how it works and what to do when things go wrong is crucial.

In an article that has since disappeared the Microsoft Knowledge Base (but still available here), Microsoft explains it best:

When a document is opened by a client program, Windows SharePoint Services puts a write lock on the document on the server. The write lock times out after 10 minutes. Users cannot modify the document during the time when the document is locked.

In a scenario where the program that opens the document unexpectedly quits or crashes and you try to open the document again before the write lock times out, the message that you receive says that the document is locked by another user. This behavior occurs even though you are user who previously opened the document.

So, what do we know:

  • There's this thing called document lock
  • Locks get automatically created when documents are opened (in edit mode)
  • Locks go away when we close the document
  • If the document crashes, we won't have a chance to close it, so the lock will stay until the timeout expires
  • Locks aren't necessarily bad things. Word documents use "shared" locks, meaning that a user who enters edit mode doesn't prevent other users to also enter edit mode on the same document
  • In most versions of SharePoint, if an Excel document is edited through the Office Client, an "exclusive" lock is created, thus preventing co-authoring
  • There are properties are available which contain information about the lock:
    • vti_sourcecontrollockid - the ID of the lock which can be used to release the lock
    • vti_sourcecontrollocktype - the type of lock (0-Exclusive, 1-Shared, 2-None)
    • vti_sourcecontrollockexpiresvalue - the date (UTC) when the lock expires
    • vti_sourcecontroltimecheckedout - the date (UTC) when the lock began
    • vti_sourcecontrolcheckedoutby - the user who owns the lock

In the below diagram, I try to explain this very concept.

How to avoid document locks

The best way to avoid getting into a situation where you cannot change a document is to follow some best practices.

  • Save and close documents often
  • If possible, close all Office documents at the end of the working day
  • Ensure to have a stable connection to servers hosting the document (prefer cable over WiFi)
  • Avoid leaving documents open for long periods of time

What can be done to force release of the lock

There are a few ways to unlock a document, but most of them can only be performed by administrator, or be exposed in a user interface by a developer.

Option #1 - Server-side PowerShell

The most straight-forward way, if you're able to access the server, is to run SPFile.ReleaseLock.

This method will effectively release the lock from the document.

While you could simply run

$file = $web.GetFile($fileURL)


This may fail, as exclusive locks can only be released by their users.

So, to account for that, we could add a few extra lines as so:

Option #2 - Server-side component (C#)

If you want to expose this code in a page or web component, you could create a package and deploy this function as below.

Option #3 - HTML/Javascript page

Thanks to Peter Hoplar's post back in 2014 (!), we have a method in which we can allow users themselves to unlock documents that they have themselves locked.

There are some limitations, such as requiring Internet Explorer, but it is working today in SharePoint 2010/2013/2016/2019.

The idea is to create a page and add the script to the style library. Then add a Content Editor to the page and reference it to the URL of the script. The script will allow the user to input the Web URL and Document URL and allow to check the lock state and grant the option to force release the lock, if there is one.

Due to the length of the script, I'll just post a link to it here.


How To Use JavaScript To Delete Short-Term Locks From Documents Opened From SharePoint?

Can we unlock a ShortTerm lock via CSOM for SP2013

Behind the scenes: Opening a Document from a SharePoint 2010 Document Library



Excel file is locked for editing by another user (MS article on troubleshoot steps)

Saturday, December 21, 2019

Changing from feedburner to ifttt!

Yet another api deprecated!

Trying if-this-then-that to automatically share my blogger posts to Twitter!

Wednesday, December 18, 2019

#Office | Programatically uncheck the option to remove personal information


Office documents have a setting to remove personal information each time the document is saved.
If you have track changes and people commenting for example, their names get removed and replaced by a generic "Author" name.

This is sort of a security setting that it's nice to have, most in most cases it gets enabled unintentionally when using the Document Inspector to clean-up documents.

While we can simply open up the trust settings and disable this, there might be times when we need to go over a substantial amount of documents and manual work is no longer effective.

At the same time, this is a good proof of concept should we want to start automating other things on Office documents.

While OpenXML seems to be a clear successor, the Interop Objects still allows us today to interact and manipulate Word documents, in writable Visual Studio C# applications.
Where we can do something as easy as

doc.RemovePersonalInformation = false


Monday, December 16, 2019

#SharePoint | Harmful warning when copying/moving between webdav sites and libraries

Sometimes, when copying or moving documents between mapped libraries in SharePoint (whether they are network drives or just opened temporarily) we get the Windows Security warning:
"These files might be harmful to your computer".

This message is specially strange since the site already belongs to the Local Intranet Zone.

What we need to do is add the path "\\your-sharepoint-site" to the local intranet zone, save and log off and re-sign in.

Internally, this change will add a file entry to the zones in the registry.

Therefore, a way to perform this trough a .reg file, is:

Windows Registry Editor Version 5.00

[HKEY_CURRENT_USER\Software\Policies\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap\Domains\\*.subdomain]

After adding this key and signing off or restarting, the warning no longer appears.

Tuesday, June 4, 2019

#SharePoint | Open Office documents in any browser using Office URI scheme

One of the problems when shifting to a non-IE browser in SharePoint, is that immediately documents start downloading instead of opening in the browser.
While we can overcome that by tweaking the "Open in browser" library option, sometimes it is just very difficult to interact with Office since we cannot call "CoreInvoke" or functions such as createNewDocumentWithProgIDEx from the SharePoint OpenDocuments Class (OWSSUPP.DLL), often ending up with error message,

The document could not be opened for editing. A Microsoft SharePoint Foundation compatible application could not be found to edit the document.

One possible scenario could be:
How to open an instance of a .dotx Word Document Template in Google Chrome and allow a suggested save location?

Starting SharePoint 2013 and Office 2010 SP2, we can use URI schemes, which are basically URLs starting with "ms-X" which trigger the desktop application to open, and works in both IE and Chrome.



  • ofv (office file view)
  • ofe (office file edit)
  • nft (new from template)

Office Apps:

  • ms-word
  • ms-powerpoint
  • ms-excel
  • ms-visio
  • ms-access
  • ms-project
  • ms-publisher
  • ms-spd
  • ms-infopath


View a document (read-only)

Edit a document

Create a new instance of a document template

Example of a link in Google Chrome, opening a new document instance from a .dotx template and with a specified save location.
Internet Explorer opens the document immediately, and Chrome prompts for confirmation with an option to suppress it.

Office URI Schemes
Create a link to “New Document” template link

Thursday, May 16, 2019

#SharePoint | Document ID not being generated


In a scenario where we have many document Content Types, we may want to enable Document ID - aka "Permanent Links" - which is a feature in SharePoint that assigns a specific ID for each document. That ID never changes even if the document is renamed or moved to another location.

In this scenario, the Document ID field was being generated for some libraries/content types, but not others, which is a surprisingly common problem.


There are several stages we need to go through to enable use this mechanism.

Stage 1

We enable the feature in the Site Collection Features.

Stage 2

Under Site Collection Administration > Document ID Settings, we need to enable the assignment and pick a prefix.

The IDs use the format:

PREFIX: the preconfigured prefix for the site collection, which can be any set of numbers and letters between 4-12 characters
LISTID: the property docid_msft_hier_listid of the list's rootfolder object where the document was initially uploaded to
LISTITEMID: the ID of the list item when it was first created

Stage 3

The next step is the most prone to confusion and difficulties.
In each web application there is a timer job called:

Document ID enable/disable job
Work item that propagates content type changes across all sites when the Document ID feature is reconfigured.

When this job is executed (either manually or automatically once per day), it will schedule a task.
That's keyword here is schedule, as in delay for later.
You would think it performed right away, but that is not the case.
Internal code shows that if the current site is "too big" the task will be done asynchronously some other time. And any site with more that 1 subsite is already too big for the purposes of this job.

When this job does run (there are some naughty ways to force it by messing with server time but they are not advised), a few things will happen.

The Document content type will be updated to contain the Document ID and Document ID Url fields.
The Document Set content type will be updated to contain the Document ID and Document ID Url fields.
The content type changes will be pushed to all child content types.

This can be followed through the ULS Viewer:

Document ID Enable: Starting Document CT Update; site='http://...'
Document ID Enable: Starting Document Set CT Update; site='http://...'

This is where most problems will appear, e.g.:

  • The document ids don't generate after the job runs
  • We are having trouble checking if the job ran, is running or will run
  • Something went wrong when propagating the content types

We will have a look at issues and solutions later, but, in short, if everything succeeds, 4 event receivers will be added to each library to account for document events (adding, updating, checking in, checking out). The event receivers are added for each content type that exists in the library which is descendant of Document or Document Set.
This, plus the fact that we can simply drop a document and see if the field Document ID is populated, can tell us if everything is ok.

Stage 4

The last stage accounts for documents that already exist in libraries, and therefore will not be subject to the event receivers, but still need Document IDs.
Another timer job, now Document ID assignment job, is also scheduled for daily schedule and can be manually started, and this time run straight away without waiting times.

Document ID assignment job
Work item that assigns Document ID to all items in the site collection.

This should setup Document IDs for all documents in all libraries, and may even assign Document IDs for libraries which failed in step 3 due to outstanding issues (even receiver assignment != automated process assignment).

Analyzing problems and identifying solutions

So your document ids don't generate?
Let's try to fix that.

In an ideal setup, what happens is that 4 event receivers are added to each library, per content type, which generates a Document ID for each document that is uploaded or changed. So, we might have 4, 8, 12, etc. depending on how many content types we have in the list.

In some cases, the generation doesn't take place, but we might still see a Document ID, and that is because the previous Document ID value, already stored in the physical document (e.g. .docx) is taken. This of course gives us a wrong sense that the assignment is working, when it isn't. So be aware of false positives.

Event Receivers

While these could be programatically added to the library as a workaround solution, it doesn't fix the underlying problem.

NAMEDocument ID Generator
ASSEMBLYMicrosoft.Office.DocumentManagement, Version=, Culture=neutral, PublicKeyToken=71e9bce111e9429c
NAMEDocument ID Generator
ASSEMBLYMicrosoft.Office.DocumentManagement, Version=, Culture=neutral, PublicKeyToken=71e9bce111e9429c
NAMEDocument ID Generator
ASSEMBLYMicrosoft.Office.DocumentManagement, Version=, Culture=neutral, PublicKeyToken=71e9bce111e9429c
NAMEDocument ID Generator
ASSEMBLYMicrosoft.Office.DocumentManagement, Version=, Culture=neutral, PublicKeyToken=71e9bce111e9429c

Forcing the job

Setting aside a few less than usual issues, most of the times the problem will lie in the event receivers side. The enable timer job doesn't run successfully, causing the event receivers to not be added and preventing the generation for newly added documents.

The first thing you'll want to do is ensure the job runs again as soon as possible.
The best you can do here is schedule a new job and after that you must wait for about 1 hour depending on internal system configurations (and as mentioned, only changed by messing with server time).
If you open the Document ID settings and save without changing anything, you won't schedule a new job.
Therefore, you need to make a change and save, even if you disable, save and then enable and save the assignment.
Ideally, you want to see the a message on top:
"Configuration of the Document ID feature is scheduled to be completed by an automated process."

In the logs, you will also see a scheduled job being registered:

Document ID/IsSiteTooBig: Site 'http://...' - Too many webs, 31 > 1 (really!?)
Document ID Enable: ScheduleWorkitem for 'http://...': enable=True, schedule=True, overwrite=False

Read only or problematic content types

Since this operating involves updating the ootb Document content type and pushing all the changes to all content types, if there are read-only content types at the web or list level, or they are problematic, this can block the process.
My suggestion is to run a query and identify any custom web or content types that are read-only without any especial need to be so, and make them non read-only.
Generally speaking, content types should be updatable. For example, a problem I had was a missing document template for a content type. Even though everything seemed fine, when the content type was saved, an exception would occur.

New libraries or existing libraries?

A simple yet effective test that I suggest is to create a brand new document library and add the problematic content type to it.
Odds are, assuming it's a site content type, it received the updates from the document content type, and will therefore generate ids. If it does, it likely means that the existing list content type on the problematic list has a problem, or failed to receiver the update. If it does not work, maybe you have a more broad issue.

Logs logs logs

Although not always to the point, the ULS logs can be the best (or only) way to understand a bit more about that is happening.
Filtering by Category contains "Document ID" will ensure you only get exceptions related to the Document ID server, whether it's enough enabling or assigning IDs.


Obviously the Document ID feature is one of those mechanisms in SharePoint that look and sound simple in theory but can be a real headache, especially if we have large systems with many content types and solutions.
With this post I hope to go a bit in depth on what this feature is, what does it do in the background, and, more importantly, what are the most common issues and troubleshoot paths.


Web Application Document ID

Using Document Templates with Lists and Content Types in SharePoint

Internal details of Document ID feature activation or why Document ID is not added to content type after feature activation – Part 1

Internal details of Document ID feature activation or why Document ID is not added to content type after feature activation – Part 2

Document ID not being set in Libraries

Wednesday, March 27, 2019

SharePoint Designer | JavaScript changes causing SPD to hang

Recently I started to notice that SharePoint Designer has been behaving strangely.

  • When we click save on a JS file it goes on an eternal quest to try and save it
  • The simple opening of an .aspx file that contains javascript functions will take minutes and fail with "The server could not complete your request. For more specific information, click the Details button."

  • After it gives up, the error is displayed: "The server unexpectedly closed the connection."
  • After this (and if you have pending changes) the Designer will prompt to save the file to an alternate location, but not before throwing yet another error: "Could not find a Web server at 'SiteUrl'. Please check to make sure that the Web server name is valid and your proxy settings are set correctly. If you are sure that everything is correct, the Web server may be temporarily out of service."


It seems that some file contents sort of get blocked by an internal validation and after correcting these changes, the file will save instantaneously without any issues.

So, this seemingly complex infrastructure issue, turned out to be, in fact, the inability of SPD to deal with the additional space between "function" and its parenthesis!

To fix it, you can find and replace spaces before the parenthesis,

"function (" by "function("

and after the parenthesis,

") {" by "){"

After ensuring that all my script was replaced with these changes, opening, editing and saving files (either .js files or files with js content) became instantaneous.

Related threads:

Friday, October 19, 2018

SharePoint and the magic of query strings

One of typical requests from users is the ability to have a page open in a certain state, where things automagically appear and happen by themselves.

Throughout my travels, I have come by many of these magic strings of text that we put into the url bar and something amazing happens, some of these not even accessible - at least easily - through the user interface.

My goal with this post is to lay down some of these bad boys, who they are and what they do.

Old but gold, open the "Web Part Page Maintenance"

Query: ?contents=1

Result: opens a page with all the web part instances, effectively allowing us to manage (close, delete, etc.) web parts and very useful to troubleshoot problematic web parts

Go into the current page's edit mode

If - for whatever reason - you need to open a page in edit mode, you can add this one.

Query: ?ToolPaneView=2

Result: page opens automatically in edit mode

Open a view in quick edit mode

We can open up a view already in quick edit mode without users having to click List > Quick Edit.

Query: ?ShowInGrid=True

Result: opens a view in quick edit (datasheet) mode for editing items in place

Open a view with headings as filters

List view fields can be used to filter out items, but this is not shown by default.

Query: ?filter=1&filterclear=1

Result: filter will enable the fields as filters and filterclear will ensure that the filters persist through out the session

Filtering views with advanced query strings

This is by far the most helpful and flexible query string mechanism available in list views. So much so that I will need to have sub-categories for them:

Filtering a single field with a single value

Query: ?FilterField1=Country&FilterValue1=Portugal

Result: returns results with Portugal as country

Filtering multiple fields and with single values

Query: ?FilterField1=Country&FilterValue1=Portugal&FilterValue2=City&FilterValue2=Lisbon

Result: returns results with Portugal as Country and Lisbon as City

Filtering a single field with multiple possible values

Query: ?FilterName=Country&FilterMultiValue=Portugal;United States of America

Result: returns results where the country is either Portugal or USA

Filtering a single field with multiple possible values using wildcards

Query: ?FilterName=Country&FilterMultiValue=*ortuga*;*states of ameri*

Result: returns results where the country is either Portugal or USA, while specifying only a subset of the string we want to search for

Now let's put them all together

Query: ?Filter=1&FilterField1=Country&FilterValue1=Portugal&FilterValue2=City&FilterValue2=Lisbon&FilterName=Subject&FilterMultiValue=*human resources*;*procurement*

Result: This url will enable headings as filters, it will query results with Portugal as Country and Lisbon as City, and it will also look for results where the Subject contains either human resources or procurement

Further reading:

Connect a Filter Web Part to a List View Web Part on a classic page

Wednesday, July 4, 2018

Solved co-authoring failure for Office 2016 users causing locked documents

Users keep getting locked out of a document.
A user is able to make changes, but then everyone else isn't.

Among the errors:

"We're having trouble connecting to the server. If this keeps happening, contact your help desk."

"Upload failed. This file is locked for editing by ..."

Fiddler displays a 404 cobalt error:


Apparently, there is a key in the registry that if non-existent, will prevent co-authoring!

There is a hotfix, the October 13, 2015, update for Office 2016 (KB2920679).

Or simply add the following registry key, and then restart Word:

Windows Registry Editor Version 5.00


(this could be saved as a .reg and executed)


Friday, January 12, 2018

Easily access linked files in Word/Office

With the constant changes in the Office User Interface, it's gets increasingly difficult to find things.
One feature that I am particularly annoyed about is the linked files. Whenever a user has a document with links to external documents, I never seem to find how to manage them.

As usual, the complete list of commands comes to the rescue.
Simply by going to File > Options > Quick Access Toolbar, we can see a list with commands, and if we pick "All commands", we will now find "Edit Links to Files" and bring it to the right for a quick shortcut on the top left.

Tuesday, August 29, 2017

Enable SharePoint Designer & code snippets in Office 365

I recently had to setup a new Office 365 tenant.

One of the first things I noticed was that SharePoint Designer was disabled, and could not find an option anywhere to enabled it.

Another problem caused by this is trying to add code snippets to pages, ending up with the message:

"This HTML cannot be inserted because you don't have access to add scriptable Web Parts in this site. Please contact your administrator if you think you should have rights to do so."

After searching for a while I found a solution using SPO.

1. Download and install SharePoint Online Management Shell

2. Load up the newly installed SharePoint Online Management Shell console

3. Run the following code snippet (after setting up your tenant details)

Wednesday, January 4, 2017

Using bookmarklets to instantly export webparts and other abilities

I've recently bumped into a blog post with mention of a tool that would convert a piece of JavaScript (that you would otherwise store in the server on in the browser through user scripts) to a bookmark!

This way, whenever you want to run a big chunk of code in your browser that would affect the currently opened website, you can simply push a button and off you go.

This is known as a "bookmarklet".

"A bookmarklet is a bookmark stored in a web browser that contains JavaScript commands that add new features to the browser."

The concept is simple:

  1. Develop a script that would add some function to the existing website (but will be executed on demand instead)
  2. Go to the Bookmarkleter page to convert the script
  3. Drag-and-drop the bookmark (or create it yourself) with the new "link"

Examples of SharePoint bookmarklets:

My very own "SPDevTools"

This script will inject several buttons near the SharePoint ribbon with several functions, such as:

  • Display the file:// url for the current location (so that it can be quickly opened in explorer in non-IE browsers)
  • Display the site template ID
  • Refresh the page (no postback)
  • Get a link to the current page (handy in iframes)
  • Sign in as a different user (feature removed in SP2013)

Anatoly Mironov's Webpart Exporter

A long way has come since we used other methods to export webparts, such as:

  • Opening the URL: /_vti_bin/exportwp.aspx?pageurl=http://mysite/SitePages/Home.aspx&guidstring=WEBPARTIDGUID
  • manually inject AllowExport="true" in the html of the WebpartID element while in edit mode, which brings back the Export contextual menu option

This new method, bookmarlet supported of course, will actually find all the webparts in the current page and allow you to export every single one of them with a simple click.

Dan Saedén's Web Properties Updater

This other handy piece of code will display all the current Web's properties, and allow you to add, edit or remove any of them very easily.

There is also a great Google Chrome plugin - "ChromeSPEditor" - that can manage UserCustomActions and Web Properties.


Wednesday, August 10, 2016

Working around X-Frame-Options for iframes

The problem:

One of the first things we noticed after migrating to SharePoint 2013 was that our iframes have stopped working, giving the error:

"Refused to display 'http://contoso/pages/home.aspx' in a frame because it set 'X-Frame-Options' to 'SAMEORIGIN'."

Quickly we could see that this is in fact a security mechanism to prevent click-jacking and others:

"The X-Frame-Options HTTP response header can be used to indicate whether or not a browser should be allowed to render a page in a frame or iframe. Sites can use this to avoid Clickjacking attacks, by ensuring that their content is not embedded into other sites."

Soon we realized that, starting SharePoint 2013, an HTTP Module is adding the header "X-Frame-Options" by default, with the value "SAMEORIGIN".

This is effectively preventing the rendering of pages outside the scope of the current web application.

The (attempted) solution:

So what to do?
At first we though we could fiddle around with the IIS by adding a new header.

But since the old header is still there, we end up getting two header entries.

The (actual) solution(s):

After some more research, we finally found some options:

Ventigrate has a WSP farm solution in CodePlex that "will prevent SharePoint from trying to inject the header in the first place". It gets the job done, at the cost of having to deploy yet another (Full Trust) solution into the farm

Use AllowFraming in a Master Page or specific page/app that we want to enable embedding for. In cases where this actually works, this can potentially be one of the best workarounds.

<WebPartPages:AllowFraming ID="AllowFraming" runat="server" />

Install URL Rewrite in IIS and setup a new rule to remove the header (as proposed here)

For servers being load-balanced, there is typically an easier way, which is to setup in the configuration of the load balancer that this header should not be used (F5, etc.)

Needless to say, take all this with a grain of salt, and understand what X-Frame-Options actually is, and that removing it can potentially open up your website for malicious intents.

The aftermath:

The header issue is now worked out (hopefully), but we may not exactly be in "the clear" just yet.

In my case I was iframing a SharePoint 2013 page inside a SharePoint 2010 site. So I discovered the hard way that "it's not possible to trigger a different rendering mode in a child iframe in IE9" as mentioned by this StackOverflow post and documented here.

So now I could use iframes, but my SharePoint 2013 frame was rendering not in IE10 but in the same mode as the main site (IE8).

To further extend my problem, the jQuery version being used in the iframed site was v2, which BTW is NOT supported in IE8.

There are many ways around this, like having 2010-mode site collections in 2013 or conditionally loading the necessary jQuery version, but I just wanted to alert for this particularity of IE9+ and iframes.


  • SharePoint 2013 introduces X-Frame-Options header which will prevent the embedding of iframes to external websites
  • Simply adding the header in IIS is not enough of a solution in order to work around this (potentially works outside the SharePoint ecosystem)
  • AllowFraming is a great way of supporting iframes on specific pages or sites
  • Internet Explorer 9+ now prevents the embedding of an iframe in a different rendering mode than it's parent (in another words, it uses the parent's rending mode)
  • jQuery v2 is not supported in IE8 (or any IE version in a page with IE8 rendering mode)




Monday, March 14, 2016

#SharePoint #2010 | Caveats when upgrading master page to IE9 (with solutions)

One of the biggest issues with branding is our "friend", Internet Explorer.
When we talk about SharePoint, specially SharePoint 2010, we are typically bound to one version of Internet Explorer, where everything works great, but as soon as we move to another version, all hell breaks loose.

In practice, this happens when we change this line,

< meta http-equiv="X-UA-Compatible" content="IE=8" />

to something like

< meta http-equiv="X-UA-Compatible" content="IE=9" />

Why would you do this?
Well, among other things, you get access to CSS3. Otherwise, even if you have IE9+ installed, you will not get CSS3 because the browser will be rendering in IE8 mode. Many other limitations of IE8 will apply as well.

Should you do this?
Ideally, no. This raises issues as soon as this is enabled and I'm sure not all issues have been found.
However, in a small and/or public facing application, where only a few areas, easily testable, are available, this can be achieved using the code fixes provided below.

What happens?

  • Rich text editors break (form.js)
    • IE8 handles "document.frame" differently (replace curly brackets with square)
  • People pickers break (entityeditor.js)
    • 'data' is not properly serialized


The solution, at least for these two javascript errors, is to override the ootb functions that are breaking.

So we will create a form.ie9fix.js and a entityeditor.ie9fix.js and reference them on our master page, in the < head > area.






Monday, June 29, 2015

#SharePoint | Value cannot be null, parameter name: g


Moving documents in SharePoint can be a difficult task at times.
One of the biggest challenges is keeping previous document versions.
In some cases, it works if you simply move the documents from one place to the other, while using explorer views (nb: only works for MOVE operation, not for COPY).

In other cases, explorer view, site content and structure and even 3rd party tools will fail to accomplish this.
One of the reasons - and the one explained in this post - is a problematic field schema. More specifically, the lack of a certain attribute on a certain type of field, that will prevent the operation from running successfully, resulting in screens such as the ones below.

        http://.../Demo Document 001.docx: Value cannot be null. Parameter name: g


In this case, it appears that we have a field with type "User", which does NOT have the attribute "List" set to "UserInfo", which seems to be a must.
Although we might not have problems adding and editing documents without this attribute, we will definitely have problems with migrations, import/export, etc., in the future.

To solve this, I have a script with two steps.

1) identify fields of type "User" in the document library

The script outputs four entries or fields. three of them are internal fields, and one is our custom field.

function PrintFields($web, $listName)
 $lib = $web.Lists.TryGetList($listName)
 if($lib -eq $null)
  throw "List $listName does not exist!"
 Write-Warning "Displaying User fields for list $listName"
 write-host ""
 $lib.Fields | where-object {$_.Type -eq "User"} | select InternalName, Type, SchemaXml | format-list 

2) Add the missing attribute (SPField.SetAttribute)

function EditFieldXML($lib, $targetField)
  $fld = $lib.Fields.GetFieldByInternalName($targetField)
  write-warning "Field $targetField does not exist!"
  write-host ""
 Write-Host Current field definition $fld.SchemaXml
 write-host $newLine
 $fld.Sealed = $false
 $fld.PushChangesToLists = $true
 $schema = [xml]$fld.SchemaXml
 $fld.SchemaXml = $schema.PSBase.OuterXml
 $fld.PushChangesToLists = $true
 Write-Host New Field definition $fld.SchemaXml

Complete source code here

Related links

Friday, April 3, 2015

Mobile development opportunities

I was recently asked about the differences between the major mobile development markets what opportunities are there for developers.
After some research to fill-in the gaps, I have came up with a comparison table.

mobile development plan
storepriceprice for companiesdurationmobile apps (2014)languageref
google play$25$25one-time1.3MJavahere
windows store$19$99one-time300KC#here
apple store$99$1991 year1.2MObjective-C / Swifthere

What this table tells us, is that we have right away some big differences. If you are a typical Microsoft-oriented developer, you may opt with the windows store. You will get one of the best access fees (only $19) and you will be able to develop using the frameworks that you are used to.
Being a still growing market, there is a larger chance to create new apps that haven't been thought up before.

If you want to enter the Apple ecosystem, many bloggers recommend that you start with Objective-C. But you should know that you have to pay a minimum of $99 every year, and if you have a published app, it will be removed from the store as soon as you stop paying. Apple is also known for having one of the most rough validation procedures, where most "silly" apps will be blocked from entering the store. One could argue that there is a higher risk due to these prices, but this can also mean higher return of investment if you have a good product.

The Android / Play store is known for being more permissive. This can be an advantage and disadvantage at the same time. This store has cheap, life-time access, and the application approvals don't seem to be very thorough, so although it might be good for you, often we see news of viruses or hacks making their way into the store. But if Java is your comfort zone, creating a nice app with a good monetization plan could be the way to go.

How many apps per store?
A study has found that in 2014, the Google Play store was the one with the most apps, with around 1.3 million applications!

Is there a way to develop once and for all?
Yes, PhoneGap and Xamarin are just a few of the many options.
Check this stackoverflow comparison for more.

How can I make money?
Most markets will keep a 30% commission and let you keep 70%.
Here are a few monetization options:
  • Create apps with advertisements
  • Create one free (limited/adds) and one paid
  • Create a trial app (payment required to continue using)
  • Implement In-App purchases (e.g. "Gold coins")

Tuesday, March 31, 2015

Ultimate Debugging Enabler for SharePoint

On every new environment we often need to enabling custom errors, debug mode and other settings that will make your development environment spit-up all the exception in a human readable way.

What we don't usually we know is that, not only can be disable CustomErrors for SharePoint sites - just as we would for any .NET application - but there are also other things that we may need to take into considering in order to end up seeing something different than this:

Although we can see many different methods describing what, where and how to prepare an environment for debugging, the truth is that this procedure is not easy or straight-forward, and there doesn't seem to be a single place with all this information.
- different methods (CallStack, Debug, CustomErrors, AllowPageLevelTrace, Trace)
- different locations (IIS web.config, Template\Layouts, Template\Admin)

My proposal is to run a simple PowerShell script that will easily enable all the debugging attributes on all the necessary web.config locations, without harming the configuration files, and being able to revert the process to an original state.
As an added perk, a backup of the configuration file is made on every script execution (using the same format that regular backups use).
In case the debug mode was not changed, there will be no changes, to avoid having the pool recycle unnecessarily.



Monday, March 16, 2015

5 Conventions that can make our life easier

All my life I longed to work in a place were a list of rules were glued to the wall, saying what you can and cannot do, should or should not not, in your code. Doesn't have to be anything fancy like design patterns, life-cycle management, or unit testing, but the basic things that everyone should know, and keeping them in mind so they don't forget.

Because of the professional path (I guess), I was never told how to create my variables, what to name my methods, and to what extend I should use comments.
More and more I think that conventions are great because they allow everyone to do things in similar ways, ways that are recognized as good practice, and that will keep individuals from straying away and doing things "their way". Conventions are also good simply because we can create an organization-based list of rules so that everyone in the team speaks the same language and takes into consideration the same challenges (defaults for creating classes, pages, libraries, etc.).

With this in mind, I am going to lay down a few of this simple practices that help my everyday.

1. Write in English
This one goes out to those who, like me, don't speak English as their natural language.

Writing in English is great because, first, it is a convention that if everyone follows, everyone will be able to read each one's code. Other than that, well, it will be easier for you to sell your code if you want to, or just have someone read it to help you. In a globalized world, this is becoming a must, tough we sometimes forget or get lazy about it.

And yes, this applies to code comments as well!

2. Follow Naming Conventions
There are good generic conventions for variables, methods, classes and constants.
For each individual programming language, there are also some specifications.
E.g. for .NET, lowerCamelCase for variables and UpperCamelCase for methods:

float myWidth;


class Raster;


Namespaces can also follow a convention, e.g.

For PowerShell functions, it makes more sense to use the [Verb]-[Noun] convention for methods.
E.g. Add-DocumentToLibrary

If we run "get-command", we can see some common verbs, such as Get, Test, Update, Set, Start, Stop, New, and many others. Some may like his verb/noun signature just enough to use it on other languages.

3. Avoid spaces
Coming from an MS-DOS background, I am still used to keeping my folders with no spaces and low amount of characters. It is just easier for a script to go there without having strange characters in it, or requiring the usage of commas. Specially for the web, spaces are a huge no-no.

Some examples:

- Folders "\\computer\c$\file share"
\\computer\c$\file share VS \\computer\c$\fileShare

When you try to copy a link so you can send it to someone, the first link will break due to the space

- Websites and pages
http://site/site 1 vs http://site/site1

In some scenarios, the first url will have its space replaced by a "%20", which basically looks ugly.
In some other cases, the link may break as it did for the previous example

My advise is don't use spaces for websites, folders, or libraries.
For pages or views, sometimes it can make sense to separate two or more words. For those cases, use the hifen , "-", not the underscore, "_".
By doing this, you are also potentially improving SEO.

- SharePoint columns
If you worked with SharePoint, you know how bad can your columns' internal names look like
e.g. "Procedure Name" --> "Procedure_x0020_Name" or "Procedure%5Fx0020%5FName" (web)

Overall in SharePoint, it is a good practice to create something with a friendly name first, and then go back and change it (websites, lists, pages, views, columns).
It is also preferable to query "internal names" or "guids" instead of "display names", since the later are susceptible to change.

4. Refactor
It is common to hear, "if your code is more than one page long, it is too big".
I am not a fan of absolutes, but the idea is that if your method is becoming too big, than it probably does more than what it is selling.
It is usually good to split a function into several different functions in order to improve things such as:
- Reusability
- Unit testing
- Readability

5. Leverage helper methods
Common functions should go to a special place, where they can be easily reused.
Quite often you will end up doing repetitive code so you will want to refactor those "DownloadDocument" or "GetItem" methods to keep them generic enough so they can be reused, and at the same time, being able to copy them to other projects with little or no changes at all.

What many people are not aware of is the danger or creating static methods in static classes.
This is usually done so we can call a method without an instance of a class, but it goes down to what the method does. If you are relying on shared data, you may be in trouble.

Thursday, February 19, 2015

#Umbraco | Creating a contact form for dummies

As a .NET-based, MVC CMS, Umbraco sounds good right from the start.
However, for beginners, the easiest things may be hard to understand, specially if we want to get our hands dirty.

The post will cover the procedure to create a simple contact form in Umbraco, but also how to really integrate it with the platform, for those just starting to know the CMS.

1 - Provision your Umbraco environment
(check out my previous post, Installing Umbraco for the first time)

2 - Really understand the structure concept

The Model-View-Controller pattern is just one of the things you will have to get comfortable with.
The main challenge is to undertand how Umbraco structures its information, specially regarding pages, templates, view and master pages.

3 - Create a model
The model is the "Data Model" were we will need to setup the data structures and types to be used by our interacting elements.

Model (Models\ContactModel.cs):

4 - Create a controller
A controller is a piece of code that will handle the server side code.
The controller will reference the model to understand data structures, and will be referenced by views to create a link between the front end elements and the code behind.

Controller (Controllers\ContactController.cs):

5 - Create a partial view (UI) A partial view is an independent element, that displays html, css and others, while communicating with a controller for the execution of operations. It can be created by opening the /umbraco admin area and going to Settings > Partial Views and hit "Create".
Partial View (Views\Partials\ContactForm.cshtml):

6 - Create a document type & template (UI) If you have created your model, controller and partial view, you have almost everything you need, except there is no interaction with Umbraco just yet. You have functionality, but not structured functionality, like navigation and pages that show your features. This is what is really new when working with Umbraco. You will have to decide weather to create a specific document type for your future page, or use an existing document type, and just create a new template. From Umbraco: "Step 1 of any site is to create a "Document Type" – after a few installations you’ll become familiar with this terminology but at the start it might be a little bit bewildering. A Document Type is a data container in Umbraco where you can add data fields / attributes where the editor user can input data and Umbraco can use it to output it in the relevant part of a "template" (more on these later).  " The template is a .cshtml file, just as the partial view, except that the template will define the master page and render the view, usually with Html.RenderPartial(viewPath) Umbraco on templates. Procedure:
  • Next to "Master", create a new Document Type "Contact Form"
  • Agree to create a new Template with the same name
  • In "Home" Document Type, check "Contact Form" in "Allowed templates"
  • In "Home" Document Type, check "Contact Form" in "Allowed child node types"
  • In Templates, set the Master template of "Contact Form" to "Layout"
By now you should be able to create pages based on the contact form template, either in "Content" or under "Home". Template (Views/ContactForm.cshtml):

7 - Create a content page (UI) The page is, obviously, the endpoint for interaction with the user. You will setup a friendly url, a page title, and the body contents, if any. A page is associated with a specific document type, and the templates available for that document type. You can create a page in the "Content" area, assuming you have already setup the proper document types and templates. 

Quick and Easy Umbraco 7 Razor Contact Form

Umbraco 7 and MVC - Contact Form

If you are looking for a full deliverable, you can find it here.
This is a fully documented plugin that also contains a precompiled dll that can be included in your projects.

Installing #Umbraco for the first time


Installing a new CMS can be tricky, and Umbraco is no different.
There are many ways, but even the easiest path can be a bumpy road.
For this reason, I like to do things manually, and define a process that really works.


1 - Setup SQL Server and SQL Server Management Studio

Umbraco will install a SQL Compact Database by default, but in my opinion it is better to start working with SQL Server from the very beginning.
While opening the site for the first time (Step 7), there will be a setting "Customize" to allow selecting an SQL Server and instance, as opposed to the default compact database in the AppData folder.

I personally installed, from web platform installer:
"SQL Server 2008 R2 Management Studio Express with SP1" (management studio)
"SQL Server Express 2008 R2 Service Pack 2" (sql server instance)

If you want to use SQL Server, now it is a good time to use Management Studio and create a new database.

2 - Setup the Visual Studio Solution / Project

Open up Visual Studio (make sure you do it as Administrator to avoid future problems) and create a new project, ASP.NET Web Application, using .NET Framework 4.5.

3 - Add Umbraco to the project

Go to Tools > NuGet Package Manager > Package Manager Console and type:

Install-Package UmbracoCMS

4 - Setup a proxy, if you need one

A proxy can be defined in the web.config file, adding the below:

5 - Deactivate LoaderLock to avoid "ProfileDB.cs" request error

Debug > Exceptions > Managed Debugging Assistants > uncheck "LoaderLock"


6 - Compile the solution

Make sure the solution is fully working and that a new build is generated.
If you have classes in the App_Code folder, they might need to be set with "Build Action" to "Compile", or move them altogether.

7 - Deploy Umbraco using SQL Server

- Run (F5) or add your path to an IIS application to start Umbraco
- You will need to click "Customize" and then setup the SQL Server settings
- Finally, you will need to select a template

8 - Configure the project as an MVC application

If you are trying to right-click "Models" or "Controllers", you will not get the usual "Add Controller" or "Add Model" contextual options.

The workaround for this is to add the guid,  "{E3E379DF-F4C6-4180-9B81-6769533ABE47};"
to your .csproj file, inside the following tags:


However, you might want to consider creating a different project to place these elements.

9 - Delete the "Install" folder

Umbraco recommends that once you have installed the application, delete the install folder to prevent problems in the future.

Known issues

I have experienced some issues myself, some of them resolved with running one or more of the following commands, but for those making their first instances of Umbraco, I definitely recommend backing up the site and database often, to be able to rollback to a working state.

Install-Package -Id  Microsoft.AspNet.WebHelpers
Install-Package Microsoft.AspNet.Mvc -Version 5.0.0

Other resources:
Setting up Umbraco with Visual Studio
Install Umbraco manually


Version 7.2.2 was throwing a "missing plugins folder" error when choosing certain site templates.
This can be fixed by creating a folder "App_Plugins" in the Umbraco root folder.

There you go!