Wednesday, September 14, 2016

SharePoint Automation: unified Powershell script to copy a page or a document in SharePoint Online and "on-premises"

This simple Powershell script automates copying a page with all web parts on it in the scope of the same sub-site. It can also be used for copying regular files situated inside or outside document libraries as well as just for deleting redundant pages and files (through the setting $onlyRenoveDestinationFile = $true).

This script can be useful, for example, when you create an Enterprise Search Center and want to add more pages to your search navigation. The ordinary routine process would include exporting all web-parts from an existing page, creating new pages and deploying the exported web-parts to them. However, this script provides more elegant and lightweight alternative for similar operations.

The main script

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
param ( 
  $webUrl = "news",                      # Site collection relative URL of the sub-site that contains source file to copy (use $null or "" for the root site).
  $sourceFile = "home.aspx",             # Name or site relative URL of the source file to copy (documents/file1.docx, set $usePublishingPageUrl=$false).
  $destinationFile = "newpage.aspx",     # Name or site relative URL of the destination file to copy the source one to (documents/file2.docx, set $usePublishingPageUrl=$false).
  $usePublishingPageUrl = $true,         # A flag that instructs to search for the source and the destination file URLs in the pages library of the sub-site.
  $overwrite = $true,                    # A flag that instructs to overwrite the destination file in the case it already exists.
  $publish = $true,                      # A flag that instructs to publish major version of the destination file after copying (concerns SharePoint Online).
  $approve = $true,                      # A flag that instructs to approve the destination file after copying (concerns SharePoint Online).
  $onlyRenoveDestinationFile = $false    # A flag that instructs only to remove the destination file without copying the source.
)
 
# Step 1. Connect to SharePoint environment amd get a client context.
write-host
write-host "Connecting to SharePoint environment..." -NoNewLine
. $PSScriptRoot\__LoadContext.ps1
if( $context -eq $null ) {
  write-host "Connection failed. Further execution is impossible."
  return
}
write-host "Done"

# Step 2. Open a sub-site, if required.
$serviceName = $null
$sourceUrl = $null
$destinationUrl = $null
$targetWeb = $null
if( ![string]::IsNullOrEmpty($webUrl) ) {
  $targetWeb = $context.Site.OpenWeb($webUrl)
  $context.Load($targetWeb)
  $context.ExecuteQuery()
} else {
  $targetWeb = $context.Site.RootWeb
  $serviceName = $context.Site.RootWeb.ServerRelativeUrl
}
$serviceName = $targetWeb.ServerRelativeUrl
if( $usePublishingPageUrl ) {
  $context.Load($targetWeb.Lists)
  $context.ExecuteQuery()
  $targetWeb.Lists | ? {$_.BaseTemplate -eq 850} | % {
    $context.Load($_.RootFolder)
    $context.ExecuteQuery()
    if( $sourceUrl -eq $null ) {
      $sourceUrl = [string]::Format("{0}/{1}", $_.RootFolder.Name, $sourceFile)
    }
    if( $destinationUrl -eq $null ) {
      $destinationUrl = [string]::Format("{0}/{1}", $_.RootFolder.Name, $destinationFile)
    }
  }
} else {
  $sourceUrl = $sourceFile
  $destinationUrl = $destinationFile
}
write-host
$message = "Processing the URL " + $targetWeb.Url
write-host $message
write-host

# Step 3. Delete the destination file if it already exists.
if( $onlyRenoveDestinationFile ) {
  $requestUrl = $targetWeb.Url + "/_vti_bin/_vti_aut/author.dll"
  $request = CreateWebRequest $requestUrl
  $request.Method = "POST"
  $request.ContentType = "application/x-www-form-urlencoded"
  $request.Headers["X-Vermeer-Content-Type"] = "application/x-www-form-urlencoded"

  $requestData = new-object "System.Collections.Generic.Dictionary[[string],[string]]"
  $requestData.Add("method", "remove documents:" + $context.ServerVersion)
  $requestData.Add("service_name", $serviceName)
  $requestData.Add("url_list", "[" + $destinationUrl + "]")

  $errorMessage = "(?i)<ul>\s+?<li>status=\d+.+<li>msg=([^<]+)<"
  $successMessage = "(?i)<body>\s+?<p>method=[^<]+\s+?<p>message=([^<]+)<.+<li>document_name=([^<]+)<"

  $content = ExecuteWebRequest $request $requestData ([ref]$errorMessage) ([ref]$successMessage)
  if( $errorMessage.Length -gt 0 ) {
    write-host $errorMessage -ForegroundColor Yellow
  }
  if( $successMessage.Length -gt 0 ) {
    write-host $successMessage -ForegroundColor Green
  }

  return
}

# Step 4. Copy the source page with all web parts to a new one.
$put_options = "createdir,migrationsemantics"
if( $overwrite ) {
  $put_options = "overwrite," + $put_options
}

$requestData = new-object "System.Collections.Generic.Dictionary[[string],[string]]"
$requestData.Add("method", "move document:" + $context.ServerVersion)
$requestData.Add("service_name", $serviceName)
$requestData.Add("oldUrl", $sourceUrl)
$requestData.Add("newUrl", $destinationUrl)
$requestData.Add("url_list", "[]")
$requestData.Add("rename_option", "findbacklinks")
$requestData.Add("put_option", $put_options)
$requestData.Add("docopy", "true")

$requestUrl = $targetWeb.Url + "/_vti_bin/_vti_aut/author.dll"
$request = CreateWebRequest $requestUrl
$request.Method = "POST"
$request.ContentType = "application/x-www-form-urlencoded"
$request.Headers["X-Vermeer-Content-Type"] = "application/x-www-form-urlencoded"

$errorMessage = "(?i)<ul>\s+?<li>status=\d+.+<li>msg=([^<]+)<.+</ul>\s+?<p>message=([^<]+)<"
$successMessage = "(?i)<body>\s+?<p>method=[^<]+\s+?<p>message=([^<]+)<"

$content = ExecuteWebRequest $request $requestData ([ref]$errorMessage) ([ref]$successMessage)
if( $errorMessage.Length -gt 0 ) {
  write-host $errorMessage -ForegroundColor Yellow
}
if( $successMessage.Length -gt 0 ) {
  write-host $successMessage -ForegroundColor Green
}

# Step 5. Publish and approve the destination file, if required.
if( !($errorMessage.Length -gt 0) ) {
  $fileUrl = $targetWeb.ServerRelativeUrl.TrimEnd('/') + '/' + $destinationUrl
  $file = $targetWeb.GetFileByServerRelativeUrl($fileUrl)
  $list = $file.ListItemAllFields.ParentList
  $context.Load($file)
  $context.Load($list)
  $context.ExecuteQuery()
  if( $? -eq $false ) {
    return $null
  }

  $enableMinorVersions = $list.EnableMinorVersions
  $enableModeration = $list.EnableModeration
  $processed = $false
  if( $publish -and $enableMinorVersions -and $file.MinorVersion -gt 0 ) {
    $file.Publish("")
    $context.ExecuteQuery()
    write-host " Published" -NoNewLine
    $processed = $true
  }
  if( $approve -and $enableModeration ) {
    if( $processed ) {
      write-host "," -NoNewLine
    }
    $file.Approve("")
    $context.ExecuteQuery()
    write-host " Approved" -NoNewLine
    $processed = $true
  }
  if( $processed ) {
    write-host
    write-host
  }
}

Limitations

The script shown above works in the scope of a single sub-site (web); it cannot copy pages or files between different sub-sites of site collections.

The current version of SharePoint Online does not seem to support copying file versions like it was supported in the past by SharePoint 2013 and 2010 "on-premises".

Context loader

The script below should have the name __LoadContext.ps1. It is loaded and used by the main script shown above. Also change values of the settings in the header of this script to comply with your SharePoint environment.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
<#
Unified automation script that loads a client context of SharePoint Online and "on-premises" (2013, 2016).
- This is a common script basic functions of which are loaded and executed from other scripts.
- The latest version: http://paulborisov.blogspot.fi/2016/09/sharepoint-automation-unified_14.html
#>
param(
  $siteCollectionUrl = "https://contoso.sharepoint.com/sites/sitecollection2016", # URL of either SharePoint Online or "on-premises" site collection
  $username = "paul.borisov@contoso.onmicrosoft.com",                             # Login name of either SharePoint Online account or "on-premises" Windows user in format DOMAIN\account
  #$siteCollectionUrl = "https://dev.contoso.local/sites/sitecollection2016",     # URL of either SharePoint Online or "on-premises" site collection
  #$username = "vanilla\paul.borisov",                                            # Login name of either SharePoint Online account or "on-premises" Windows user in format DOMAIN\account
  $password = "**********",                                                       # User's password
  $useLocalEnvironment = $false,    # Set to $true if you intend to use local SharePoint environment. This can be usefull in case of "on-premises" site collections (not in the SharePoint Online).
  $useDefaultCredentials = $false,  # Set to $true if you intend to use default network credentials instead of username and password
  $pathToDlls = "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\ISAPI", # Full path to the folder that contains SharePoint Client components (use /15/ for SP2013 and /16/ for SP2016)
  $logFile = $PSScriptRoot + "\" + (get-date).ToString("yyyy-MM-dd-HH-mm-") + "log.txt"
)
######################################################## FUNCTIONS ########################################################
function GetAuthenticationCookie() {
  $sharePointUri = New-Object System.Uri($context.Url)
  $authCookie = $context.Credentials.GetAuthenticationCookie($sharePointUri)
  if( $? -eq $false ) {
    return $null
  } else {
    $fedAuthString = $authCookie.TrimStart("SPOIDCRL=".ToCharArray())
    $cookieContainer = New-Object System.Net.CookieContainer
    $cookieContainer.Add($sharePointUri, (New-Object System.Net.Cookie("SPOIDCRL", $fedAuthString)))
    return $cookieContainer
  }
}
   
function GetClientContext() {
  $context = New-Object Microsoft.SharePoint.Client.ClientContext($siteCollectionUrl)
  if( $useDefaultNetworkCredentials ) {
    $context.Credentials = [System.Net.CredentialCache]::DefaultCredentials
  } elseif( $useLocalEnvironment ) {
    #$domain = $null
    #$login = $null
    #if( $username.Contains("@") ) {
    #  $domain = $username.Substring($username.IndexOf('@') + 1)
    #  $login = $username.Substring(0, $username.IndexOf('@'))
    #  $context.Credentials = (New-Object System.Net.NetworkCredential($login, $password, $domain))
    #} elseif( $username.Contains("\") ) {
    #  $domain = $username.Substring(0, $username.IndexOf('\'))
    #  $login = $username.Substring($username.IndexOf('\') + 1)
    #  $context.Credentials = (New-Object System.Net.NetworkCredential($login, $password, $domain))
    #} else {
    $context.Credentials = (New-Object System.Net.NetworkCredential($username, $password))
    #}
  } else {
    $securePassword = ConvertTo-SecureString $password -AsPlainText -Force
    $credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($username, $securePassword)
    $context.Credentials = $credentials
  }
  $context.Load($context.Site)
  $context.Load($context.Site.RootWeb)
   
  $context.ExecuteQuery()
  if( $? -eq $false ) {
    return $null
  } else {
    return $context
  }
}

function CreateWebRequest($requestUrl) {
  $cookieContainer = $null
  if( !$useLocalEnvironment -and !$useDefaultNetworkCredentials ) {
    $cookieContainer = GetAuthenticationCookie
    if( $cookieContainer -eq $null ) {
      return
    }
  }

  $request = $context.WebRequestExecutorFactory.CreateWebRequestExecutor($context, $requestUrl).WebRequest
  $request.Method = "GET"
  $request.Accept = "text/html, application/xhtml+xml, */*"
  if( $cookieContainer -ne $null )  {
    $request.CookieContainer = $cookieContainer
  } elseif ( $useDefaultNetworkCredentials ) {
    $request.UseDefaultCredentials = $true
  } elseif( $useLocalEnvironment ) {
    $request.Credentials = $context.Credentials
  }
  $request.UserAgent = "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
  $request.Headers["Cache-Control"] = "no-cache"
  $request.Headers["Accept-Encoding"] = "gzip, deflate"
  return $request
}

function ExecuteWebRequest($request, $requestData, [ref]$errorMessage, [ref]$successMessage) {
  if( $requestData -ne $null -and $requestData.Keys -ne $null ) {
    # Format inputs as postback data string
    $strData = $null
    foreach( $inputKey in $requestData.Keys )  {
      if( -not([String]::IsNullOrEmpty($inputKey)) ) {
        $strData += [System.Web.HttpUtility]::UrlEncode($inputKey) `
          + "=" + [System.Web.HttpUtility]::UrlEncode($requestData[$inputKey]) + "&"
      }
    }
    $strData = $strData.TrimEnd('&')
    $requestData = [System.Text.Encoding]::UTF8.GetBytes($strData)
  } 
  $request.ContentLength = $requestData.Length
   
  # Add postback data to the request stream
  $stream = $request.GetRequestStream()
  try {
    $stream.Write($requestData, 0, $requestData.Length)
  } finally {
    $stream.Close()
    $stream.Dispose()
  }
     
  $response = $request.GetResponse()
  $stream = $response.GetResponseStream()
  try {
    if( -not([string]::IsNullOrEmpty($response.Headers["Content-Encoding"])) ) {
      if( $response.Headers["Content-Encoding"].ToLower().Contains("gzip") ) {
        $stream = New-Object System.IO.Compression.GZipStream($stream, 
          [System.IO.Compression.CompressionMode]::Decompress)
      } elseif ( $response.Headers["Content-Encoding"].ToLower().Contains("deflate") ) {
        $stream = New-Object System.IO.Compression.DeflateStream($stream, 
          [System.IO.Compression.CompressionMode]::Decompress)
      }
    }
         
    # Retrieve response content as string
    $reader = New-Object System.IO.StreamReader($stream)
    $content = $reader.ReadToEnd()
    #$content > $logFile
    if( $successMessage.Value -ne $null ) {
      $match = [regex]::Match($content, $successMessage.Value, "Singleline")
      $successMessage.Value = $null
      if( $match.Success ) {
        for( $i = 1; $i -lt $match.Groups.Count; $i++ ) {
          $tmp = $match.Groups[$i].Value
          if( $tmp.Length -gt 1 ) {
            $tmp = $tmp.Substring(0,1).ToUpper() + $tmp.Substring(1)
          }
          $successMessage.Value += $tmp
        }
      }     
    }
    $match = $null
    if( $errorMessage.Value -ne $null ) {
      $match = [regex]::Match($content, $errorMessage.Value, "Singleline")
    } else {
      $match = [regex]::Match($content, 
        '(?i)<div id="ms-error">.+<span id="ctl\d+_PlaceHolderMain_LabelMessage">(.[^<]+)<\/span>', "Singleline")
    }
    $errorMessage.Value = $null
    if( $match.Success ) {
      for( $i = 1; $i -lt $match.Groups.Count; $i++ ) {
        $tmp = $match.Groups[$i].Value
        if( $tmp.Length -gt 1 ) {
          $tmp = $tmp.Substring(0,1).ToUpper() + $tmp.Substring(1)
        }
        $errorMessage.Value += $tmp
      }
    }
    $reader.Close()
    $reader.Dispose()
  } finally {        
    $stream.Close()
    $stream.Dispose()
  }
}
####################################################### //FUNCTIONS #######################################################
   
####################################################### EXECUTION #########################################################
# Step 1. Ensure the client components loaded
[Reflection.Assembly]::LoadFile(($pathToDlls.TrimEnd('\') + "\Microsoft.SharePoint.Client.dll")) | out-null
[Reflection.Assembly]::LoadFile(($pathToDlls.TrimEnd('\') + "\Microsoft.SharePoint.Client.Runtime.dll")) | out-null
$context = GetClientContext

Known issues

If your connection attempt failed with the error "The remote server returned an error: (403) Forbidden." make sure you are connecting to the correct environment using valid credentials and correct configuration parameters.
  • If you connect to the SharePoint Online both configuration parameters $useLocalEnvironment and $useDefaultCredentials must be set to $false.
  • If you connect to an "on-premises" SharePoint environment the configuration setting $useLocalEnvironment must be set to $true while  $useDefaultCredentials can be $true or $false depending on the credentials you use to connect. If $useDefaultCredentials = $false is specified you should use valid $username and $password for your local environment.
  • Verify the credentials specified in the configuration parameters $username and $password are correct and correspond to the chosen environment.
  • Also verify a value for $pathToDlls. If you run the script on a machine that has a local SharePoint Farm installed this setting usually points a standard SP-hive. It can contain /15/ or /16/ depending on your local version of SharePoint.
    • I have had a challenge once when I have tested the script on a Virtual Machine with SharePoint 2013 installed that also had a folder of 16-hive with a stuff for SharePoint 2016 added by some tool. And I have tried connecting to SharePoint Online.
    • In that case the setting $pathToDlls = "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\ISAPI" have worked perfectly while $pathToDlls = "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\16\ISAPI" have refused to work producing the error 403 (see above).
Posted by at No comments:
Labels: , , ,

Tuesday, September 6, 2016

Simple and functional code syntax highlighter gadget for the Blogger, part 2

In my previous blog post I was evaluating a number of free code syntax highlighters that could be used in my blog. I have also developed a simple and usable gadget that extends the formatted content produced by the the Syntax Highlighter with convenient buttons that allow selecting the complete code blocks. This eliminates annoying  process of selection with a mouse throughout multiple screens.

I have also mentioned about a true lightweight and usable alternative tool, hilite.me. It impressed me with "no code" approach, rich formatting options, and complete absence of usually annoying tiny bugs. In other words, you can just quickly format your source code on the site and directly insert it into your blog post. I mean you do not need to bother of embedding any kind of custom JS-libraries to support the formatting of your code in the post.

A gadget

Traditionally, I have developed a second gadget for my blog. It extends the formatted code, produced by "hilite,me" and inserted into a blog post, with convenient buttons that can select the complete code blocks. It also adds a green line separator between row numbers and lines of code to unify the "look-and-feel" with the highlighting functionality of GitHub Gist and the Syntax Highlighter,

How to use this gadget to highlight your code

1. Copy the source code you plan to add to your blog post into he clipboard from the code block shown below.

2. Open the hilite.me, paste your source code into the left text area, select a desired Language below it, change CSS option in the text box below Language to "padding:0px 0px;", select Style (I use vs), toggle the checkbox Line numbers, and click the button "Highlight!".

3. Copy the produced formatted output from the right text area into the clipboard.

4. Paste the content into your blog post and save it. It should already look good.

5. Open your blog settings, change to Layout, add an "HTML/JavaScript gadget" into a section "footer-<n>" of your blog's layout, copy-paste the complete source code of the gadget shown above, and save changes.

  • Use the button "Select the code", copy-paste and save changes.

After that, your inserted source code should look like on the screen below.

Complete code of the gadget 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
<!-- 
The gadget that refines the formatted output from http://hilite.me. 
Author: Paul Borisov, http://paulborisov.blogspot.fi/2016/09/simple-and-functional-source-code.html
-->
<!--script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.1.0/jquery.min.js" type="text/javascript"></script-->
<script type="text/javascript">  
  jQuery(document).ready(function() {
    jQuery.fn.selTextHlm = function() {
      if( this.length === 0 ) return;
      var doc = document
        , text = this[0]
        , range, selection
      ;    
      if (doc.body.createTextRange) {
        range = document.body.createTextRange();
        range.moveToElementText(text);
        range.select();
      } else if (window.getSelection) {
        selection = window.getSelection();        
        range = document.createRange();
        range.selectNodeContents(text);
        selection.removeAllRanges();
        selection.addRange(range);
      }
    }

    jQuery("tr > td:nth-child(2) > pre").each(function() {
      var uniqueId = Math.random().toString(16).slice(2);
      var idTop = "btnSelectCode" + uniqueId + "top";
      var idBottom = "btnSelectCode" + uniqueId + "bottom";
      if( !jQuery(this).parent().hasClass("code-block-container-hlm") ) {
        var htmlButton = '<div class="code-selection-container-hlm"><div id="{0}">Select the code</div></div>';
        jQuery(this).wrap('<div class="code-block-container-hlm"></div>');
        if( !jQuery(this).hasClass("hide-top-button") ) {
          var buttonTop = htmlButton.replace("{0}", idTop);
          var codeTopDiv = jQuery(this).closest("table").parent();
          codeTopDiv.before(buttonTop);
          if( !codeTopDiv.hasClass("code-top-container-hlm") ) {
            codeTopDiv.addClass("code-top-container-hlm");
          }
        }
        if( !jQuery(this).hasClass("hide-bottom-button") ) {
          var buttonBottom = htmlButton.replace("{0}", idBottom);
          var codeTopDiv = jQuery(this).closest("table").parent();
          codeTopDiv.after(buttonBottom);
          if( !codeTopDiv.hasClass("code-top-container-hlm") ) {
            codeTopDiv.addClass("code-top-container-hlm");
          }
        }

        function SelectTextInCodeContainerHlm(codeContainer) {
          var codeTopContainer = codeContainer.closest(".code-top-container-hlm");
          // Get and keep scroll positions
          var scrollLeft = codeTopContainer.scrollLeft();
          var scrollTop = jQuery(document).scrollTop();
          codeContainer.selTextHlm();
          // Next line eliminates unpleasant bug with changed scroll positions in IE.
          codeTopContainer.scrollLeft(0);
          jQuery(document).scrollTop(scrollTop);       
        }
        jQuery("#" + idTop).click(function() {
          var codeContainer = 
            jQuery(this).closest(".code-selection-container-hlm")
              .next().find(".code-block-container-hlm pre");
          SelectTextInCodeContainerHlm(codeContainer);
        });
        jQuery("#" + idBottom).click(function() {
          var codeContainer = 
            jQuery(this).closest(".code-selection-container-hlm")
              .prev().find(".code-block-container-hlm pre");
          SelectTextInCodeContainerHlm(codeContainer);
        });
      }
      
      var re1 = /<span\s+?style="border\:\s+?1px\s+?solid\s+?#FF0000;">/gi;
      var re2 = /<span\s+?style="border\:\s+?1px\s+?solid\s+?rgb\(255,\s+?0,\s+?0\);\s+?border-image\:\s+?none;">/gi;
      var replacement = '<span style="color:#00f;">';
      var html = jQuery(this).html();
      if( html.match(re1) ) {
        jQuery(this).html(html.replace(re1,replacement));
        html = jQuery(this).html();
      }
      if( html.match(re2) ) {
        jQuery(this).html(html.replace(re2, replacement));
        //html = jQuery(this).html();
      }
    });
  });
</script>
<style type="text/css">
  .code-selection-container-hlm {text-align: left;}
  .code-selection-container-hlm > div {cursor:pointer; border:1px #ccc solid;padding:2px 2px 2px 8px; width: 100px; display:inline-block; border-radius: 7px 7px;}
  .code-top-container-hlm {margin-top:15px;margin-bottom:15px;width:875px !important;overflow:hidden;}
  .code-top-container-hlm tr td:nth-of-type(1) {border-right:3px solid #6ce26c !important;color:#afafaf !important;padding-right:5px;}
  .code-top-container-hlm tr td:nth-of-type(2) {padding-left:5px;}
  .code-block-container-hlm {}
</style>



Posted by at No comments:
Labels:

Monday, September 5, 2016

Installation of Windows Domain Controller on an existing SharePoint Farm

A couple of weeks ago I faced with an interesting challenge. I have been upgrading our development machine that contained an existing Farm of SharePoint 2013 running in a workgroup of Windows 2012.

After I have installed a Windows Domain Controller on this machine the SharePoint Farm has stopped working.

Error message

 Any attempts to connect to a Farm via browser or Powershell immediately failed with a very misleading error message that read as

This operation can be performed only on a computer that is joined to a server farm by users who have permissions in SQL Server to read from the configuration database. To connect this server to the server farm, use the SharePoint Products Configuration Wizard, located on the Start menu in Microsoft SharePoint 2010 Products.


Yes, namely a legacy message for SharePoint 2010 was visible on the failed Farm of 2013.

Anyway, the error looked quite familiar to me. I had seen this message many times before in our data migration projects and the reason was usually related to insufficient permissions of the application pool and (or) Farm account on the configuration database.

But this time, the reason was different.

The actual reason

Installation of a Domain Controller significantly tightens security permissions on the registry hives and certain folders of the file system in compare with more permissive settings in a Workgroup setup.

How to repair your SharePoint Farm

The required steps read as:

  • Open Start > Run > regedit.exe > HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\15.0\Secure > Right mouse button > Permissions...
  • Grant "Read" permissions to the domain group DOMAIN\WSS_WPG and "Full Control" to the domain group DOMAIN\WSS_Admin_WPG.
  • Open Properties > Security tab of the folder ‪C:\ProgramData\Microsoft\SharePoint
  • Grant "Read & Execute" permissions to the domain group DOMAIN\WSS_WPG and "Full Control" to the domain group DOMAIN\WSS_Admin_WPG
    • This should eliminate the problems with Distributed Cache permissions visible in the ULS logs.
  • Reprovision Distributed Cache via SharePoint's PowerShell
    • Use-CacheCluster
    • Get-CacheHost -- > It can show "DOWN" in the status, in this case, execute the next 4 lines
    • Stop-SPDistributedCacheServiceInstance -Gracefully
    • Remove-SPDistributedCacheServiceInstance
    • Add-SPDistributedCacheServiceInstance
    • Get-CacheHost --> This time, it must show "UP" in the status
    • Open Properties > Security tab of the folder ‪C:\Program Files\Microsoft Office Servers\15.0\Data\Office Server\Applications
    • Grant "Read & Execute" permissions to the domain group DOMAIN\WSS_WPG and "Full Control" to the domain group DOMAIN\WSS_Admin_WPG
      • This step should fix weird looking errors visible in the Content Search Web Parts and in the Search Scheme

Posted by at No comments:
Labels:

Simple and functional code syntax highlighter gadget for the Blogger, part 1

When I was writing one of my previous posts to this blog I faced with quite a typical problem of any developer who wanted to publish a source code to a web resource in a familiar and "easy to read" format. I have quickly tried a number of online and offline tools but found most of them having specific issues and missing a certain functionality.

My expectations of the highlighting functionality for the posted source code
  1. The code must be formatted to look like in the familiar or standard development tools, i.e. to contain easily recognizable and readable code pieces and properly colored items.
  2. Row numbers should be present on the left side of each line. This allows readers to refer to a particular line of the posted source code easily.
  3. The posted source code should be easily selectable on the page as a complete area without a necessity to drag a mouse throughout multiple screens. When I make a "copy-paste"of  a selected code snippet the copied content should not contain any extra characters like row numbers or contaminating HTML-tags. In the perfect case, the inserted code could be immediately evaluated and executed by a reader without any additional cleaning.
  4. Alternation of line background color should be supported although I do not consider this option as absolutely necessary.
  5. The absence of visual problems in the highlighted code that may turn a simple attempt to highlight a code into a very time-consuming task for a blogger. Example: annoying and difficult to fix wraps to other lines that make the code uneasy to read.
  6. Lightweight of the used components with quick time to load and to apply highlighting to multiple posts on the same page.
  7. Free of charge, possibly with an option to donate the author if I liked the functionality. As a good practice, the component should not display any annoying visual signs that would remind readers to make a donation.

Easy to find known free tools that can be used to highlight the posted source code dynamically

GitHub Gist

 This tool provides a rich highlighting functionality that includes line numbers, line selection, and alternation but requires adding your code to a repository (secret anonymous or named public). It also adds quite a clumsy "iframe" element to your blog post, which means loading time can be occasionally poor.

ToHtml

 An online code highlighting engine that wraps your code into nice looking lightweight HTML tags to be directly inserted into your post. This tool offers relatively simple formatting options without row numbers and alternation. I have also noticed some bugs of this tool in my tests, for example, when the first part of my code was highlighted correctly but the residual part was not (after the tool could not correctly handle a specific character in the middle of the code area). Anyway, this can be a fair choice for simple code snippets.

hilite.me

One more online tool that produces lightweight HTML out of your source code, supports row numbers in a separate column (important for excluding it from selection), can render the classic styles of Visual Studio, and does not seem to have obvious bugs like the previous one. Excellent although requires manual copy paste and possible cleaning of undesired elements; I slightly dislike its red highlighting of some punctuation elements.

Syntax Highlighter

This is the old but still very functional tool that provides a rich highlighting functionality with row numbers. Surprisingly enough, it does not seem to support alternation of line background colors. On the other hand, it supports lightweight extendable "look and feels" (so called brushes) that cover most of the code languages. I found this tool also contains non-significant tiny visual bugs, for example, it adds unexpected extra characters to the last line of Powershell comments. It also displays an annoying "ad" in the right top corner of the formatted code snippet. Anyway, all issues were easy to fix.

My choice

After conducting a number of tests, I have finally decided to use the last tool in this blog (Syntax Highlighter). It fits most of my mentioned expectations except for line background color alternation, provides fairly rich coloring schemes, and has very moderate loading time.

Usage of the tool is not intuitive but still relatively simple.
  • First of all, wrap your source code that you insert into your blog post in <pre>...</pre> tags.
  • After that add a class "brush: <acronym>" into <pre>, for example, <pre class="brush: html">. Unfortunately, I could not quickly find a way to apply multiple brushes to the same code area, which can definitely contain a mix of several code snippets, typically, HTML + JavaScript + CSS.
  • Add a number of references to .css and .js files of the "Syntax Highlighter" available from any fast hosting provider, for example, from a CDN or the site of the tool's author. I provide more specific details below.

Custom gadget for highlighting source code in all my blog posts

 Fairly, I have been pleasantly surprised by the functionality of the "Syntax Highlighter". So I have immediately developed a simple custom gadget for my own blog that seamlessly ensures the code highlighting options to all my posts automatically on the first load. All I need to do when I create a new blog post with code snippets, to place my code inside the tag <pre></pre> and specify a brush in its class settings.
  • In addition, my gadget renders the buttons "Select the code" above and below the code blocks that allow selecting the complete block on a single click. Syntax Highlighter definitely lacks this option.
  • The gadget also slightly widens a visible code area to make it better readable and hides the annoying tool's "big green dot" ad that appeared in the right top corner of every formatted code snippet.
Complete code of the gadget

I used a CDN cdnjs.cloudflare.com in my examples (found it in Google). I have no information who maintains this CDN but it works faster than the original site of the tool.

<!-- The gadget that refines the formatted output produced by the Syntax Highlighter. Author: Paul Borisov, http://paulborisov.blogspot.fi/2016/09/simple-and-functional-source-code.html -->
<link href="https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/styles/shCore.css" rel="stylesheet" type="text/css" />
<link href="https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/styles/shThemeDefault.css" rel="stylesheet" type="text/css" />
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.1.0/jquery.min.js" type="text/javascript"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shCore.js" type="text/javascript"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shAutoloader.js" type="text/javascript"></script>
<script type="text/javascript">
  jQuery(document).ready(function() {
    SyntaxHighlighter.autoloader(
      "C# c-sharp csharp https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushCSharp.js",
      "PowerShell ps powershell https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushPowerShell.js",
      "CSS css https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushCss.js",
      "JavaScript js jscript javascript https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushJScript.js",
      "Java java https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushJava.js",
      "SQL sql https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushSql.js",
      "XML xml xhtml xslt html xhtml https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushXml.js",
      "ActionScript3 as3 actionscript3 https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushAS3.js",
      "Bash/shell bash shell https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushBash.js",
      "ColdFusion cf coldfusion https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushColdFusion.js",
      "C++ cpp c https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushCpp.js",
      "CSS css https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushCss.js",
      "Delphi delphi pas pascal https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushDelphi.js",
      "Diff diff patch https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushDiff.js",
      "Erlang erl erlang https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushErlang.js",
      "Groovy groovy https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushGroovy.js",
      "JavaFX jfx javafx https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushJavaFX.js",
      "Perl perl pl https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushPerl.js",
      "PHP php https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushPhp.js",
      "PlainText plain text https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushPlain.js",
      "Python py python https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushPython.js",
      "Ruby rails ror ruby https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushRuby.js",
      "Scala scala https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushScala.js",
      "VisualBasic vb vbnet https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushVb.js"
    );
    SyntaxHighlighter.config.bloggerMode = true;
    SyntaxHighlighter.all();
    
    // I noticed a small bug of the highliter and fixed it (Paul Borisov, 4.9.2016).
    jQuery("code[class*='comments']:last").each(function() {
      jQuery(this).text(jQuery(this).text().replace(/((<)|(<))\/input\.\+\?\\((>)|(>))/,""));
    });
    var tmp = setInterval(function() {
      jQuery("code[class*='plain']:last").each(function() {
        if( jQuery(this).parent().html().toLowerCase().match(/<code.+>((<)|(<))\/input\.\+\?\\((>)|(>))<\/code>/) != null ) {
          jQuery(this).parent().detach();
          clearInterval(tmp);
        }
      });
    }, 200);
    // Remove the ugly looking "big green dot" component's ad
    jQuery("div.syntaxhighlighter > div.toolbar").detach();
    
    jQuery.fn.selTextShl = function() {
      if( this.length === 0 ) return;
      var doc = document
        , text = this[0]
        , range, selection
      ;    
      if (doc.body.createTextRange) {
        range = document.body.createTextRange();
        range.moveToElementText(text);
        range.select();
      } else if (window.getSelection) {
        selection = window.getSelection();        
        range = document.createRange();
        range.selectNodeContents(text);
        selection.removeAllRanges();
        selection.addRange(range);
      }
    }

    jQuery("pre[class*='brush:']").each(function() {
      var uniqueId = Math.random().toString(16).slice(2);
      var idTop = "btnSelectCode" + uniqueId + "top";
      var idBottom = "btnSelectCode" + uniqueId + "bottom";
      if( !jQuery(this).parent().hasClass("code-block-container-shl") ) {
        var htmlButton = '<div class="code-selection-container-shl"><div id="{0}">Select the code</div></div>';
        jQuery(this).wrap('<div class="code-block-container-shl"></div>');
        if( !jQuery(this).hasClass("hide-top-button") ) {
          var buttonTop = htmlButton.replace("{0}", idTop);
          jQuery(this).before(buttonTop);
        }
        if( !jQuery(this).hasClass("hide-bottom-button") ) {
          var buttonBottom = htmlButton.replace("{0}", idBottom);
          jQuery(this).after(buttonBottom);
        }

        function SelectTextInCodeContainerShl(codeContent) {
          var codeTopContainer = codeContent.closest(".syntaxhighlighter");
          // Get and keep scroll positions
          var scrollLeft = codeTopContainer.scrollLeft();
          var scrollTop = jQuery(document).scrollTop();
          codeContent.selTextShl();
          // Next line eliminates unpleasant bug with changed scroll positions in IE.
          codeTopContainer.scrollLeft(0);
          jQuery(document).scrollTop(scrollTop);       
        }
        jQuery("#" + idTop + ",#" + idBottom).click(function() {
          var codeContainer = jQuery(this).parents(".code-block-container-shl");
          var codeContent = codeContainer.find("td.code:first");
          if( codeContent.length === 0 ) {
            codeContainer.selTextShl();  // In the case the highlighter has failed to load just select the content of <pre>...</pre>
          } else {
            SelectTextInCodeContainerShl(codeContent);
          }
        });
      }
    });
  });
</script>
<style type="text/css">
  .code-selection-container-shl {text-align: left;}
  .code-selection-container-shl > div {margin-left:5px;cursor:pointer;border:1px #ccc solid;padding:2px 2px 2px 8px;width:100px;display:inline-block;border-radius:7px 7px;}
  .code-block-container-shl {width:875px !important;overflow:hidden;margin-left:-5px;}
  div.syntaxhighlighter {padding-top:10px;padding-bottom:10px;}
  div.syntaxhighlighter > div.toolbar {display: none !important;}
  div.syntaxhighlighter table td.gutter .line {padding:0px 5px 0px 0px !important;}
</style>

Split into separate code blocks for HTML, JavaScript, and CSS to demonstrate different brushes

HTML 

<link href="https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/styles/shCore.css" rel="stylesheet" type="text/css" />
<link href="https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/styles/shThemeDefault.css" rel="stylesheet" type="text/css" />
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.1.0/jquery.min.js" type="text/javascript"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shCore.js" type="text/javascript"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shAutoloader.js" type="text/javascript"></script>

JavaScript 

<script type="text/javascript">
  jQuery(document).ready(function() {
    SyntaxHighlighter.autoloader(
      "C# c-sharp csharp https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushCSharp.js",
      "PowerShell ps powershell https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushPowerShell.js",
      "CSS css https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushCss.js",
      "JavaScript js jscript javascript https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushJScript.js",
      "Java java https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushJava.js",
      "SQL sql https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushSql.js",
      "XML xml xhtml xslt html xhtml https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushXml.js",
      "ActionScript3 as3 actionscript3 https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushAS3.js",
      "Bash/shell bash shell https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushBash.js",
      "ColdFusion cf coldfusion https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushColdFusion.js",
      "C++ cpp c https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushCpp.js",
      "CSS css https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushCss.js",
      "Delphi delphi pas pascal https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushDelphi.js",
      "Diff diff patch https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushDiff.js",
      "Erlang erl erlang https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushErlang.js",
      "Groovy groovy https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushGroovy.js",
      "JavaFX jfx javafx https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushJavaFX.js",
      "Perl perl pl https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushPerl.js",
      "PHP php https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushPhp.js",
      "PlainText plain text https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushPlain.js",
      "Python py python https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushPython.js",
      "Ruby rails ror ruby https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushRuby.js",
      "Scala scala https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushScala.js",
      "VisualBasic vb vbnet https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushVb.js"
    );
    SyntaxHighlighter.config.bloggerMode = true;
    SyntaxHighlighter.all();
    
    // I noticed a small bug of the highliter and fixed it (Paul Borisov, 4.9.2016).
    jQuery("code[class*='comments']:last").each(function() {
      jQuery(this).text(jQuery(this).text().replace(/((<)|(<))\/input\.\+\?\\((>)|(>))/,""));
    });
    var tmp = setInterval(function() {
      jQuery("code[class*='plain']:last").each(function() {
        if( jQuery(this).parent().html().toLowerCase().match(/<code.+>((<)|(<))\/input\.\+\?\\((>)|(>))<\/code>/) != null ) {
          jQuery(this).parent().detach();
          clearInterval(tmp);
        }
      });
    }, 200);
    // Remove the ugly looking "big green dot" component's ad
    jQuery("div.syntaxhighlighter > div.toolbar").detach();
    
    jQuery.fn.selTextShl = function() {
      if( this.length === 0 ) return;
      var doc = document
        , text = this[0]
        , range, selection
      ;    
      if (doc.body.createTextRange) {
        range = document.body.createTextRange();
        range.moveToElementText(text);
        range.select();
      } else if (window.getSelection) {
        selection = window.getSelection();        
        range = document.createRange();
        range.selectNodeContents(text);
        selection.removeAllRanges();
        selection.addRange(range);
      }
    }

    jQuery("pre[class*='brush:']").each(function() {
      var uniqueId = Math.random().toString(16).slice(2);
      var idTop = "btnSelectCode" + uniqueId + "top";
      var idBottom = "btnSelectCode" + uniqueId + "bottom";
      if( !jQuery(this).parent().hasClass("code-block-container-shl") ) {
        var htmlButton = '<div class="code-selection-container-shl"><div id="{0}">Select the code</div></div>';
        jQuery(this).wrap('<div class="code-block-container-shl"></div>');
        if( !jQuery(this).hasClass("hide-top-button") ) {
          var buttonTop = htmlButton.replace("{0}", idTop);
          jQuery(this).before(buttonTop);
        }
        if( !jQuery(this).hasClass("hide-bottom-button") ) {
          var buttonBottom = htmlButton.replace("{0}", idBottom);
          jQuery(this).after(buttonBottom);
        }

        function SelectTextInCodeContainerShl(codeContent) {
          var codeTopContainer = codeContent.closest(".syntaxhighlighter");
          // Get and keep scroll positions
          var scrollLeft = codeTopContainer.scrollLeft();
          var scrollTop = jQuery(document).scrollTop();
          codeContent.selTextShl();
          // Next line eliminates unpleasant bug with changed scroll positions in IE.
          codeTopContainer.scrollLeft(0);
          jQuery(document).scrollTop(scrollTop);       
        }
        jQuery("#" + idTop + ",#" + idBottom).click(function() {
          var codeContainer = jQuery(this).parents(".code-block-container-shl");
          var codeContent = codeContainer.find("td.code:first");
          if( codeContent.length === 0 ) {
            codeContainer.selTextShl();  // In the case the highlighter has failed to load just select the content of <pre>...</pre>
          } else {
            SelectTextInCodeContainerShl(codeContent);
          }
        });
      }
    });
  });
</script>

CSS 

<style type="text/css">
  .code-selection-container-shl {text-align: left;}
  .code-selection-container-shl > div {margin-left:5px;cursor:pointer;border:1px #ccc solid;padding:2px 2px 2px 8px;width:100px;display:inline-block;border-radius:7px 7px;}
  .code-block-container-shl {width:875px !important;overflow:hidden;margin-left:-5px;}
  div.syntaxhighlighter {padding-top:10px;padding-bottom:10px;}
  div.syntaxhighlighter > div.toolbar {display: none !important;}
  div.syntaxhighlighter table td.gutter .line {padding:0px 5px 0px 0px !important;}
</style>

How to use my custom gadget

Obviously, this custom gadget is free. You can add it to your own blog as well if you wish. Just open your blog settings, change to Layout, add an "HTML/JavaScript gadget" into a section "footer-<n>" of your blog's layout, copy-paste the complete source code of the gadget shown above, and save changes..
  • Use the button "Select the code", copy-paste and save changes.

Next add your source code into <pre class="brush; <acronym>">...</pre> as described above in the chapter "My choice" and save the post. After that, your inserted source code should be highlighted.

Posted by at No comments:
Labels:

Sunday, September 4, 2016

SharePoint Automation: unified Powershell script to redeploy sandbox solutions to SharePoint Online and "on-premises"

As you probably know, Microsoft has recently declared restriction of the Code-Based Sandbox Solutions (CBSS) in SharePoint Online.

 On the other hand, Microsoft has kept the complete functionality of No Code Sandbox Solutions (NCSS) in SharePoint Online untouched so, technically, NCSS work as previously. Apparently, this is a wise decision that allows us to strip and convert only a server side code of legacy WSP-solutions and still keep the old working feature framework untouched*.

 * Despite Microsoft insistently proposes to change the legacy solutions to so-called “Add-In model” this is not always possible to redesign complex legacy “Feature-based model” applications in a cost effective manner. There is an excellent article of Chris O'Brian on how to deal with conversions of similar legacy code. 
- Shortly, you should get rid of any server side code in your wsp-solutions and exclude DLLs from the packages. 
- "On-premises" versions of SharePoint can still use legacy server side code. For example, sandbox solutions that have been inherited from SharePoint 2010 and worked in SharePoint 2013 can still work in 2016.

 About automatic redeployment of sandbox solutions

 Certainly, there is a number of samples in various blogs that demonstrate how to redeploy sandbox solutions with a code. 

 For example, a couple of years ago I used a nice SharePoint Online Helper Library to support automatic redeployment of sandbox solutions in my projects. There is also a Powershell-based variant of the same solution developed by Jeffrey Paarhuis.

 However, I have found none of those solutions could provide seamless and complete automation for both environments, "on-premises" SharePoint and SharePoint Online.

 So I have developed a compact and more unified Powershell-script that covers both environments in my projects. The script works in SharePoint Online as well as in SharePoint 2013 and 2016 "on-premises". It may work in SharePoint 2010; I have not tested the script with that version, though.
The script

 The important configurable settings with usage descriptions can be found in the "param" section of the script's header.

<#
Unified automation script for (re)deployment of sandbox solutions to SharePoint Online and "on-premises" (2013, 2016).
- Inspired by https://spohelper.codeplex.com and https://raw.githubusercontent.com/janikvonrotz/PowerShell-PowerUp/master/functions/SharePoint%20Online/Switch-SPOEnableDisableSolution.ps1
Copyright (c) Paul Borisov, 2016.
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
#>
param(
  $siteCollectionUrl = "https://contoso.sharepoint.com/sites/sitecollection2016", # URL of either SharePoint Online or "on-premises" site collection
  $username = "paul.borisov@contoso.onmicrosoft.com",                             # Login name of either SharePoint Online account or "on-premises" Windows user in format DOMAIN\account
  #$siteCollectionUrl = "https://dev.contoso.local/sites/sitecollection2016",     # URL of either SharePoint Online or "on-premises" site collection
  #$username = "vanilla\paul.borisov",                                            # Login name of either SharePoint Online account or "on-premises" Windows user in format DOMAIN\account
  $password = "**********",                                                       # User's password
  $useLocalEnvironment = $false,    # Set to $true if you intend to use local SharePoint environment. This can be usefull in case of "on-premises" site collections (not in the SharePoint Online).
  $useDefaultCredentials = $false,  # Set to $true if you intend to use default network credentials instead of username and password
  $pathToDlls = "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\ISAPI", # Full path to the folder that contains SharePoint Client components (use /15/ for SP2013 and /16/ for SP2016)
  $pathToFolderWIthWspSolutions = "$PSScriptRoot\WSPs"                            # Full path to the folder, which contains sandbox wsp-solutions to be (re)deployed
)
######################################################## FUNCTIONS ########################################################
function ActivateOrDeactivateSolution(
  $context, $cookieContainer, $solutionId, $activate, $useLocalEnvironment, $useDefaultNetworkCredentials, [ref]$errorMessage) {
    
  $op = $null
  if($activate) { 
    $op = "ACT"
  } else { 
    $op = "DEA"
  }
    
  $activationUrl = $context.Site.Url + "/_catalogs/solutions/forms/activate.aspx?Op=$op&ID=$solutionId"
  
  $request = $context.WebRequestExecutorFactory.CreateWebRequestExecutor($context, $activationUrl).WebRequest
  if( $cookieContainer -ne $null )  {
    $request.CookieContainer = $cookieContainer
  } elseif ( $useDefaultNetworkCredentials ) {
    $request.UseDefaultCredentials = $true
  } elseif( $useLocalEnvironment ) {
    $request.Credentials = $context.Credentials
  }
  $request.ContentLength = 0
    
  $response = $request.GetResponse()
    
  # Decompress response stream if needed
  $stream = $response.GetResponseStream()
  try {
    if( -not([string]::IsNullOrEmpty($response.Headers["Content-Encoding"])) ) {
      if( $response.Headers["Content-Encoding"].ToLower().Contains("gzip") ) {
        $stream = New-Object System.IO.Compression.GZipStream($stream, 
          [System.IO.Compression.CompressionMode]::Decompress)
      } elseif ( $response.Headers["Content-Encoding"].ToLower().Contains("deflate") ) {
        $stream = New-Object System.IO.Compression.DeflateStream($stream, 
          [System.IO.Compression.CompressionMode]::Decompress)
      }
    }
        
    # Retrieve response content as string
    $reader = New-Object System.IO.StreamReader($stream)
    $content = $reader.ReadToEnd()
    $reader.Close()
    $reader.Dispose()
  } finally {        
    $stream.Close()
    $stream.Dispose()
  }
  
  $inputMatches = $content | Select-String -AllMatches -Pattern "<input.+?\/??>" | select -Expand Matches
  $inputs = @{}
      
  # Iterate through inputs and add specific values to the dictionary for postback
  foreach( $match in $inputMatches ) {
    if( -not($match[0] -imatch "name=\""(.+?)\""") ) {
      continue
    }
    $name = $matches[1]
      
    if( -not($match[0] -imatch "value=\""(.+?)\""") ) {
      continue
    }
    $value = $matches[1]
  
    $inputs.Add($name, $value)
  }
  
  # Search for the id of the button "activate"
  $searchString = $null
  if ($activate) {
    $searchString = "ActivateSolutionItem"
  } else {
    $searchString = "DeactivateSolutionItem"
  }
          
  $match = $content -imatch "__doPostBack\(\&\#39\;(.*?$searchString)\&\#39\;"
  $inputs.Add("__EVENTTARGET", $Matches[1])
    
  $response.Close()
  $response.Dispose()
    
  # Format inputs as postback data string, but exclude the one that ends with iidIOGoBack
  $strPost = $null
  foreach( $inputKey in $inputs.Keys )  {
    if( -not([String]::IsNullOrEmpty($inputKey)) -and -not($inputKey.EndsWith("iidIOGoBack")) ) {
      $strPost += [System.Uri]::EscapeDataString($inputKey) + "=" `
        + [System.Uri]::EscapeDataString($inputs[$inputKey]) + "&"
    }
  }
  $strPost = $strPost.TrimEnd('&')
    
  $postData = [System.Text.Encoding]::UTF8.GetBytes($strPost);
  
  # Build postback request
  $activateRequest = $context.WebRequestExecutorFactory.CreateWebRequestExecutor($context, $activationUrl).WebRequest
  $activateRequest.Method = "POST"
  $activateRequest.Accept = "text/html, application/xhtml+xml, */*"
  if( $cookieContainer -ne $null )  {
    $activateRequest.CookieContainer = $cookieContainer
  } elseif ( $useDefaultNetworkCredentials ) {
    $activateRequest.UseDefaultCredentials = $true
  } elseif( $useLocalEnvironment ) {
    $activateRequest.Credentials = $context.Credentials
  }
  
  $activateRequest.ContentType = "application/x-www-form-urlencoded"
  $activateRequest.ContentLength = $postData.Length
  $activateRequest.UserAgent = "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)";
  $activateRequest.Headers["Cache-Control"] = "no-cache";
  $activateRequest.Headers["Accept-Encoding"] = "gzip, deflate";
  
  # Add postback data to the request stream
  $stream = $activateRequest.GetRequestStream()
  try {
    $stream.Write($postData, 0, $postData.Length)
  } finally {
    $stream.Close();
    $stream.Dispose()
  }
    
  # Perform the postback
  #$response = $activateRequest.GetResponse()
  #$response.Close()
  #$response.Dispose()
  $response = $activateRequest.GetResponse()
  $stream = $response.GetResponseStream()
  try {
    if( -not([string]::IsNullOrEmpty($response.Headers["Content-Encoding"])) ) {
      if( $response.Headers["Content-Encoding"].ToLower().Contains("gzip") ) {
        $stream = New-Object System.IO.Compression.GZipStream($stream, 
          [System.IO.Compression.CompressionMode]::Decompress)
      } elseif ( $response.Headers["Content-Encoding"].ToLower().Contains("deflate") ) {
        $stream = New-Object System.IO.Compression.DeflateStream($stream, 
          [System.IO.Compression.CompressionMode]::Decompress)
      }
    }
        
    # Retrieve response content as string
    $reader = New-Object System.IO.StreamReader($stream)
    $content = $reader.ReadToEnd()
    $match = [regex]::Match($content, '(?i)<div id="ms-error">.+<span id="ctl\d+_PlaceHolderMain_LabelMessage">(.[^<]+)<\/span>', "Singleline")
    if( $match.Success ) {
      $errorMessage.Value = $match.Groups[1].Value
    }
    $reader.Close()
    $reader.Dispose()
  } finally {        
    $stream.Close()
    $stream.Dispose()
  }
  
}
  
function GetAuthenticationCookie($context, $siteCollectionUrl) {
  $sharePointUri = New-Object System.Uri($siteCollectionUrl)
  $authCookie = $context.Credentials.GetAuthenticationCookie($sharePointUri)
  if( $? -eq $false ) {
    return $null
  } else {
    $fedAuthString = $authCookie.TrimStart("SPOIDCRL=".ToCharArray())
    $cookieContainer = New-Object System.Net.CookieContainer
    $cookieContainer.Add($sharePointUri, (New-Object System.Net.Cookie("SPOIDCRL", $fedAuthString)))
    return $cookieContainer
  }
}
  
function GetClientContext($siteCollectionUrl, $username, $password, $useLocalEnvironment, $useDefaultNetworkCredentials) {
  $context = New-Object Microsoft.SharePoint.Client.ClientContext($siteCollectionUrl)
  if( $useDefaultNetworkCredentials ) {
    $context.Credentials = [System.Net.CredentialCache]::DefaultCredentials
  } elseif( $useLocalEnvironment ) {
    #$domain = $null
    #$login = $null
    #if( $username.Contains("@") ) {
    #  $domain = $username.Substring($username.IndexOf('@') + 1)
    #  $login = $username.Substring(0, $username.IndexOf('@'))
    #  $context.Credentials = (New-Object System.Net.NetworkCredential($login, $password, $domain))
    #} elseif( $username.Contains("\") ) {
    #  $domain = $username.Substring(0, $username.IndexOf('\'))
    #  $login = $username.Substring($username.IndexOf('\') + 1)
    #  $context.Credentials = (New-Object System.Net.NetworkCredential($login, $password, $domain))
    #} else {
    $context.Credentials = (New-Object System.Net.NetworkCredential($username, $password))
    #}
  } else {
    $securePassword = ConvertTo-SecureString $password -AsPlainText -Force
    $credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($username, $securePassword)
    $context.Credentials = $credentials
  }
  $context.Load($context.Site)
  $context.Load($context.Site.RootWeb)
  
  $context.ExecuteQuery()
  if( $? -eq $false ) {
    return $null
  } else {
    return $context
  }
}
  
function GetSolutionFileWithListItem($context, $solutionName) {
  $solutionFileUrl = $context.Site.ServerRelativeUrl.TrimEnd('/') + "/_catalogs/solutions/" + $solutionName
  $solutionFile = $context.Site.RootWeb.GetFileByServerRelativeUrl($solutionFileUrl)
  $context.Load($solutionFile.ListItemAllFields)
  $context.ExecuteQuery()
  return $solutionFile
}
 
function GetSolutionId($context, $solutionName) {
  $solutionFile = GetSolutionFileWithListItem $context $solutionName
  return $solutionFile.ListItemAllFields.Id
}
 
function GetSolutionStatus($context, $solutionName) {
  $solutionFile = GetSolutionFileWithListItem $context $solutionName
  return $solutionFile.ListItemAllFields["Status"]
}
  
function UploadSolution($context, $filePath) {
  $fileInfo = New-Object Microsoft.SharePoint.Client.FileCreationInformation
  $fileInfo.Content = [System.IO.File]::ReadAllBytes($filePath)
  $fileInfo.Url = $filePath.Substring($filePath.LastIndexOf('\') + 1)
  $fileInfo.Overwrite = $true
  
  $folderUrl = $context.Site.Url + "/_catalogs/solutions"
  $folderUri = New-Object System.Uri($folderUrl)
  
  $solutionsFolder = $context.Site.RootWeb.GetFolderByServerRelativeUrl($folderUri.AbsolutePath)
  $uploadedFile = $solutionsFolder.Files.Add($fileInfo)
  $context.Load($uploadedFile)
  $context.ExecuteQuery()
}
####################################################### //FUNCTIONS #######################################################
  
####################################################### EXECUTION #########################################################
# Step 1. Ensure the client components loaded
[Reflection.Assembly]::LoadFile(($pathToDlls.TrimEnd('\') + "\Microsoft.SharePoint.Client.dll")) | out-null
[Reflection.Assembly]::LoadFile(($pathToDlls.TrimEnd('\') + "\Microsoft.SharePoint.Client.Runtime.dll")) | out-null
  
# Step 2. Connect to SharePoint environment amd get a client context
Write-Host
Write-Host "Connecting to $siteCollectionUrl..." -NoNewLine
$context = GetClientContext $siteCollectionUrl $username $password $useLocalEnvironment $useDefaultNetworkCredentials
if( $context -eq $null ) {
  Write-Host "Connection failed. Further execution is impossible."
  return
}
Write-Host "Done"
  
# Step 3. Retrieve authentication cookie out of a client context
$cookieContainer = $null
if( !$useLocalEnvironment -and !$useDefaultNetworkCredentials ) {
  Write-Host
  Write-Host "Retrieving authentication cookie..." -NoNewLine
  $cookieContainer = GetAuthenticationCookie $context $siteCollectionUrl
  if( $cookieContainer -eq $null ) {
    return
  }
  Write-Host "Done"
}
  
# Step 4. Deactivate existing solutions, (re)deploy, and (re)activate new versions of solutions
Get-ChildItem -Path $pathToFolderWIthWspSolutions -Filter "*.wsp" | ? {!$_.PSIsContainer} | % {
  $filePath = $_.FullName
  $solutionName = $_.Name
  $solutionId = GetSolutionId $context $solutionName
  if( $solutionId -is [int] ) {
    $message = "Deactivating an old solution $solutionName..."
    Write-Host
    Write-Host $message -NoNewLine
    try {
      ActivateOrDeactivateSolution `
        $context $cookieContainer $solutionId $false $useLocalEnvironment $useDefaultNetworkCredentials
    } catch {
      #The action can fail if the solution has already been deactivated earlier
    }
    Write-Host "Done"
  }
  
  $message = "Uploading a new solution $solutionName..."
  Write-Host
  Write-Host $message -NoNewLine
  UploadSolution $context $filePath
  if( $? -eq $false ) {continue} # Uploade error; no need for any further actions on this solution. Skip and continue to the next one.
  Write-Host "Done"
  
  # Refresh client context
  $context = GetClientContext $siteCollectionUrl $username $password $useLocalEnvironment $useDefaultNetworkCredentials
  
  $solutionId = GetSolutionId $context $solutionName
  if( $solutionId -is [int] ) {
    $message = "Activating a new solution $solutionName..."
    Write-Host
    Write-Host $message -NoNewLine

    $errorMessage = $null
    ActivateOrDeactivateSolution `
      $context $cookieContainer $solutionId $true $useLocalEnvironment $useDefaultNetworkCredentials ([ref]$errorMessage)

    $status = GetSolutionStatus $context $solutionName
    if( $status -eq $null ) {
      write-host "Not activated" -ForegroundColor Red
    } elseif( $status.LookupValue -eq 1) {
      write-host "Activated" -ForegroundColor Green
    } else {
      write-host "Status is undefined. Check in the library /_catalogs/solutions." -ForegroundColor Yellow
    }
    if( $errorMessage -ne $null ) {
      write-host $errorMessage -ForegroundColor Yellow
      $errorMessage = $null
    }
    Write-Host "Done"
  }
}
Write-Host
###################################################### //EXECUTION ########################################################

Known issues

If your connection attempt failed with the error "The remote server returned an error: (403) Forbidden." make sure you are connecting to the correct environment using valid credentials and correct configuration parameters.
  • If you connect to the SharePoint Online both configuration parameters $useLocalEnvironment and $useDefaultCredentials must be set to $false.
  • If you connect to an "on-premises" SharePoint environment the configuration setting $useLocalEnvironment must be set to $true while  $useDefaultCredentials can be $true or $false depending on the credentials you use to connect. If $useDefaultCredentials = $false is specified you should use valid $username and $password for your local environment.
  • Verify the credentials specified in the configuration parameters $username and $password are correct and correspond to the chosen environment.
  • Also verify a value for $pathToDlls. If you run the script on a machine that has a local SharePoint Farm installed this setting usually points a standard SP-hive. It can contain /15/ or /16/ depending on your local version of SharePoint.
    • I have had a challenge once when I have tested the script on a Virtual Machine with SharePoint 2013 installed that also had a folder of 16-hive with a stuff for SharePoint 2016 added by some tool. And I have tried connecting to SharePoint Online.
    • In that case the setting $pathToDlls = "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\ISAPI" have worked perfectly while $pathToDlls = "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\16\ISAPI" have refused to work producing the error 403 (see above).
Posted by at No comments:
Labels: , , ,
Subscribe to: Posts (Atom)