Getting Started with SharePoint RM

SharePoint Expert Blog

Mike Alsup

Community Topic(s): SharePoint

By Mike Alsup, Senior Vice President at Gimmal Group

October 23, 2010 - 7:23 PM

I am giving a presentation next month to a group of executives who asked, "How do we get started with SharePoint for ECM and RM? They get that SharePoint is winning if it hasn't already won. Whether this is fair or right or wrong is immaterial. SharePoint unifies much of the infrastructure for content and records in many organizations. It is less clear how SharePoint relates to Enterprise Content Management (ECM) and Records Management (RM). Their question is: where do we start to address these issues?

Here is my response:

The first step in any journey is to define where you are going, because otherwise (as Alice said), any road will get you there. A roadmap is the set of tasks and steps that need to be addressed. Typically, this includes multiple projects related to SharePoint's impact on people, processes, and technologies in the organization. A SharePoint ECM and RM roadmap, including SharePoint 2010, is particularly important because there are so many new components.  The definition of a SharePoint ECM and RM partner and product ecosystem is another important step because there are so many SharePoint partners that have made important, but overlapping contributions to SharePoint 2010 to support ECM and RM within an enterprise. These include product companies like KnowledgeLake, Nintex, AvePoint, Quest, traditional ECM providers such as EMC, Open Text and IBM, as well as professional services providers. Next week, in a follow-up post, I will outline how to evaluate the SharePoint partner ecosystem in the context of SharePoint ECM and RM. This week, I will focus on the non-ecosystem decisions and work to be done.

Content Types are Critical
Content Types are the foundation for RM within SharePoint. When properly defined, Content Types allow organizations to define the retention and records management rules for each type of content and enable the consistent enforcement of retention policies. Standard definitions of Content Types are maintained in the SharePoint Content Type Hub based on managed metadata from the SharePoint Term Store. This is powerful. We have seen the Content Type model work in companies with tens of thousands of sites, and we believe it is a foundation capability that enables the achievement of records management in large organizations.  It takes a lot of up front planning, and is easiest to achieve at a version changeover of SharePoint (e.g. 2007 to 2010), but it works at the scale of a global company.

One of the criticisms of using SharePoint 2010 for records management that we have heard recently is that many elements need to come together to implement a large file plan and retention schedule across hundreds of Content Types and thousands of SharePoint sites. This is true today, but we have found that the application of best practices can minimize the complexity and arduousness of this task.

Organizations need to standardize their Content Type definitions and rules for routing and inheritance in order to scale them to the enterprise.

Define an Enterprise Information Lifecycle
SharePoint supports the definition of an information lifecycle. A lifecycle defines the states that content goes through and the rules that are enforced in the transition between states. A standard lifecycle enables consistent treatment of content retention and disposition decisions from the creation through destruction of a piece of content. It is straightforward to enforce lifecycles within SharePoint. Part of the implementation is included in best practices to define the lifecycle states, and part of the implementation is enforced within the definition of rules within each content type. A lifecycle is fundamentally separate from business process management, or workflow, and can be enforced across multiple repositories. Our experience is that a Draft, Work-in-Process, and Final lifecycle state model works well to enable SharePoint RM.

SharePoint Skills are a Major Issue
SharePoint skills are not exactly like ECM Skills. SharePoint implementation requires much more systems integration experience on the team than the implementation of the traditional ECM tools, such as Documentum, FileNet or Open Text. These vendors have spent years (decades) enabling the implementation of their products to be configuration tasks that require less systems integration than they did in the 1990s. We have found that the skills associated with the implementation of traditional ECM tools, particularly records management skills, are essential to a SharePoint RM project and that teams that lack these skills stumble as they try to learn about records management and implement it at the same time in SharePoint.

Site Provisioning is the Time to Implement Records Management within Sites
One of the most challenging elements we have seen in justifying the deployment of SharePoint 2010 for ECM and RM is that enterprise records management in SharePoint requires many elements to have been defined in advance to be consistently enabled and enforced across all sites. Furthermore, these capabilities are only useful if they can be easily integrated into the SharePoint sites that as they are widely and rapidly deployed in an organization. This requires that a site provisioning process is carefully governed and that the standards for site provisioning, including site administration, metadata management, and navigation are defined in advance. SharePoint RM is much more difficult and expensive to impose on sites after they have been put into production than when they are created. The best time to impose these standards is at the migration to a new version of SharePoint and the time to plan for this is now if you haven't migrated to SharePoint 2010 already.

Build your Business Case Now!
SharePoint is challenging to implement for ECM and RM. SharePoint may be free to an RM project because SharePoint ECM and RM capabilities are contained in the SharePoint Standard Client Access License (CAL) that have been acquired for the enterprise, but this can create the wrong cost expectation. SharePoint add-on components are expensive and SharePoint customization can also be expensive. A Business Case is critical to justify this investment and a Roadmap is important to defining how these capabilities will be implemented in a way that adds more value than they cost.

SharePoint 2010 will enable records management to finally happen for electronic content in many enterprises, because it is tightly integrated with Microsoft Office and because it reflects how users like to interact with their content.  We believe that, with careful planning and the implementation of best practices for SharePoint 2010 RM, the SharePoint Record Center will enable compliance and provide a trusted source of information for records management purposes.

Report

Add a Comment

You need to log in to post messages. Click here to login.

Comments

Mark Mandel

Submitted by Mark Mandel October 26, 2010 - 3:58 PM

Mike,

Now that I am an end user I have a wholly different perspective on things :-) While SP is an excellent tool I must question using it for records management, if it is your primary records repository. If it is a front end for a full fledged ECM/RM product, that's OK.

Here is my reasoning: For records management, your time horizon is 500 years or more for permanent records. Permanent for government is "for the life of the republic" so 500 years is a good minumum target. A records management system must therefore maintain records for a long time, and you must plan to migrate not only the records themselves, but also the metadata, file plans, retention rules, etc. periodically to new platforms.

SharePoint, in my opinion, really does not fit that model. Its life span for each release has, since it was first released, been only a few years. Users who built their records management system on the first release found that when the next version came out, virtually nothing would migrate.

I am concerned that by making SP your primary repository, you are risking being locked in to the current version, with limited choices when a new version comes out. Each version has more features than the previous one, and the kicker is that the underlying technology changes with each new operating system and development platform. As we all move to 64 bit OS and the next great thing beyond .NET, what happens to the records and the whole RM infrastructure?

I am interested to hear your take on this issue...
Report
Was this helpful? Yes No
Reply

Mike Alsup

Submitted by Mike Alsup October 26, 2010 - 6:05 PM

Mark -

Thanks for your comments. I respond to the main ones below:

1. Limitations on SharePoint as a front end for a full fledged RM product

I fundamentally disagree with this. I don’t disagree with using SharePoint as a front-end to IT systems and solutions. I disagree with using SharePoint as a front end for an RM product (such as IBM, EMC, and Open Text) product as a primary strategy for RM in an organization. SharePoint as a front-end for RM comes two primary flavors:

• SharePoint as a portal. This connects a SharePoint user to the RM resources of an ECM repository. While this sounds good in principle, it defeats many of the benefits of SharePoint. Data cannot be readily transferred between SharePoint applications and the ECM repository. The navigation is ECM navigation, not SharePoint navigation. It requires many ECM applications be constructed for the management of SharePoint records. It is also much more difficult to establish lifecycle controls over content. We see this as an interim measure while you figure out what you are really going to do to enable RM.

• SharePoint as a repository. LaserFiche, EMC and HP support External Blob Storage of SharePoint content integrated into an RM context within their ECM/RM solutions. Open Text supports the same process using alternative methods. In this context, SharePoint is governing the user experience and most of the metadata, and the ECM repository is governing the records management lifecycle of the content. This is a better approach that supports content types, but mainly it is better because it is letting the strengths of SharePoint be achieved. It also lets the single RM repository of EMC, HP, and Open Text span thousands of SharePoint sites. This is an appealing solution that includes the cost of multiple ECM repositories and some additional integration, but it is clear that it scales.

Here are the things that make SharePoint RM a more viable alternative than you suggest.

• Content types rule the school. Microsoft has built extensive governance capabilities into SharePoint content types. If you use SharePoint as a portal, you circumvent this opportunity for SharePoint governance. I believe that the approach of using SharePoint as a portal for an ECM back-end basically gives up on the governance of content within SharePoint because it is too hard to build the ECM applications and associated object models for all the SharePoint sites that would need them. This is what has kept records management from happening for so long. If you believe that SharePoint is of growing importance in organizations, then you need to use SharePoint content types.

• Lifecycle controls should be enabled within SharePoint. SharePoint is really good as a repository for draft and work in process content. When this content becomes final (and achieves “record” status) it needs to be placed in a repository of record regardless of whether it is SharePoint or another repository. This requires automated lifecycle controls within SharePoint, even if you wish that users will know that they should declare the record. Policy is not enough without enforcement.

• It takes significantly more work to implement multiple repositories than it does to implement a single one. We have built an enterprise-scaled SharePoint RM solution that is tightly integrated with one of the big three ECM solutions for a Fortune 25 company. A significant portion of our work was dedicated to synchronizing content type governance in SharePoint with file plan governance in the ECM repository. We believe that this work is required simply because there are multiple repositories.

2. Limitations on SharePoint as a file plan container

It is true that SharePoint 2010 was not designed as a tool to manage hundreds of content types and a thousand nodes in a file plan across thousands of SharePoint sites. This is a limitation of SharePoint that has been recognized (belatedly) by the Microsoft product team. If you are at the ARMA Conference next week in San Francisco, stop by and we will show you how an organization can achieve enterprise scalable file plan and retention schedule definitions within SharePoint. We have done a lot of work to figure out enable this in a scalable way. It is a core limitation of SharePoint 2010 out of the box.

3. Life of the Republic = 500 years.

That sure is a long time and is a harder requirement for me to digest. Generally, I agree that SharePoint is less mature than the ECM suites. On the other hand, there were probably 50 times more seats of SharePoint sold last year than any competing ECM product. So, the analogy I would draw for you is this. Would you bet that you could retrieve and read Microsoft Word documents, PDF/A documents, or Vydec or Lexitron documents in electronic form in 500 years? I think you go with the mass market acceptance of Word and PDF/A. But people like AIIM’s Betsy Fanning and Cohasset’s Dr. Charles Dollar know way more about this topic than me. If the content is in an archival format (like PDF/A) than the question is whether you trust your ability to migrate data forward in SQL Server. We have migrated Documentum RM 5.2.5 records content forward to Documentum 6.0 and this was also challenging.

4. Locking into the current version of SharePoint

I am not sure what this means exactly, but I get that it is challenging to migrate records in MOSS 2007 to SharePoint 2010. It is critical that you:

• Plan and architect a SharePoint RM implementation so that the components are not spread far and wide across SharePoint.

• Centralize the administration of both the file plan and the repository of record so that the assets of the organization can be protected.

• Establish policy and process so that SharePoint is governed.

I just don’t get why this is more impractical in SharePoint 2010 with the implementation of best practices and a framework that enables the full governance of content within SharePoint. Do not accept that SharePoint is just a collaboration, portal and WCM platform. Hope to see you in SF.
Report
Was this helpful? Yes No
Reply

Robert Bailey

Submitted by Robert Bailey October 27, 2010 - 9:58 AM

The concern I have is that some organizations which are using an ECM repositiory like TRIM are going to either have to develop a complex process in using SharePoint or will use them independently and not have all the full benefits of either. There organizations that will not be willing or able to bridge the gap.
Report
Was this helpful? Yes No
Reply

Mark Mandel

Submitted by Mark Mandel October 27, 2010 - 10:07 AM

I use SP - and am advocating its use this way - as a temporary solution for agencies if they don't have the time or budget to move to the enterprise ECM now. Your point above about lifecycle management is what I am promoting as a key benefit. Use SharePoint for working drafts, for collaboration, then sweep the final versions into the primary ECM RM repository.

See more about how I am currently using SharePoint at http://www.aiimcommunities.org/erm/blog/sharepoint-musings.

The key benefit of SP in my opinion, as an end user, is that it is essentially free. In times of budget constraints and resource challenges, standing up a SP site is very inexpensive, especially if you just use the out of the box features. The challenge then becomes how to make sure your organization uses it in a best practice manner.
Report
Was this helpful? Yes No
Reply

Mary Lynne Nelson

Submitted by Mary Lynne Nelson October 27, 2010 - 12:39 PM

I agree with Mike and Mark that Sharepoint has already won, and the Sharepoint 2010 has RM features that are actually usable (unlike prior versions). However as a long term Sharepoint user, in the trenches, there are still some key issues that I think make Sharepoint less-than-stellar as an RM solution (and I say these based on pre-implementation research into 2010, not active use - yet). These seem tactical but really are fairly major at least in our environment.

The first isssue is that Sharepoint continues to use the upload date of the document as the create date, and there is no easy (or Microsoft-supported) way to change this behavior. (Our developers have tried). Moving a document from one library to another also modifies the create date. Managing records over years/decades requires that the create date of the record not change just because the storage location is altered, or the record is moved from one repository to another.

The second, related, issue is that the document metadata is not maintained separate enough from the document itself, meaning that if I edit metadata about a document, it changes the last modified date of both the document and the metadata. This prevents me from editing the metadata about a record without also making the record look as if it was modified too. In SP 2007 and earlier, marking a record uneditable (read-only) also prevents editing of the document's metadata, which in turn prevents us from tracking records management activity of the record over its life (changing active record steward, for example, or current Office of Record). If that behavior persists in 2010, we'll be looking to our Records Management system to backend most of our Sharepoint sites that require long term auditable RIM solutions. Proper records management simply requires that metadata about a record be alterable, while maintaining proof that the record itself has not been altered.

My third issue is more one of Microsoft intent than Sharepoint capability, and that one is based on Microsoft statements made at AIIM about compliance with standards such as DoD and MoReq - basically, Microsoft doesn't intend to actively pursue Sharepoint compliance with these standards, instead driving development from "stated customer needs." While the marketing logic here is obvious, the loss of decades of good work in creating ERM system standards is upsetting to say the least.

I like the single repository idea - a lot - but in practical terms we have to be able to manage records, as well as non-records, across many platforms. I'm glad to hear you successfully integrated content types and file plans between Sharepoint and an ERM system, because while integration isn't easy, I think for the foreseeable future it is going to be necessary.
Report
Was this helpful? Yes No
Reply

Mike Alsup

Submitted by Mike Alsup October 27, 2010 - 4:04 PM

This note responds to multiple questions that have been posted about my article. Mike Alsup

Mary Lynne Nelson made several points…

1. The first issue is that SharePoint continues to use the upload date of the document as the create date and there is no easy way to change this behavior. Moving a document from one library to another also modifies the create date. Managing records over years/decades requires that the create date of the record not change just because the storage location is altered, or the record is moved from one repository to another repository or system. The second issue is that the document metadata is not maintained separate enough from the document itself. This prevents me from editing the metadata about a record without also making the record look as if it was modified too.

Response - True, create date is system maintained automatically and the document metadata is not separate enough from the document. DoD 5015 requires and uses separate attribute for this purpose. This is the work-around that needs to be implemented to address both issues, regardless of whether it is a DoD 5015 certified system or not.

2. Proper records management simply requires that metadata about a record be alterable, while maintaining proof that the record itself has not been altered.

Response - We agree that this is an issue are actively working with clients to address it. Our approach is to trap and synch any changes to content separate from metadata.

3. Microsoft doesn't intend to actively pursue SharePoint compliance with these standards (DoD and MoREQ2), instead driving development from "stated customer needs." While the marketing logic here is obvious, the loss of decades of good work in creating ERM system standards is upsetting to say the least.

Response - There is broad recognition in Redmond that certification and standards compliance is a requirement for some customers, and firms like Gartner require DoD 5015.2 certification to list the product as an enterprise RM solution. There is a session at ARMA on Monday on the full enablement of capabilities within SharePoint 2010 to enable DoD 5015.2 certification.

4. Mark Mandel wrote… use SharePoint for working drafts, for collaboration, then sweep the final versions into the primary ECM RM repository.

Response - Best practice enables users to retrieve documents regardless of their records status through a single interface. We believe that this will be SharePoint in many cases. Why should a user need to know if what he is searching for was declared as a record? Reliable record determination comes down to maintaining a lifecycle and reliable records management metadata through the life of a document that you can count on. This makes the records management dimension much easier to deal with, regardless of whether SharePoint or another ECM repository is the repository of record.
Report
Was this helpful? Yes No
Reply

This post and comment(s) reflect the personal perspectives of community members, and not necessarily those of their employers or of AIIM International