The Truth About Folders

Community Topic(s):

Keywords: Taxonomy, Folders, Sharepoint, Content Management

Current Rating:
(0 ratings)

Chris Riley wrote a post last week here at AIIM, complete with video, against folders. He gave lots of reasons and explained many problems, but he ignored the positives. I feel his opinion is tainted by his prolonged exposure to SharePoint. I’m not saying that the points he brought up are wrong, just that there is a flip side.

Some Background

The folder debate is old. I remember having these discussions back in the 90s, I discussed the folder v search debate in detail four years ago, and I expect we are going to have this discussion for decades to come unless I save myself and change careers. There have been two reasons that folders have endured, people and technology.

People are used to folders. They used them in DOS and then Windows. Before “My Documents”, people were creating their own mechanisms to save and track their documents. They use them in their email systems now to track things. Sure, younger people may prefer and depend on folksonomies, but there are still two older generations in the workforce. It is all about adoption. Chris Walker shows the difference simply in one of his previous posts here at AIIM.

As for the technology, it fails. Search engines are typically the hardest part of the system to optimize in any Content Management System. You can bypass that by using database fields, aka attributes, but as you increase volume, you have to make plans to scale up.

These problems will shrink every year, though they aren’t in any danger of being eliminated. Let’s talk about the positives.

Folders Help

Did you know that in most systems, content can reside in multiple locations? This means that if I think a piece of content belongs in one location, that doesn’t stop someone from placing a link in a second location, or a third.

Chris talked about taxonomies. He actually lost me there because taxonomies are most often implemented as a folder hierarchy. A well-executed taxonomy will automatically place content in all relevant areas. This allows content to be found through the concepts that are applicable when needed. While you can implement them without using folders, you are sacrificing performance and simplicity.

Of course as times change, the folder structure should change as well. I keep track of things and as my content changes, I modify my structure. Of course, prior to SharePoint 2010, when you did that you broke all links to that content. This isn’t a problem in 2010 or in traditional Content Management systems.

One of the problems that you get when you don’t use folders is that you can cripple most systems. While few systems claim a limit to the number of documents that can reside in one location, there is a practical limit. User interfaces become unhappy and some systems actually slow down as a whole because of how they implement document containers. You can swear that nobody will ever browse to that location, but unless you remove that capability, someone will do it. Heck, administrators like to browse to content to see if there is a problem with the search functionality.

Meanwhile, people can still search for the content to their hearts content. Folders don’t stop that. Sometimes you don’t know what you are searching for in detail, so metadata and full-text search is crucial.

Until search engines can read your mind reliably and always give you what you need, use folders.

For more reading, I highly recommend the post I wrote four years ago, referenced above, and titled Taxonomies, Good, Bad, or Ugly? It holds up now and I suspect it will hold up for a decade or two.

Report

Rate Post

You need to log in to rate blog posts. Click here to login.

Add a Comment

You need to log in to post messages. Click here to login.

Comments

Chris Riley, ECMp, IOAp

Ouch!

Laurence,

Great post. A lot of people have used the same comment you did above about Taxonomies and folders being synonymous. Technically they are not. Again folders are part of the file name of a document, where as a taxonomy, at least implemented properly is apart of a documents meta-data. Meaning that taxonomies are not tied to storage location of a file, where folders are. So while folders behave often 100% the same, they are not taxonomies. Folders can help build a taxonomy, folder structure can even be replicated by taxonomy ( not best practice ), still does not make them folders. This point is why I created the next blog post http://www.aiim.org/community/blogs/expert/If-you-have-OCD-you-love-Taxonomy!

The other point, "tainted by SharePoint". Actually the reason I wrote the post was out of frustration of SharePoint clients creating too many folders in SharePoint libraries. The typical SharePoint user loves folders. Be default in SharePoint you can create them, unless you explicitly check the setting not to. So I do not think this is the case.

Folders unlike taxonomies are generated organically. This makes a big assumption that users actually put thought into how they organize content. I would argue that they did not. Maybe the AIIM member puts more thought into their folder structure, but the average knowledge worker is doing it on the fly. This spells disaster and why share drives are not a proper replacement for ECM systems.

And finally, it's not all about search. Search is not the only way content is gathered. I would prefer someone to be able to more quickly drill into content with a good taxonomy then rely on search, although that is a major piece.

My point is that folders served their purpose, but are extremely limiting. Meta-data solves that problem. Why fall on old crunches when we have modern technology to improve what we do?
Report
Was this helpful? Yes No
Reply
Laurence Hart

Close, but not quite

Riley, if the Folder name is part of the document name as you espouse, then nobody taught you how to use folders. It is true when you are just sub-dividing invoices to reduce clutter, but not true in mature Content Management systems.

Yes, folder structures tend to be very organic, but you don't just give users a blank slate. You create an overriding Information Architecture. You let it be dynamic below a certain point as you can over-engineer taxonomies and structures. Even in a taxonomy, there reaches a point where a small subset of people organize it as they see fit. Things change and if you give them the tools, they can structure things quite readily.

I agree that taxonomies are not necessarily equivalent to taxonomies. What I have seen is that successful implementations allow people to drop things directly into a folder and quickly add a few more details. Even in a "taxonomy", those names/categories act as metadata for documents residing there. When it comes to execution, the two concepts are tied at the hip.

Search is not just full-text search. If you don't implement a taxonomy with folders, which I don't advise anyone, then you are dependent on search within the underlying system to pull the information. Folders do more than provide a visible structure to the taxonomy, they provide a technical viable way to browse the taxonomy. Without folders, search/queries are the navigation tool.

Folders are not limiting. They drive adoption. Metadata has been around for decades and we still use folders. Why? Because users don't fill out metadata. They may upon initial roll-out and OCD people will fill out any field you present to them, but most don't. You can't force them to fill it out either. If you do, they start storing the content on their desktop or in a basic file store in the cloud like Box.

The tech hasn't changed significantly in years. It is faster, but we have more stuff as well. Tags is about the only innovation that I have seen in IA in the past 10 years and that is flawed in execution.
Report
Was this helpful? Yes No
Reply
Chris Riley, ECMp, IOAp

Missunderstood

Technically, to the file system folders are apart of the file name. When you call the object model for any file in the windows API, the full name of the file is the entire folder path plus the name plus the extension. There is no way to argue this. I think you might have been confused by what I was saying. It's a purely technical difference. When you move a file from one folder to another, the file system calls the method "rename", you can test this yourself. Being that that is true, it would be better to have the hierarchy not tied to the name of the file but in meta-data i.e. a taxonomy term.

Explorer view in SharePoint also drives adoption. But bad adoption. It forces users to use SharePoint as a Share Drive. Might as well do nothing at that point.
Report
Was this helpful? Yes No
Reply
Laurence Hart

Microsoft isn't the World

You have to pull yourself out of the world of Microsoft. In actual Content Management systems, it ISN'T part of the name. I don't care what an operating system does, it is what is done in Content Management systems that matters.
Report
Was this helpful? Yes No
Reply
Chris Riley, ECMp, IOAp

When we start to agree

Agree. But this is where the majority of the world is stuck, why we AIIM community exist. So maybe the real post is. Get your head put of your share drives.
Report
Was this helpful? Yes No
Reply

Sean Hederman

A contrary view

I'm afraid that a lot of your points display a unfamiliarity with search-based systems. I've posted a more detailed rebuttal at (http://blogs.palantir.co.za/signate/archive/2011/07/06/the-truth-about-folders-a-rebuttal.aspx).

Disclaimer: I work at a vendor of a search-based DM system.
Report
Was this helpful? Yes No
Reply
Laurence Hart

Rebutting the Contrary View

Sean, I wrote a rebuttal to your rebuttal of my rebuttal of the original post. It is located on my primary blog http://wordofpie.com. (http://wp.me/p4OLk-nc)

Report
Was this helpful? Yes No
Reply

Laurence Hart

Rebutting the Contrary View

Sean, I wrote a rebuttal to your rebuttal of my rebuttal of the original post. It is located on my primary blog http://wordofpie.com. (http://wp.me/p4OLk-nc)

Report
Was this helpful? Yes No
Reply

Daniel O'Leary

Taxonomy and Hierarchy, living in perfect harmony

This is where the world of ECM and WCM really start to separate. When I look at my own folders (yep) in Box.net, that is my primary method structure and makes up a nice chunk of my taxonomy. Things like metadata are all there, but really the structure matters, especially to people who are browsing and need to discover content, not search for it.

Now contrast that with the WCM world, where WordPress runs all of the websites I work with. In that realm, folders don't exist, and content is all in a flat hierarchy. Using some plugins for sorting and search, that also works for me.

Neither one is perfect, but both works really well for managing data, and my guess is most people are happy with it just working more so than how or why it works.
Report
Was this helpful? Yes No
Reply
Laurence Hart

Brilliant!

The business problem drives the IA, not a preconceived notion of which one is correct. I have solutions out there that don't use folders, but most general-purpose DM and KM solutions rely on them heavily. Folders aren't always the answer, but they aren't something to be shunned.

For the record, it isn't where ECM and WCM diverge. They do that several requirements back. :)
Report
Was this helpful? Yes No
Reply
Chris Riley, ECMp, IOAp

Solutions

Discovery and planning is key. But I've never had a solution without taxonomy. But to-date I've never had one without folders at some level. User imposed. This is my frustration.
Report
Was this helpful? Yes No
Reply

Chris Walker

It's Still Metadata

In a proper content management system (e.g.: the one shown in the post that LH refers to), folders are a virtual structure, have nothing to do with the file name, and are actually one of the metadata properties.
Report
Was this helpful? Yes No
Reply

Barclay Blair

Yay, people disagreeing!

So happy to see this. Our community faces some really, really big problems, and I think the only way through them is to have frank discussions that challenge our dogma.

As for the issue itself, I don't see a lot of disagreement between Chris Riley and Laurence, at least in what they have written. Folders as an organizational metaphor are useful (and not just because they are familiar), and will continue to be. However, when implemented in a "physical" or inflexible way, they limit our ability to present and use information.
Report
Was this helpful? Yes No
Reply
Chris Riley, ECMp, IOAp

Good point

It's not about weather to organize or not. That is what I was trying to say at the beginning of the "Folders are the new F word" post. It's more about how limiting they can be.
Report
Was this helpful? Yes No
Reply

joseph martins

Not mutually exclusive

It's simple really. Too many keystrokes and clicks (too many search options or folders) and too much scrolling (too many items in a folder or search results) will always frustrate employees who care about efficiency and time management. Their preference is for the tool(s) that enable them to find content most quickly and accurately.

I've seen both implemented very poorly and very well. And to-date I use a combination of the two. Only the most important files, bookmarks and such are placed into folders in a very lightweight hierarchy I created long ago. Everything else goes into a general bucket and search is my tool of choice. I suspect I'll continue in this manner for a long time to come.
Report
Was this helpful? Yes No
Reply
Laurence Hart

Exactly!

My post was in defense of folders, not against search. It may sound that way, but I was trying to point out why you can't abandon folders.

On a personal level, I use search a lot. I also use folders. Folders help me store and retrieve. Search helps me find unknowns.
Report
Was this helpful? Yes No
Reply
Chris Walker

Bingo!!!

You and bajillions of other users.
Report
Was this helpful? Yes No
Reply

Chris Riley, ECMp, IOAp

root containers

There is not question that some level of root container will exist. You could argue that the ECM system itself is one huge folder. I advocate to my clients to be as flat as possible with lots of meta-data unless you can clearly justify otherwise. Thank you for your comments!
Report
Was this helpful? Yes No
Reply
Laurence Hart

One size doesn't fit all

Chris R., dangerous approach. Not saying it doesn't work, but using the same strategy doesn't work. Business needs drive different Information Architectures. Lots of metadata is dangerous if you can't automate the completion of that metadata. I have one client with 30+ fields but only 5 are manually completed, and that is sometimes a challenge.
Report
Was this helpful? Yes No
Reply

Eric Mullerbeck

Good discussion!

I'd like to add a couple points 'pro' and 'con' the use of folders.

Pro: folder selection can be leveraged to apply metadata. Users may be more likely to choose from a set of folders than they are to fill out a set of fields, but as long as the folders are well-structured, they can be used to apply metadata. The value of this approach is to potentially make content more easily discoverable by other users who are using search tools, for which the metadata could be applied (via relevance ranking, or filtering).

Con: Folders are great for quick filing of content in small groups of users. But as the size of the user group increases, I find that folders become an increasing obstacle to transparent information sharing. They work best on small scales among users who know each other.
Report
Was this helpful? Yes No
Reply

This post and comment(s) reflect the personal perspectives of community members, and not necessarily those of their employers or of AIIM International