Example for reviews on Amazon

One of key factor for the success of a web portal is the quality of the content offered. Editorial content usually goes through a predefined quality assurance process in order to fulfill the quality requirements of an organization. Unluckily, things get a bit more difficult to control when it comes to user generated content. In this case the spread of content quality may be very high depending on the environment. In order to ensure the success of a web site based on user generated content, effective measures need to be taken to decrease the impact of these quality variations.

This blog entry is the first of series of articles dealing with concepts aiming to improve the quality of user generated content. To begin with, I will focus on social design patterns established and proven on community centered web projects.

In general a pattern describes a proven solution to a common problem within a specific context. A social design pattern in particular is an interaction pattern which deals with a specific problem in the context of social interfaces. A pattern which can be directly applied to the content item itself is the feedback pattern.

Below I evaluate the different variants of the feedback pattern regarding their chances and risks for content quality. Most patterns have been taken from Designing Social Interfaces, which is a great source for social design patterns and actively extended by a growing community of contributors.

Comments

A comment is directly related to a content item whereas virtually any type of content may be commented (e.g. photos, videos, blog entries or other comments). It provides an easy and established way for users to textually state their opinion about the item they are consuming.

• chance: Comments enhance content with further information, help to correct wrong statements, provide user knowledge and experience, encourage and motivate. They also indirectly indicate an item’s popularity by the number of posts per time unit. This information later allows filtering relevant content by activity.
• risk: Comments bear the risk of wrong, inappropriate and insulting statements or spam. They only allow indirect automatic measurement of the related item’s quality by its popularity.

Examples:

Ratings

A rating allows a user to quickly and effortlessly state an opinion about an object with a minimal interruption of the workflow she is currently in.  This way, ratings guarantee a low barrier for user interaction (depending on prerequisites like user registration). To obtain more detailed information a rating may be split up into sub-ratings (e.g. one for each object property) which allows different levels of granularity (see yahoo).

• Chance: Ratings are simple though flexible. Due to their minimal impact on a user’s workflow, ratings allow to collect lots of data which helps to reduce statistical spread. The gathered information can be used for further automatic processing and allow later filtering and sorting (e.g. by overall rating of a product, rating of a review and the like).
• Risk: When used without a proper explanation or within a wrong context, ratings may be misleading. Ratings related to a person may cause resentment and hence should be used carefully.

Examples:

Reviews

A review usually comprises a detailed statement about a virtual or real live object (e.g. a place, person, thing or service). This statement may include experiences made by the user, a personal opinion or a detailed analysis of an object’s properties. Usually reviews are combined with ratings and may themselves be rated by other users.

• Chance: Ratings allow collecting valuable content which is appreciated by other users (e.g. books, electronic devices, holiday destinations and much more).  They add further information to a reviewed object which can even be enhanced by the combination with the rating pattern. encourage better reviews through rating and reputation
• Risk: Like comments, reviews are prone to disinformation and spam. The subjective component of a rating can never be excluded. Trying to cover too many different aspects of the object to be rated may lead to forms which are too complicated.

Examples:

Favorites

In order to mark an item for later retrieval, a user may add it to the list of favorites. Clicking the add to favorites icon, which is usually in close proximity to the content item, allows to easily flag an item for later retrieval. This pattern is often extended by a field which allows adding keywords or tags for organization and later retrieval.

• Chance: The favorites pattern provides an easy way for users to collect relevant content items for later processing. Although it does not reveal anything about a contents quality first hand, it may at least give an indication through the number of users who added an item to their favorites. By sharing favorites or top lists, user interaction with the respective content items may be further stimulated.
• Risk: The low impact on the overall end user workflow may lead to an overloaded favorites list which can make it difficult to find relevant content later.

Examples:

Vote to promote

This pattern is used to bring forward a particular piece of content in a community pool of submissions. A promotion takes the form of a vote for an item, and items with more votes rise in the rankings to be displayed with more prominence.

• Chance: This pattern provides an immediate representation of the voting action through the position in the ranking list. It’s low impact on the superior user workflow permits lots of participations which reduces statistical spread.
• Risk: The representation through a ranking list may cause, ranking- not content based voting. Higher ranked items may attract more attention and hence receive more votes; others may disappear into the long tail of the popularity ranking list. The community members may try to game the system for whatever motivation or the rating community may not be large enough for significant feedback (although the pattern itself encourages participation). Without proper measures users may vote for items they did not consume.

Examples:

Thumbs up/down

Users are enabled to express a like / dislike (e.g. love/hate) type opinion about an object (person, place or thing) they are consuming / reading / experiencing by applying this pattern. It’s simple and intuitive enough to quickly grab a user’s view about an object.

• Chance: The benefit to other users is that these ratings, when assessed in aggregate, can quickly give a sense for the community’s opinion of a rated object. They may also be helpful for drawing quick qualitative comparisons between like items (this is better than that) but this is of secondary importance with this ratings-type. It may also encourage discussion among the community users by polarization. Highlighting items with a certain number of positive/negative votes may improve later filtering of content.
• Risk: The lack of intermediate choices may lead to mood based decisions. The black/white character of this pattern does not provide the fine granularity of ratings and the intrinsic polarization may cause conflicts within the community. A low number of ratings may cause insignificant and misleading results (e.g. one thumb up may lead to 100% positive rating). Finally the symbolic representation of up / down may be misleading or not understood by different cultures.

Examples:

Bury / flag as inappropriate

Although not separately listed by Designing Social Interfaces I find this pattern important enough to mention it separately. As a special case of the patterns mentioned above it allows the community to deal with unwanted content like spam, inappropriate, off topic, insulting or illegal items. Reaching a predefined threshold of user flags may indicate the site administrator or moderator to evaluate and if necessary remove item. Less critical cases could lead to hiding or moving an item. In order to simplify later evaluation an option to select the reason for flagging may be provided

• Chance: Applying this pattern allows to filter out unwanted content and increase overall content quality as well as the sites reputation.
• Risk: Good content may be buried and automatic processing may lead to loss of content items. Especially on large portals with a broad target group heavy workload for moderators may be caused.

Examples:

Soliciting feedback

Related to the comment pattern soliciting feedback allows to collect user feedback about the site itself. As a pattern which is familiar to most web users it is one of the most important feedback sources for site owners.

• Chance: Textual feedback may allow site owners to react and take measures in order to guarantee the quality of site and content.
• Risk: Without suitable precautions the site administrator will be covered with spam. The community could be heavily disappointed if the site owner does not respond to input or takes proper actions. Moreover the community may claim too much voice when it comes to defining the portals future direction. In both cases users might move to other portals.

Examples:

Conclusion

When applied properly, feedback patterns provide an effective way to increase the overall content quality of a community portal. They

• motivate users to create above average quality
• encourage users to add further information
• provide quality measures
• help separating good from bad content
• allow sorting and filtering by different quality attributes
• help to control inappropriate content

The list above is not complete and may grow with further evolving feedback patterns based on new concepts in the field of UGC. The review is based on my personal experience with this topic und I would appreciate your comments and feedback.

Audi Facebook fan page

Social media and user generated content are going through a paradigm shift right now. But it’s not one of these big revolutions we’ve seen with YouTube or Flickr. It’s more subtle and some kind of a logical consequence of existing principles.

The early days of social media were all about the quick experience related to media generation and consumption. Viral effects unleashed powers unknown so far and hence got a fundamental marketing tool for brands. In the age of UGC businesses start to learn about the deeper value of customer reviews, feedbacks, discussions or reports. This special type of UGC provides precious information for a company which has mostly been ignored in the recent past.

A good example of a company harnessing the power of their customers is Audi which, for this reason, has set up a dedicated Facebook fan page. Besides providing background information about the company, the cars and motorsports, Audi tries to engage users by also providing a game called coast to coast rally.

The exciting part in terms of UGC and customer interaction is how Audio tries to extract knowledge and experience from their enthusiastic customer base. The following examples have been taken from Audi’s Facebook fan page:

• Audi design challenge: engage fans to contribute to Audis participation in the “Youth Mobile 2030” design competition
• Discussion boards: have been around for a while, but still deliver deep insight into customer opinions and experiences
• Polls: a very flexible and simple way to collect customer feedback without costly preparations

Although the examples mentioned above are not based on completely new concepts, it is the dimension that’s so impressive. By involving hundreds or even thousands of users into generating content, the amount of top value information will also increase. It’s up to existing businesses and new startups to find appropriate business cases monetizing this new kind of good. What’s still unsolved for me is the question on how to filter relevant content from the masses of contributions created under such circumstances.

Components and processes related to user generated content

In my previous post I described the relationship between different user generated content types. In order to make things more comprehensible I decided to create a diagram and adapt the naming schema. The diagram provides an overview on the most common content types and their relation to each other.

The following data types are covered by the diagram:

Simple (formerly Primary): Simple content items can be seen as the basic building blocks of UGC. They are usually created by using a dedicated device (e.g. digital camera or mobile phone).
Complex (formerly Secondary): Complex content items are made of simple as well as other complex content types. In the majority of cases different data sources are used to assemble this type of content.
Meta (formerly Tertiary): Applying meta data to simple and complex content items generates additional value and improves further processing.

In the diagram I try to visualize the relationship between these three content types in terms of UGC. As there are virtually infinite combinations for complex content types I just took a limited set to demonstrate the concept. The colored squares illustrate different simple content items. Each simple content type is characterized by a separate color tone. This way one can identify the main building blocks of a complex content item.

The ratios between the different simple content types which compose the complex ones are not based on any statistical data – in fact they just reflect my personal perception.

The three basic content types listed above are related to each other in different ways:

Create: complex content items can be created by assembling simple or complex content items to new ones.
Decompose: in order to process and comprehend complex content items they need to be decomposed and relevant information needs to be extracted (e.g. read the description of a diagram).
Apply: meta data is applied to simple as well as complex content types in order to increase their overall value and improve search, filter and organization capabilities.
Filter & Search: meta data can be used to organize content in order to look up or find it later.
Organize: content items may be organized by assigning them to a tree structure or taxonomy or by tagging them.

Conclusion

Creating this diagram helped me to better understand the basics required to define a thorough content structure for scaleblog.net. I’m convinced that the data structure of a system is the most important part. Errors done here in an early planning phase are very difficult to be fixed later. Hence I plan to spend sufficient time to define the entire data structure of scalebook.net. As a matter of fact I might not be able to consider every detail of the data model, hence I will also try to plan for extensibility.

Frequency of activities realated ot UGC based on age groups

A Forrester Research study published by NewsWeek examines the relationship between UGC activities and age groups. The study splits the relevant activities into six different categories which are defined as follows:

Creators: publish web pages, write blogs, upload videos to sites like YouTube
Critics: comment on blogs and posts, ratings and reviews
Collectors: use really simple syndication (RSS) and tag web pages to gather information
Joiners: use social networking sites
Spectators: read blogs, watch peer generated videos and listen to podcasts
Inactive: are online but don’t yet participate on social media

Unluckily the group of content Creators has been defined rather vague in terms of content source. Content creation is not only limited to the production process of content but may also include mixing and filtering. Hence it would have been interesting to increase the granularity of the study by introducing two further groups:

Mixers: take current pieces of content, reassemble them and create additional value

Distributors: search for cutting-edge, high quality content items and distribute them to peer-groups

Review

The most creators can be found in the age groups of Young Teens (12 to 17) and Youth (18 to 21). This may have two reasons. First content is mainly created out of emotion in these groups. Increasing handset capabilities and the availability of appropriate services and infrastructure further push emotion based content creation. Second due to the emotional aspect of creation, content is rarely going through any quality or review processes. This leads to an increased rate of content creation among people aged between 12 and 21.

Critics tend to be older (18 to 26) as they start to care more about content quality and authenticity. They question the content distributed and are more likely to correct wrong facts or state their opinion. The groups of Youth and Generation Y scrutinize content which results in more reactions like comments or ratings.

Although the number of Inactives is very high for Boomers and Seniors I expect this fact to change within the next decades. These groups will also learn to deal with user generated content and the aging Generation X will take it for granted anyway.

Consequence for scalebook.net

Scalebook will target the age group of Generation X and Boomers which are not as active as other groups regarding content creation. Hence it will be interesting to see which measures will be the most efficient ones to encourage them to create and post content.

What makes thinks even more challenging is the fact that the target group of modelers seems to be very reluctant when it comes to changing portals and services. I didn’t yet figure out if this is a consequence of most modelers being Generation X or above.

Creating a vivid, dynamic poratal for modelers would require an accurate selection of content types. In order to achieve this I started to do some research on UGC related use cases. The aim was to collect typical use cases around UGC and later see how they match with the model makers needs. Based on the collected use cases I plan to extact the most relevant content types for the modelers’ portal.

The list of use cases grew pretty fast and I was amazed how diverse the processes around UGC could be. The mor or less organized result of my investigation is shown in the list below. It contains mainly keywords indicating the related use case. I’m sure it’s far from being complete and I missed a lot of important items but it will serve as a starting point for me. What I failed doing for now was to bring those key words into some kind of structure that would provide additional information (e.g. grouping them by categories like entertainment, education, research). Maybe you have some interesting ideas here.

A definition

So what is UGC? In terms of the list below UGC is every piece of data created by an individual or a group that may provide a value to another. It is usually created by amateurs and available for free.

So what is UGC actually used for?

…, news, gossip, reports, stories, articles, how-tos, customer reviews, previews, knowledge, experience, research results, notes (personal and public), resumes, forums, q&a, mailing lists, comments, ratings, tags, geo information (tracks for running, biking, sailing…), ad hoc authoring (wikis), lists ( play lists for audio or video tracks, link lists), books, documents, presentations, spreadsheets, applications,  plugins, extensions, widgets, games, levels, mods, textures, pictures, galleries, graphics, videos, audio tracks, mashups,…

As described below the process of content creation and content consumption is closely related to the hierarchical structure of content types.

Content Hierarchy

Having a closer look on typical web content reveals a hierarchical organization of content types. Three types of content can be distinguished:

• Primary content: This type of content is the base for all other content types and cannot be split into smaller subtypes. Typical primary content types are texts, videos, photos, graphics, charts, audio files and the like.
• Secondary content: Every combination of primary content which is treated as self contained entity makes up secondary content. Examples for secondary content are blog entries, wiki articles or galleries. Mashups are a generic example of secondary content.
• Tertiary content: This type of content usually comprises small pieces of information which add further value to primary and secondary content. Tertiary content examples are tags, flags, ratings, hierarchies, taxonomies or geo data.

Content Creation

The process of content creation is a bottom up process within the hierarchy of content types beginning at the level of primary content.

1. Creating Primary content: writing texts like articles, reviews, comments, product descriptions, taking photos and videos or recording audio tracks
2. Creating Secondary content: assembling different entities of primary content in various ways to create additional value (e.g. a text describing a chart)
3. Creating tertiary content: attaching small pieces of information to generate additional value with regard to content organization and retrieval; typical activities to create this form of content are tagging, flagging, organizing, rating, geo tagging or adding any other kind of adding metadata

Content Consumption

In contrast to content creation the process of content consumption is a top down process within the hierarchy of content types beginning at the level of tertiary content.

1. Content retrieval: Whenever a given content item needs extracted from a large pool of other content items this can be achieved with the aid of tertiary content type items (e.g. search by tag, order by rating).
2. Scan content: Secondary content can be further consumed by decomposing and scanning for relevant primary content items.
3. Consume content: The intrinsic process of content consumption is located at the level of primary content. Usually we consume this kind of content by reading, watching or listening.

Conclusion

I’ve been evaluating various web content management systems (amongst them well known products like Drupal, Joomla or TikiWiki)  in order to find a solution which is sufficiently serving content creation as well as content consumption for a large community. So far I can say that even by enhancing these web content management systems through further extensions I was not able to prototype a solution which would meet my requirements.

I think it’s crucial for web content management systems to adjust their design towards the concept described above in order to allow a seamless integration of content creation and consumption. Currently assembling secondary and tertiary content is not sufficiently covered by web content management systems. The developer communities of Joomla and especially Drupal are trying to address this topic with separate modules to create complex (e.g. Drupals CCK) and hierarchical structured (e.g. Node References in Drupal) content types. Nevertheless the process to setup and maintain these content types is currently insufficiently supported in terms of architectural design and usability and hence not practicable for creating structured content types on a large community level.

That’s weird, I thought to myself. This guy spends so much time on building stunning models, writing articles, taking pics and giving tips without being given any reward (apart from reputation of course). I gave him a call that night and told him about the idea of having an own blog, which I would set up for him. Klaus was excited and agreed instantly.

One day later we had another talk about that idea and discovered one big drawback. In order to keep the site interesting for the visitors there needed to be content updates in short intervals. Due to the fact that Klaus was building only one or two models per year his fans might get bored sooner or later and drift back to the bulletin boards and portals they came from. In order to make sure that contributions are posted on a regular basis we would need more authors. But even though Klaus has a large circle of friends also being passionate model builders and hence potential authors for the blog I was not sure if we could maintain a portal that was active enough to make people visit the site regularly.

This led me to the next level of my vision. What if we would not only have a few authors posting on the site but instead leverage the power of a whole community? It’s obvious that the average quality of the content being posted would suffer if everyone is allowed to contribute. But with an increasing number of members the absolute number of excellent contributions would increase too and therefore attract more visitors to the site. The crucial point is to separate the wheat from the chaff by providing the optimal features to evaluate and retrieve content.  In a nutshell my vision can be described as implementing a portal for modelers that features all the tools necessary to create a vivid community which is entirely managed by its members.

The gap between vision and reality can be a big one. It didn’t take me long to understand that. After a few days of research I realized that it would take more than a simple CMS with some extensions to implement my idea. I would need to detail the requirements, do market research and technology scouting, implement a sophisticated social web content management system, generate initial content and of course draw visitors to the site. It seems as if I’m at the beginning of a long and stony road where throwbacks lurk behind every corner. This is why I decided to write this blog. It will be a companion which will help me to cope with frustration, share my experience and hopefully attract you to come back again and see how I’m doing…