Design for UGC quality – Feedback patterns

Example for reviews on Amazon

One of key factor for the success of a web portal is the quality of the content offered. Editorial content usually goes through a predefined quality assurance process in order to fulfill the quality requirements of an organization. Unluckily, things get a bit more difficult to control when it comes to user generated content. In this case the spread of content quality may be very high depending on the environment. In order to ensure the success of a web site based on user generated content, effective measures need to be taken to decrease the impact of these quality variations.

This blog entry is the first of series of articles dealing with concepts aiming to improve the quality of user generated content. To begin with, I will focus on social design patterns established and proven on community centered web projects.

In general a pattern describes a proven solution to a common problem within a specific context. A social design pattern in particular is an interaction pattern which deals with a specific problem in the context of social interfaces. A pattern which can be directly applied to the content item itself is the feedback pattern.

Below I evaluate the different variants of the feedback pattern regarding their chances and risks for content quality. Most patterns have been taken from Designing Social Interfaces, which is a great source for social design patterns and actively extended by a growing community of contributors.

Comments

A comment is directly related to a content item whereas virtually any type of content may be commented (e.g. photos, videos, blog entries or other comments). It provides an easy and established way for users to textually state their opinion about the item they are consuming.

• chance: Comments enhance content with further information, help to correct wrong statements, provide user knowledge and experience, encourage and motivate. They also indirectly indicate an item’s popularity by the number of posts per time unit. This information later allows filtering relevant content by activity.
• risk: Comments bear the risk of wrong, inappropriate and insulting statements or spam. They only allow indirect automatic measurement of the related item’s quality by its popularity.

Examples:

Ratings

A rating allows a user to quickly and effortlessly state an opinion about an object with a minimal interruption of the workflow she is currently in.  This way, ratings guarantee a low barrier for user interaction (depending on prerequisites like user registration). To obtain more detailed information a rating may be split up into sub-ratings (e.g. one for each object property) which allows different levels of granularity (see yahoo).

• Chance: Ratings are simple though flexible. Due to their minimal impact on a user’s workflow, ratings allow to collect lots of data which helps to reduce statistical spread. The gathered information can be used for further automatic processing and allow later filtering and sorting (e.g. by overall rating of a product, rating of a review and the like).
• Risk: When used without a proper explanation or within a wrong context, ratings may be misleading. Ratings related to a person may cause resentment and hence should be used carefully.

Examples:

Reviews

A review usually comprises a detailed statement about a virtual or real live object (e.g. a place, person, thing or service). This statement may include experiences made by the user, a personal opinion or a detailed analysis of an object’s properties. Usually reviews are combined with ratings and may themselves be rated by other users.

• Chance: Ratings allow collecting valuable content which is appreciated by other users (e.g. books, electronic devices, holiday destinations and much more).  They add further information to a reviewed object which can even be enhanced by the combination with the rating pattern. encourage better reviews through rating and reputation
• Risk: Like comments, reviews are prone to disinformation and spam. The subjective component of a rating can never be excluded. Trying to cover too many different aspects of the object to be rated may lead to forms which are too complicated.

Examples:

Favorites

In order to mark an item for later retrieval, a user may add it to the list of favorites. Clicking the add to favorites icon, which is usually in close proximity to the content item, allows to easily flag an item for later retrieval. This pattern is often extended by a field which allows adding keywords or tags for organization and later retrieval.

• Chance: The favorites pattern provides an easy way for users to collect relevant content items for later processing. Although it does not reveal anything about a contents quality first hand, it may at least give an indication through the number of users who added an item to their favorites. By sharing favorites or top lists, user interaction with the respective content items may be further stimulated.
• Risk: The low impact on the overall end user workflow may lead to an overloaded favorites list which can make it difficult to find relevant content later.

Examples:

Vote to promote

This pattern is used to bring forward a particular piece of content in a community pool of submissions. A promotion takes the form of a vote for an item, and items with more votes rise in the rankings to be displayed with more prominence.

• Chance: This pattern provides an immediate representation of the voting action through the position in the ranking list. It’s low impact on the superior user workflow permits lots of participations which reduces statistical spread.
• Risk: The representation through a ranking list may cause, ranking- not content based voting. Higher ranked items may attract more attention and hence receive more votes; others may disappear into the long tail of the popularity ranking list. The community members may try to game the system for whatever motivation or the rating community may not be large enough for significant feedback (although the pattern itself encourages participation). Without proper measures users may vote for items they did not consume.

Examples:

Thumbs up/down

Users are enabled to express a like / dislike (e.g. love/hate) type opinion about an object (person, place or thing) they are consuming / reading / experiencing by applying this pattern. It’s simple and intuitive enough to quickly grab a user’s view about an object.

• Chance: The benefit to other users is that these ratings, when assessed in aggregate, can quickly give a sense for the community’s opinion of a rated object. They may also be helpful for drawing quick qualitative comparisons between like items (this is better than that) but this is of secondary importance with this ratings-type. It may also encourage discussion among the community users by polarization. Highlighting items with a certain number of positive/negative votes may improve later filtering of content.
• Risk: The lack of intermediate choices may lead to mood based decisions. The black/white character of this pattern does not provide the fine granularity of ratings and the intrinsic polarization may cause conflicts within the community. A low number of ratings may cause insignificant and misleading results (e.g. one thumb up may lead to 100% positive rating). Finally the symbolic representation of up / down may be misleading or not understood by different cultures.

Examples:

Bury / flag as inappropriate

Although not separately listed by Designing Social Interfaces I find this pattern important enough to mention it separately. As a special case of the patterns mentioned above it allows the community to deal with unwanted content like spam, inappropriate, off topic, insulting or illegal items. Reaching a predefined threshold of user flags may indicate the site administrator or moderator to evaluate and if necessary remove item. Less critical cases could lead to hiding or moving an item. In order to simplify later evaluation an option to select the reason for flagging may be provided

• Chance: Applying this pattern allows to filter out unwanted content and increase overall content quality as well as the sites reputation.
• Risk: Good content may be buried and automatic processing may lead to loss of content items. Especially on large portals with a broad target group heavy workload for moderators may be caused.

Examples:

Soliciting feedback

Related to the comment pattern soliciting feedback allows to collect user feedback about the site itself. As a pattern which is familiar to most web users it is one of the most important feedback sources for site owners.

• Chance: Textual feedback may allow site owners to react and take measures in order to guarantee the quality of site and content.
• Risk: Without suitable precautions the site administrator will be covered with spam. The community could be heavily disappointed if the site owner does not respond to input or takes proper actions. Moreover the community may claim too much voice when it comes to defining the portals future direction. In both cases users might move to other portals.

Examples:

Conclusion

When applied properly, feedback patterns provide an effective way to increase the overall content quality of a community portal. They

• motivate users to create above average quality
• encourage users to add further information
• provide quality measures
• help separating good from bad content
• allow sorting and filtering by different quality attributes
• help to control inappropriate content

The list above is not complete and may grow with further evolving feedback patterns based on new concepts in the field of UGC. The review is based on my personal experience with this topic und I would appreciate your comments and feedback.