See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/264935893
A Multidimensional Perceptual Map Approach to Project Prioritization and
Selection
Article June 2011 DOI: 10.17705/1thci.00028
CITATIONS READS
6 590
2 authors:
Jack G Zheng
Kennesaw State University
40 PUBLICATIONS 315 CITATIONS SEE PROFILE
Some of the authors of this publication are also working on these related projects:
IT Education and Curriculum Development View project IT Lecture Notes View project
Vijay Vaishnavi
Georgia State University
100 PUBLICATIONS 2,639 CITATIONS SEE PROFILE
All content following this page was uploaded by Jack G Zheng on 13 May 2015. The user has requested enhancement of the downloaded file.
A Multidimensional Perceptual Map Approach to Project Prioritization and Selection
Manuscript for information only Full citation
1. Guangzhi Zheng and Vaishnavi, Vijay K. (2011) A Multidimensional Perceptual Map Approach to Project Prioritization and Selection, AIS Transactions on Human-Computer Interaction (3) 2, pp. 82-103
2. Zheng G. and V. Vaishnavi (2009), A Multi-dimensional and Visual Exploration Approach to Project Prioritization and Selection, the 15th Americas Conference on Information Systems, San Francisco, CA, Aug 2009
Abstract
When prioritizing projects, managers usually have to evaluate multiple attributes (dimensions) of project data. However, these dimensions are usually condensed into one or two indicators in many existing analysis processes. For example, projects are commonly prioritized using a scoring approach: they are evaluated according to predefined categories, which are then aggregated into one or two priority numbers. We argue that aggregated scores may only offer a limited view of project importance. This often leads decision makers to ignore the possible differences masked by the aggregation. Following the design science research paradigm, this paper presents a visual exploration approach based on multi-dimensional perceptual maps. It incorporates human intuition in the process and maintains the multidimensionality of project data as a decision basis for project prioritization and selection. A prototype system based on the approach was developed and qualitatively evaluated by a group of project managers. A qualitative analysis of the data collected shows its utility and usability.
Keywords
Multidimensional Perceptual Map, Business Information Visualization, Visual Information Exploration, Multidimensional Data, Project Prioritization, Decision Support, Design Science Research
1
1 Introduction
Defining projects is a common and useful way to manage goals and activities in an organization. A major concern for these organizations or departments is the management of a large number of projects in an effective and efficient manner. This requires clear understanding and communication of project status, balance of resource allocation, and a projects contribution to overall organizational goals. The concept commonly known as Project Portfolio Management (PPM) or Project Program Management is adopted to treat these projects as a complete group (McFarlan, 1981). Project prioritization is one of the many tasks performed at the portfolio level in PPM. Some tasks and decisions in this process are: collection and maintenance of project data, project categorization and management understanding, project assessment in the context of portfolios, and alignment of projects with business goals/strategies. These tasks and decisions are based on the understanding of multiple aspects (dimensions) of projects, such as project size (in terms of budget, time, or people), risk level, expected return, business goal, strategic impact, etc. The methods used to analyze these dimensions are varied and often depend on different management perspectives and business styles, but they generally follow two types of approaches: an analytical approach with an emphasis on complex mathematical or statistical models, and a simpler intuitive approach with a focus on human intuition and perception of relevant facts.
The analytical approach includes methods that apply concepts and models from the field of management science or financial investment management. These methods include Multi-Criteria (attribute) Decision Making (MCDM or MADM) (Dyer et al., 1992, Santhanam and Kyparisis, 1995, Yeh, 2002), multidimensional scaling methods (Borg and Groenen, 2010), financial models (Buss, 1983, Denbo and Guthrie, 2003), and techniques that formulate the portfolio problem as a mathematical optimization problem (Dickinson et al., 2001). A major challenge of these financial and mathematical methods is their reliance on mathematical and statistical models, which many people find difficult to understand and apply in daily practices due to their lack of appropriate financial and mathematical skills
2
(Leliveld and Jeffery, 2003). The needed skills are not about knowing and calculating indicators, but the ability to apply the indicators to interpret data in decision processes. In addition, many of these methods are useful to select competing options but are not very helpful in understanding the whole portfolio in terms of knowing where all projects stand and how they relate to one other.
Another type of approach emphasizes more intuitive and practical methods and models that make the process more transparent, understandable, and easy to communicate. These include many classic ranking or scoring methods (Brandon, 2006, Cooper et al., 2000, Ghasemzadeh and Archer, 2000, Jolly, 2003, McFarlan, 1981, Ward, 1990, Weir, 2004) and analytical hierarchy process (AHP) (Al-Harbi, 2001, Ghasemzadeh and Archer, 2000, Saaty, 1990, Saaty, 2008, Vaidya and Kumar, 2006). Simple visualizations are usually provided to present the results. These models provide an assessment that is easy to understand and implement, and can offer some level of reasoning consistency and comparison basis. However, in many of these methods, no matter what the process is, the final decision relies on a single or few simple priority numbers. Multiple attributes may be used as inputs and contribute to the calculation process, but at the end, these attributes are transformed into one or two indicators for simplicity of interpretation. But such simplicity does not always satisfy business needs. The simplified results tend to homogenize different projects, hiding useful and relevant information that may effectively distinguish them (Wang and Yang, 2005). That often leads decision makers to ignore the possible differences which are masked by the aggregation, and may result in decisions that are not well justified and not clear to other people. The same weakness applies to visualizations accompanying these models. First, many diagrams are fundamentally constructed based on only two dimensions (X axis and Y axis). Additional dimensions are merely used as display properties (such as color, size, and shape) and do not contribute to the definition of map segments. Such diagrams are incapable of representing the complete picture, and may lead to a narrower understanding of project distributions. Second, many of these visualization systems primarily concern the visual representation itself but do not emphasize how a human solves
3
problems (Zhang, 2001). They are more confirmatory than exploratory; just static reflections of results after the decision making process has been completed.
In this research, we present an intuitive visual exploration approach and its instantiation based on the concept of multidimensional perceptual maps (MdPM). The approach addresses the weaknesses of traditional scoring/ranking approaches and visualization approaches, while keeping their simplicity and interpretability. It reveals the values of underlying attributes and makes them transparent in the process of viewing, understanding, and analyzing projects and portfolios. Furthermore, such a system supports visual exploration of interactions to effectively and intuitively handle multidimensional information. The research addresses the following questions:
What are the major components of such an approach/system, and how are they designed?
How can such an approach/system be used to project prioritization and selection?
Following the design science research paradigm (Hevner et al., 2004, Vaishnavi and Kuechler, 2008), we used a prototyping method to iteratively refine the approach and its instantiation followed by a qualitative evaluation of the resulting system. Theoretical concepts were also abstracted and refined in the process. In the rest of this paper, we will first review the literature relevant to the visualization approaches to project prioritization, describe the core design concepts and components of the new approach, and illustrate the approach with the developed prototype and a scenario. Then, we will describe the evaluation work and results. Finally, we will provide some discussion and conclude the paper.
2 Literature review
2.1 Visualization and decision support
Many business decisions are unstructured and involve multiple sources of information. Visualizations are often used to help make sense of data and ease the decision making process (Baker et
4
al., 2009, Lurie and Mason, 2007). Good visualizations can effectively support and integrate human intuition to comprehend information. Intuition is a psychological behavior that allows understanding without apparent efforts (such as categorization, association, estimation, comparison, etc.). It enables a person to grasp the meaning, significance, or structure of a problem without direct reliance on analytical tools. Executives and managers use managerial intuition in decision making processes, and try to maintain a balance between logical reasoning and intuition (Isaack, 1978). The study of managerial intuition has a significant impact on the design of information systems (particularly decision support systems) to fit managerial styles and support management tasks (Kuo, 1998, Robey, 1983, Vessey, 1991). It also has implications for the design of a visual exploration system.
First, in the process of applying intuition, perception is the key to reaching quick assessment, and then actions follow (Kuo, 1998). This perception, or feel of business data, is an important starting point. Good information visualization, when used appropriately, is able to enhance human information processing and problem solving capabilities. More specifically, visualizations are able to (Grinstein and Ward, 2001, Jarvenpaa and Dickson, 1988, Tegarden, 1999):
Perceive useful and relevant information from complex and large volumes of data,
Provide a high level overview of complex data sets,
Identify structures, patterns, trends, anomalies, and relationships in data, and
Assist in identifying the areas of interest and help decision-makers determine where further
exploration should be done.
Second, executives and managers are not passive choice makers but are active sense makers; after the initial perception, they actively interact with the environment based on their continuing perception, actions, and reasoning to arrive at conclusions (Kuo, 1998). The decision making process does not end when the visualization is presented. Further thinking occurs and actions follow after the initial perception. These actions can be interactions with the data and visualizations or can be interactions with other people and tools. The behavior of applying intuition is not independent from using logical reasoning and other
5
tools, but an iterative and interlacing process. In this process, visualizations are not just confirmatory, but rather exploratory to facilitate the role of managerial intuition and sense making (Baker et al., 2009). They are useful in discovery tasks to generate ideas and hypotheses (Bowers et al., 1990).
Visual exploration has also been used for information seeking and data mining to allow faster data exploration, or even assisting in discovering knowledge buried in numbers (Keim, 2002, Tegarden, 1999). The purpose of visual data exploration is not to replace good solid quantitative analysis, but instead to allow the quantitative analysis to be focused (Grinstein and Ward, 2001). It can be used as a means to gain insight into the data and to create hypotheses (Keim, 2002, Oliveira and Levkowitz, 2003). Then, the verification of the hypotheses can be accomplished by statistical analysis, or may be done through visual data exploration. In this sense, visual exploration is a good complementary approach, rather than a competing approach, to other methods and tools. The major advantages of this approach over mathematical or data mining techniques are: 1) it directly involves users, 2) it is relatively easy to use, and 3) it requires no understanding of complex mathematical or statistical algorithms or parameters (Keim, 2002). This is especially helpful for business executives and managers.
2.2 Perceptual mapping
Many visualizations in the business management world are in the form of charts and illustrational
diagrams (Tegarden, 1999), such as pie charts, bridge charts, Gantt charts, scatter plots, bubble diagrams,
and perceptual maps. Our particular interest is on the perceptual map (or position map) (Gower et al.,
2010). A perceptual map is a kind of diagram that segments the diagram space into different regions (with
different characteristics) and then positions objects in the diagram based on their values compared to
diagram regions. It can be used to plot interrelationships between any subjects that can be rated on a range
of attributes; and it can show these subjects relative positions with respect to other subjects as well as
with respect to the evaluative attributes. Perceptual mapping has been used as a strategic management tool
and a marketing research tool for many years. In project management, many methods evaluate projects
6
based on two essential attributes and then these projects are displayed in perceptual maps. In this way, users can easily perceive project distributions and portfolio composition. For example, (McFarlan, 1981) uses the attributes of project structure and technology level. Some other popular dimensions (attributes) include risk vs. reward (Brandon, 2006, Cooper et al., 2000), risk vs. relevance (Maio et al., 2002), success and value (Weir, 2004), technology attractiveness and technology competitiveness (Jolly, 2003), and risk vs. time-to-complete (Ghasemzadeh and Archer, 2000).
Most existing perceptual maps are constructed based on two perpendicular axes, which are plotted to represent two chosen dimensions/attributes (Gower et al., 2010). The scale on each axis can be quantitative and continuous, or qualitative and discrete. The mapping space is often conceptually organized into four (2 by 2) or 9 (3 by 3) regions for better business interpretations (Figure 1). The data items are then positioned (mapped) in the diagram with their values measured against those scales.
Figure 1. Quadrant (bubble) Diagram Based on Risk-Reward (Cooper et al., 2000)
Perceptual maps offer a unique ability to communicate the complex relationships between objects and the criteria used to support decisions. Their graphical simplicity also appeals to senior management and can stimulate discussion and strategic thinking at all levels. However, traditional quadrant or matrix
7
diagrams can usually model only two dimensions (attributes), while projects are constantly evaluated by more than two dimensions; for example, the Balanced Scorecard (BSC) (Kaplan and Norton, 1996) evaluates performance based on four perspectives (dimensions). It becomes difficult for traditional perceptual maps to visualize more dimensions. A common approach to visualize more attributes is the use of visual properties such as color, size, shape, or other iconic representations. For example, Figure 1 uses bubble size to represent resource requirement levels. While this technique increases the dimensions in a 2D map, the map is fundamentally constructed (positioned) based on only two dimensions. A second type of approach is to condense dimensions by mathematical or statistical transformations, such as multidimensional scaling (Borg and Groenen, 2010), in order to fit them on a classic axis-based perceptual map. However, this type of approach shares the same weakness as many scoring approaches: flattening the multidimensionality of project information may fail to effectively distinguish projects (Cooper et al., 2000).
Another approach is to maintain the number of dimensions while constructing the map, but not use classic axes to define map segments. This kind of map presents more visual complexity but provides richer dimensional information. For example, Self-Organizing Maps (SOM) (Kohonen, 2001) are able to project high-dimensional data to a lower dimensional space while at the same time preserving their relative data relationships and presenting results in a 2D space. It is one of the effective methods for analyzing multi-dimensional information (Wang and Yang, 2005) and very useful in exploratory pattern analysis (Jain et al., 1999). SOM has been successfully used as a clustering technique in many computing areas such as image analysis, optical patterns, acoustic processing, speech recognition, signal processing, and robotics (Kohonen, 2001). It has also been applied to documents organization (Kaski et al., 1998), database schema (Zhao and Ram, 2004), web search results (Roussinov and Chen, 2001), and financial areas like real estate appraisal, mutual fund portfolio, etc. (Deboeck and Kohonen, 1998, Vesanto, 1999). However, many of these applications utilize SOM mainly for clustering purposes, but do not use it as an interactive and visual exploration technique for business and management decision support. Little
8
research has been done on visual exploration processes based on multidimensional perceptual maps. This research develops a multidimensional perceptual map approach to project prioritization and selection, and uses SOM as its underlying technique.
3 A Multidimensional Perceptual Map Approach
A visual exploration approach is chosen as the basis of the proposed design as many studies show the effectiveness of visualizations in decision making (see Section 2.2, literature review). In summary, visual representation and interaction can better incorporate human intuition and support comprehension of complex information in the decision making process (Jarvenpaa and Dickson, 1988, Keim, 2002, Kuo, 1998, Meyer, 1991). The following sections describe the major system concepts and components comprising the proposed approach, a general decision making process in project prioritization and selection, and the features of the prototype demonstrated through a scenario.
3.1 Basic system concepts and components
In general, the designed approach is a computer system driven visual information seeking process (Keim, 2002, Shneiderman, 1996). The overall design of the visual exploration system is suggested by the information behavior model (Wilson, 1981), which describes the process of how people seek information for certain needs. Figure 2 shows the conceptual architecture of the proposed design, corresponding to the theoretical constructs of the information behavior model. In Figure 2, project prioritization and goal alignment corresponds to Information Need (purposes of using the system). Visual Exploration Actions generally correspond to Information Seeking Behavior (human behaviors for interacting with the visual elements). Visual Elements are the basic and often static visualizations generated by information systems. Conceptually, an information system creates a set of basic visual elements and also provides functionalities to support visual exploration actions. The following two sections discuss components in Visual Elements and Visual Exploration Actions.
9
Information Behavior Model (Partial) (Wilson, 1981)
Multidimensional Perceptual Map Approach to Project Prioritization
Information Need
Project Prioritization (and Goal Alignment)
a. (Information) Need
Visual Exploration Actions
The interactions between users and visual elements, which may include: visual examination, visual element customization, and other actions that manipulate the visual system. A series of actions form a basic process of using the system.
Visualization System (Visual Elements)
The basic visualizations generated by the system. A multidimensional perceptual map is the key visual element.
b. Information Seeking Behavior
c. Information System
Figure 2: The Design Approach Generally Follows the Information Behavior Model (Wilson, 1981)
3.2 Visual elements: profile chart and perceptual map
Visual elements are the basic visualization units created by the system. There are two key visual elements in the proposed system: profile chart and multidimensional perceptual map.
3.2.1 Profile chart
A profile chart is a visualization of an object (a project) based on the values of its multiple attributes (dimensions) selected to represent the object; such a visualization forms a representative shape pattern that can offer a unique visual impression of the object. Examples include candle-stick charts (used in stock trading technical analysis), Star and Petal (Tan and Fraser, 1998), Parallel Coordinates (Inselberg, 1985), radar (or star, spider) diagrams, and others (Hoffman and Grinstein, 2002, Soukup and Davidson, 2002, Tegarden, 1999). Profile charts are able to present complete multidimensional profiles, avoiding the reduction of multiple dimensions to a single number, and providing a strong and memorable impression that is easy for users to remember and compare.
10
A profile chart can be created using various type of basic charts such as bar charts, line graphs, area graphs, or radar diagrams (Jarvenpaa and Dickson, 1988, Tegarden, 1999). For example, Figure 3 shows a radar chart for the project Anti-Spam based on its scores on six dimensions. The system/approach itself does not provide guidance on choosing chart types but leaves them as visualization options. For consistency and illustration purposes, this paper uses the radar chart.
Figure 3: A Profile Chart in Radar Diagram
3.2.2 Multidimensional perceptual map
The multidimensional perceptual map is the center piece of the system. It is a high level overview of all or selected projects positioned in a way that best illustrates project properties and relationships. Projects and portfolio distributions can be visually examined by observing the map directly. Traditional perceptual maps are created using scatter charts or quadrant diagrams, which are based on two dimensions (X and Y axes). Then data items are plotted on the plane based on their values for the two attributes (see Figure 1). A 3D chart can be built and the third dimension (Z-axis) can be used to represent a third attribute. But then it is not meaningful to add more dimensions to the visualization as geometric projections beyond three dimensions cannot be easily used to convey information to people.
The proposed multidimensional perceptual map, however, does not rely on the definition of any fixed dimensions or axes. Two major features of this map are:
The Anti-Spam project is evaluated against six business goals: very good on Optimization of resources; moderately good on Improve reliability and integrity, Increase effectiveness, and Reduce technology risk; not so good on Reach/support customer base and Provide interoperability.
11
1. The position of a point or an area on the map is not determined by coordinates of fixed axes, but by a vector of values that represent multiple attributes that are used in the analysis. The vector of each area is directly visualized on the map.
2. The map can be divided into areas at different granularity levels to meet various exploration needs.
The smallest unit of map area is a cell. A map is usually divided into cells arranged in rows and columns. Map size (number of cells) is usually denoted by X by Y, where X is the number of cells per row (map width) and Y is the number of cells per column (map height). Figure 4 shows some variations of conceptual illustrations with a map size of 6 by 4 (6 columns, the width; 4 rows, the height). The characteristic of each cell is not determined by fixed axes, but by a vector of values which reflect the underlying attributes selected in the analysis. The vector of a cell should be similar to those of its adjacent cells. In this way, the cells show a gradual and smooth change of all attributes on the map. These vectors are visualized using profile charts which are directly displayed in all cells (similar to sparklines). Such visualization is called Cell Profile View. An implementation is shown in Figure 6. Cells in the map can be grouped into regions based on similarity (see Figure 4, bottom row). The number of regions depends on user preference and can be flexibly defined. Such visualization of the map is called Region Profile View.
12
Figure 4: Multidimensional Perceptual Map
Top row: rectangular and hexagonal layout of cells; Bottom row: region profile views show higher granularity level with different groupings in different visual representations.
Once the cell and all region features are determined, projects can be plotted on the map (Item Projection View) and placed in the cell to which they are most similar. The most common similarity metric can be the Euclidean Distance. Consistent with the cells, data items (projects) will be more similar if they are closer to one other. Table 1 summarizes the three basic map views, which offer different perspectives of the perceptual map. These views can be overlapped (Figure 8) to meet specific visual exploration needs.
Table 1. Three Perceptual Map Views
Map View
Description
Example Figure
Cell Profile View
This view is used to examine the map at a fine-grained level. Each cell is represented by a vector corresponding to the selected dimensions (attributes). This vector represents the characteristics of a map cell. In our system, each vector is visualized using the profile chart, which is embedded directly in the cell. A Cell Profile View displays profile charts of all cells. In such a view, the changing trend/pattern of all cells can be directly observed on the map so that users can have an overall understanding of the map.
Figure 6
Region Profile View
This view is used to examine the map at a coarser level. A region is a group of nearby cells with similar patterns. A map then can be divided into more coarsely identified regions. The region profile is
Figure 9 (overlapped with Item Projection
13
created based on its member cells and then visualized using the profile chart. One advantage of this view is that it reduces visual complexity and suggests higher level of grouping. Compared to the Cell Profile View, the differences among regions are more discrete.
View)
Figure 8 (3 views overlapped)
Item Projection View
This view is the result of mapping projects on the map. Each project is placed into the cell with the least difference between the project and the cell based on selected dimensions. The distribution of projects on the map should reflect portfolio characteristics. Projects that are closer are more similar than those further away in terms of all dimensions.
Figure 7, Figure 9 (overlapped with Region Profile View)
The key to creating the multidimensional perceptual map is determining how each cell represents a vector of values for the chosen set of underlying attributes. The vectors need to present a smooth changing pattern from cell to cell in all directions, with each one similar to its adjacent cells. Creation of such maps is not an easy job for humans and it is best to achieve this using computer algorithms. We chose to apply Self-Organizing Maps (Kohonen, 2001) as the underlying technique for initial map generation because its visualization capability and output can best realize our map definition. However, even with computer automation, human intervention (e.g. manually changing cell vectors) is sometimes needed when computer algorithms like SOM do not produce a desired output.
3.3 Visual exploration process and actions
The multidimensional and visual exploration approach is not merely a system providing static visualizations. It is also a series of interactions between users and systems, hence, it is an exploration process (Keim, 2002). Figure 5 shows a general visual exploration process of using the system, followed by detailed explanations of each step. To better illustrate the process, a project prioritization and selection scenario is presented in section 3.4, with screenshots of an actual running prototype system.
14
1. Map Generation
a. Data preparation
b. Map initiation and verification
2. Visual Exploration
c. Overview (map visualization, labeling, and customization)
d. Zoom and Filter (drill up and down based on the three map views)
e. Object Details-on-Demand
f. Decision making
3. Decision Making and Justification
Figure 5. A General Visual Exploration Process based on Multidimensional Perceptual Map
1. Map Generation
In this sub-process, the goal is to define and create a multidimensional perceptual map for visual exploration. The essential step is to apply SOM algorithm and further customize the results by visual exploration.
a. Data preparation
The first step is to prepare data for map construction and analysis. The most important data are the attributes of projects selected for a particular task. Once the attributes are determined, a data table is prepared based on all values of these attributes for every project. Depending on the value domain of each attribute, weighting and scaling may be applied.
b. Map initiation and verification
15
In the second step, the map is defined using the same attributes that describe the projects. The most important map setting at this step is the map size, defined as number of cells (number of rows by number of columns). The finest granularity of map regions is determined by the map size. Different users may have different perceptions of the size, so the size of the map is not predetermined or suggested by the tool, but rather to be explored and tested out by users. Based on a certain map size, the map is generated using a computer algorithm such as SOM. Then, the cell profile view (see Figure 6) of the map is visually inspected (the inspection is described in detail in the Visual Exploration sub-process) to confirm that cells present smoothly changing vectors from cell to cell. This verification step can be done initially or may be conducted later in the Visual Exploration process if any abnormality is discovered.
2. Visual exploration
In this sub-process, the key steps are user interactions with the visualization (visual exploration actions). The process and actions are designed in accordance with the visual information seeking mantra: Overview, Zoom & Filter, Details-on-Demand (Shneiderman, 1996). The mantra has been widely referenced by researchers who design novel information visualization tools as a justification for their methodological approaches (Craft and Cairns, 2005).
c. Overview
The overview is used to comprehend the whole map and customize it based on needs. The system will transform SOM results into the Cells Profile View (see Figure 6) and give users a general sense of the map (Overview). Because the changing trend is clearly shown on the map using profile charts, users can quickly make sense of the perceptual map and its regions. A major action in this sub-process is setting different granularity levels of map regions (Region Profile View). Cells Profile View is great to see smoother trends on the map, but it is visually intensive. Thus, users can define coarser levels of regions. The system supports the definition of multiple region sets (multiple ways and levels to divide a map based
16
on particular needs and perspectives). Assigning cells to regions is a manual process. A user defines these regions by directly observing, comparing, and contrasting cell patterns. Each regions profile is created by the system on the fly and presented to users though profile charts (see Figure 8). If the map is not found to have smoothly changing trends, or some cells are not quite satisfying, then users have three options: 1) try a different set of SOM parameters and re-create the map (usually when the result is largely unsatisfying); 2) increase the size of the map (to decrease the granularity level) such that the resulting vector change trends are smoother; or 3) directly change selected cell vectors in the SOM result (bypassing the algorithm) if only a few cells are not satisfactory. This kind of direct human intervention is an example of applying intuition and sometimes is very effective. Once the map is deemed to be satisfactory, it can be saved and reused later for analysis consistency. Last, selected projects can be plotted on the map to show their interrelationships based on their positions on the map (Item Projection View).
d. Zoom and Filter
In this step, users utilize three map views to quickly focus on certain parts of the map and narrow down candidate projects for final comparison. They may focus on specific regions and projects that are of interest (Zoom). Users can switch between fine grained Cells Profile View (Drill Down) and any coarse grained Region Profile View (Drill Up), or look at them at the same time by overlapping them. A set of target project groups can be defined and highlighted on the map (Filter), so users can pay attention to those projects that are close to the target projects (Figure 9).
e. Project Details-on-Demand
Once candidate projects are selected, users can go further to compare individual projects head-to- head using the profile chart comparison tool (Details-on-Demand) (Figure 10). The comparison tool offers multiple modes of visual comparison (side-by-side or overlapped), using different chart types (radar, bar, etc.). The profile charts give clear justifications for analyses and decisions.
17
f. (3.) Decision Making and Justification
Using the system, a decision (in this case, project prioritization and selection) can be made. Such a decision is better supported by the consideration of multiple attributes throughout the analysis process, and it can be better justified and communicated to others.
The process described above is a general exploration process. It can vary for different situations, skipping or repeating certain steps. For example, it does not always start from a fresh new perceptual map. SOM may only be applied once, and human intervention can be applied repeatedly to reach personal or agreed-on acceptable results. The map represents a possible and expected project distribution space, not just a particular group of projects. Thus it is relatively stable and there is no need to change it for a period of time. In addition, reuse of an existing familiar map promotes analysis consistency and maintains the common understanding in an organization so that it is easier to communicate.
3.4 An example scenario using the developed prototype
We developed a prototype (a Windows desktop application) to implement the concepts in our approach. The major components developed were the visual elements that represent the multidimensional perceptual map, and user interface functions that support various visual exploration actions. In addition, two third party programs were used: 1) SOM_PAK (Kohonen et al., 1996) which provides the basic SOM training functions; and 2) dotnetCharting (http://www.dotnetcharting.com) which provides basic chart creation functions (such as the radar chart). The data used for the prototype were real project data relevant to business goal alignment from an IS&T Department of a major research university. The following illustrational scenario is based on the use of this prototype.
Scenario: The IS&T Department of a major university manages the majority of its activities by
projects. The Project Management Office uses a scoring model to prioritize projects and reports a Top10
prioritized project list to the upper management. In this department, the scoring model typically consists
18
of six attributes (dimensions) that are related to business goals. These attributes are Optimize use of resources, Improve reliability and integrity, Increase effectiveness, Provide interoperability, Reach/support customer base, and Reduce technology risk. The upper management will specially focus on these top prioritized projects when dealing with issues like resource allocation and strategic planning. When a Top 10 project concludes, another project will be promoted to the list. In this scenario, three of the ten projects are completed and the project management office is asked to recommend three additional projects to complete the list.
Using our approach and the prototype, the following steps (corresponding to steps in Figure 5) are taken to select the three projects (details of the prototype are described with each screenshot):
a. Randall, a project manager in the PMO, selects all current projects from the database. Each project is evaluated on how it is aligned with the six business goals using a scoring approach. These scores are based on a common evaluation framework, thus there is no data transformation, scaling, or weighting to be considered.
b. Randall chooses the hexagon map type and sets the map size to 9 by 7. He then feeds the project scores data to the implemented SOM algorithm and gets the initial results.
19
This is the major user interface. On the left, the map properties are summarized in the upper corner, and detailed profile charts are displayed for the selected project, cell, and region (using mouse movement to choose). The tool bar in the middle provides interaction functions such as zooming, layering, highlighting, etc. On the right is the perceptual map this figure shows the Cell Profile View.
Figure 6: Prototype Screenshot: Perceptual Map Cell Profile View (Hexagonal Style).
c. A map with a size of 9 by 7 is visualized using the Cell Profile View (Figure 6). In the figure, the six scoring components are displayed in the top region on the left panel; they represent the six axes in each radar chart, following a clock-wise order. Randall examines the Cell Profile View and clearly sees the changing patterns of the profile charts. He thinks the map is satisfactory so no manual adjustments are made.
20
This figure shows the Item Projection View, with each project mapped to a cell according to its similarity to the cell. Two radar charts on the left shows the pattern for one project and the cell it is in.
Figure 7: Prototype Screenshot: Perceptual Map Item Projection View
Randall continues to examine the project distribution using the Item Projection View (see Figure 7; projects are visualized as labels). He may overlap the Item Projection View with the Cell Profile View to get more details. For example, the three projects in the upper left corner (EAI Grant, 2006 Tech Fee, 2007 Tech Fee) are mapped to cell #0 (compare Figure 7 to Figure 6); that means, pretty intuitively, that these three projects are similar to one another and to the profile chart pattern of cell #0; and they all seem to have low priorities on 4 of the 6 dimensions. Randall can move the cursor on project labels in the map to get its profile chart displayed on the left panel (the
21
first radar chart represents the profile of project 2006 Tech Fee and the last one represents the profile of cell #0).
Figure 8: Prototype Screenshot: Perceptual Map Views Overlapped: Cell Profile View + Region Profile View (color coded) + Item Projection View
To reduce map complexity, Randall decides to form regions instead of reading cells directly. Based on his examination of cells and projects, Randall defines six map regions (Figure 8) using the interactive region definition tool. He also labels each region and chooses colors for visual differentiation. All regions are summarized in the left panel, using profile charts to preview region patterns. The prototype also provides a detailed report of all regions and the projects in each region.
d. With a quick scan of the map, Randall puts his attention to the lower right corner which seems to be the higher priority region (Figure 8). Randall then defines a reference project group that consists of the seven existing projects in the Top 10 list. He wants to find projects that are close to these seven
Profile charts for all regions are automatically created and displayed on the left.
22
projects on the map, so he can select those as candidates to be further examined. In Figure 9, these seven projects are highlighted in green. It is easy to see that 6 of them fall in the region High Priority (red colored, lower right region marked by broken lines). There are a number of projects close to these high prioritized projects, and Randall first selects some candidate projects to focus on (Figure 9, circled).
Figure 9: Prototype Screenshot (partial): Perceptual Map Region Profile View + Top 7 Prioritized Projects
e. Randall puts 6 candidate projects in the profile comparison tool. He chooses the overlapping radar chart type and line-area style (Figure 10). As the figure shows, Randall switches on 3 projects to make them visible, and hides others (he may continue doing this with other projects). The difference is clear: the EasyView and Password Resets project scores higher on Reduce Technology Risk; the Common Graduate Application for Admission project scores higher on Reach Customer Base; and the Anti-Spam project scores higher on Improve Reliability and Integrity. Now, depending on Randalls perspective or department policy, Randall will choose one of them as one of
23
his recommendations. He will repeat this process to compare and contrast other candidate projects until he selects the final three.
Figure 10: Prototype Screenshot: Profile Charts to Compare Projects
f. Now Randall can better interpret and communicate the conclusion to others. He feels it is well justified. If he needs more data or models to enhance the conclusion, he may use other tools to do so.
This scenario and prototype demonstrate the core system components and the general process described earlier. The process and results are easy to explain and discuss with other users. The approach successfully differentiates projects with similar aggregate scores, and makes sure the selected projects are aligned with business goals as closely as possible. This scenario illustrates only one typical process of using the approach and system. The process or its variations could be repeated until results are satisfying.
There are different chart types and styles to choose for best visual effects.
24
3.5 Summary and discussion
The overall design concepts and core components are summarized in Figure 11.
Information Need:
Project Prioritization and Goal Alignment
Visual Exploration Actions
Overview: examine cells and all projects
Zoom: drill up and down to explore different layers and parts of the map.
Filter: define and highlight target project groups
Details-on- Demand: Detailed Profile Chart Comparison
Cell Profile Region Profile Item Projection View View View
Multidimensional Perceptual Map Visual System (Visual Elements)
Profile Charts
Figure 11: Conceptual Model of the Multidimensional Perceptual Map System
The new approach provides an intuitive system and process to support decision making. The key features and advantages of the new approach are twofold. First, the approach directly incorporates multiple dimensions throughout the process, from map generation and exploration, to decision making and justification. It reveals the underlying dimensional information that is usually hidden from decision makers, so that this information becomes transparent and the decision is well justified and understood. Second, it emphasizes human interaction with the visualization system throughout the process, including definition of the map. This makes complex multidimensional information more approachable and comprehensible for decision makers. The visual exploration approach fits the cognitive style of a segment of the population (Robey, 1983).
25
There are also two things to be noted about this approach. First, this research was scoped to investigate how a visual exploration system can provide assistance to prioritize projects and justify decisions based on a predefined set of attributes. It did not directly deal with criteria selection and interactions or dependencies of criteria. Second, the visual exploration approach is more of an exploratory approach and system, rather than a confirmatory one. It helps to quickly understand the big picture, discover potential patterns, narrow down areas of focus, and come up with hypotheses intuitively. After that, the visual exploration approach may continue to be used to confirm the conclusion, or other analytical models may be used. It is actually better to incorporate the approach in a larger system, and work with other systems and approaches. In such a way, the approach can be used effectively, maximizing its advantages and minimizing its disadvantages.
4 Evaluation of the Approach
The evaluation of the presented design approach is based on theoretically justifying the approach as well as on providing an initial qualitative evaluation of the effectiveness of a proof-of-concept instantiation of the approach.
4.1 Theoretical evaluation
The design approach has a strong theoretical basis. The major components in our design approach directly correspond to elements of the foundation knowledge. The major theory bases we draw from are: 1) Information behavior theory (Wilson, 1981) in Section 3.1, where we discuss the overall system (Figure 2 shows the connection between the design approach and the theory); 2) Visual information seeking mantra (Shneiderman, 1996) in Section 3.3, where we discuss the design of visual exploration process (Figure 5, step 2 Visual Exploration), and 3) Visualization in decision support and managerial intuition, in Section 2.1, where we discuss the role of visualization and intuition in decision making.
26
Figure 11 summarizes the design concepts directly based on the theory bases. These theory bases provide justification of our approach and provide insights on why the presented design approach should work.
4.2 Empirical evaluation
It is difficult to directly test the effectiveness of the design approach through actual usage of a system. The goals for such evaluation, in the current stage of the research, were therefore limited to an initial evaluation of a proof-of-concept instantiation of the approach through limited but meaningful qualitative feedback from potential users. This qualitative evaluation method is an acceptable evaluation choice as it enables an understanding of the phenomena for theory development or problem solving in an organizational context (Hevner et al., 2004), and has been used in some design science research projects (Tremblay et al., 2008).
According to the Technology Acceptance Model (Davis, 1989), the two most important factors that influence users intention to use a system are perceived usefulness and perceived ease-of-use. Thus, the major objective of the evaluation is to seek evidence to confirm the perceived usefulness and perceived ease-of-use of the system based on the proposed approach. We decided to conduct this evaluation through interviews to see what project management practitioners think about the prototype. The major type of data collected for evaluation was qualitative data obtained through interviews, with complementary quantitative data from post-interview questionnaires. The first author contacted the same IT division of the large research university where the project data was obtained, and created a target interviewee list. The target interviewees were project managers and higher-level managers who had working experience with multi-project planning and management. These people needed to have appropriate domain knowledge so they could provide sound and relevant feedback. Nineteen qualified people were then identified, including directors (who directly report to the Chief Information Officer), department managers, and project managers. They were contacted through email and were invited to interview sessions. The recruitment result is presented in Table 2, and Table 3 shows the background of final
27
participants. Notably, all three people from the Project Management Office (which oversees all projects and performs portfolio level management) participated in the evaluation.
Table 2. Evaluation Recruitment Summary
Table 3. Interviewee Background Summary
Each interview lasted about an hour. During the interview, participants were presented with the prototype and walked through major functionalities with the help of example scenarios. Efforts were made to ensure that participants understood each component of the prototype system and each step of using the system. A prepared list of questions was used to guide the interviews; interviewees could also raise questions and provide their comments freely. Small exercises on using the prototype were also conducted with interviewees. After the interview, each participant was asked to fill out a post-interview questionnaire and respond to some assessment statements about the prototype/approach and background information. They did this on their own time. This gave interviewees more time to think about the prototype and to carefully provide their feedback.
4.3 Interview analysis and results
Each interview was video recorded. After the interview, the video was reviewed and all major activities in the video were transcribed. The activities were mainly conversations between the researcher and interviewees, but also included participants actions and emotions (for example, their actions directly interacting with the system/screen). Interviews were transcribed and analyzed using a template analysis
Participant Position
Number of People Contacted
Number of People Responded
Number of People Finally Participated
Director
5
4
2
Manager/Project Manager
14
11
8
Total
19
15
10
Experience Area
Average experience (in years)
Information system/technology
11.6
Project management
8.8
Using project management software
6.9
28
method (King, 1998). First, an initial template was developed with pre-defined coding categories, which focused on the system components and design objectives (utility, via perceived usefulness, and usability, via perceived ease of use). Then, interview transcripts were examined for their meanings and implications. Conversations and activities were coded using the initial template, seeking common themes and variations that provide rich descriptions. Each activity could be coded with multiple codes or themes. During the process, other themes emerged, and more detailed themes were identified; these were incorporated into the initial template. The analysis template was then modified and eventually finalized. Then all interview activities were coded using the final template.
Evidences were identified and organized as themes that provided more details of perceived usefulness and ease-of-use. These themes are consistent with the capabilities of visual cognition (Tegarden, 1999) and show the utility and usability of the system. Table 4 and Table 5 summarize the final results with selected user comments and brief discussion. Overall, the qualitative feedback from these real world managers provided not only initial confirmatory evidence, but also generated meaningful and constructive comments that will be used to improve the design of the prototype and possibly refine the design approach. As one user stated:
I would be more inclined to use it, because this is very valuable, especially for the visual people. I think it would be a wonderful thing to use. Any manager should have all these stuff.
Table 4. Evaluation Results Summary on Perceived Usefulness (Utility)
Themes (with discussion)
Exemplar Quoted Comments
1.1 Providing overviews (big pictures) of project portfolios.
The system gives a quick overview of the portfolio based on selected attributes.
Just looking at this on the right and the left [of the interface], I can see right off the bat where they are coming from [positioned].
Its easier in the sense that if you want to take a very quick view [participant snapped his finger], just take a quick look at something, you can see very quickly where the project portfolios are in terms of meeting those targets of business goals. Yes, for that, it looks good.
I suspect that people will need to further dive into if they actually start the examination. But I think thats the play; this is just to give
29
you a quick look and feel of how things are going.
1.2 Comparing and contrasting projects.
Using the visual comparison tool, users can clearly see the difference among projects. It is also helpful in suggesting candidates that are similar to target projects.
Look if you can see based on what area each projects covering the relative benefit according to your criteria and weightings of each. Thats heck a lot more meaningful than just a number vs. another number.
It makes sense to me; a lot more than sitting there and reading a whole bunch of spreadsheets and trying to figure out the aggregates. Id much rather do this.
You can look at it, like, OK, am I learning towards technology risk or leaning towards improving reliability. What is it that I am trying to accomplish by this? You can make a decision based upon [this].
1.3 Justifying decisions.
It provides better justification as dimensions are transparent in the analysis process.
I can argue this with the boss. I can say, look, I cant do those three (pointing to the screen); right? I cant do all three of those at the same time. You cant put them as the same priority -there are more possible ways we can do it
This takes the human emotion out of it. When I work with people, they are very attached to the project, and they are very emotional about it. This takes all the emotion out, puts straight down: this is the number, this is what it shows, this is what should be, this is why we are doing it that way. And this may help to convince some people that, yes, we realize your project is important, but this one should go first because of whatever reasons.
1.4 Drawing attention.
Visualization is able to draw peoples attention in an intuitive way. First, it reminds people of things that are easily neglected. There are hundreds of projects in an organization and it is hard to remember every one of them. The tool can reveal relationships and bring attention to related projects.
Second, it also allows people to discover interesting things which emerge from visualizations and start asking questions. It enables users to discover mistakes or other unusual patterns and to conduct further focused analysis.
Yeah, sure there are many projects that we do are similar in nature. I dont think we realize it until we are really into it and we are actually working on things that are identical and many ways are pretty much the same. Whereas you put it up there, you like, wow, wait a minute, those are all pretty much the same.
The conversation is: why are these are up here, and why are these down here? So, then you bring up the individual cells, and you dive down into what the criteria were, and say OK: this why they are. Then this gives an opportunity to say, well, what if we did not rate it so high: where would it fall? Then you can probably visually say, well, it will probably flip over here, or flip down here, or just move a little bit down in the same box [cluster].
Oh yeah, definitely, because it immediately opens up a lot of questions: why is this like this? Just dive deeper to find out. Then we keep using the views you have previously to compare each project on top of each other. See whether they meet in terms of the dimensions, the business goal.
30
1.5 Facilitating discussions.
The tool can be used as a group level system to facilitate discussion and communication, provided that everyone understands and agrees to the dimension selected.
With a group of peers, we got to define some common way of negotiating. [Does the tool facilitate the seeking for such common ground?] Yes, much more so than just numbers with some secret formula behind that.
at the staff meeting, taking this to the ITSG and have this chart coming with clusters, that would be good.
1.6 Being objective.
Lack of quantitative measures and precise threshold potentially leads to multiple interpretations by different people. However, if agreed upon by a group of people, the exploration process is actually more transparent in revealing underlying hidden dimensions. From this perspective, the tool is more objective.
The only issue I have with that is that, again, if you cant define the thresholds, if you leave it up to the individual users to determine it, based on how objects look, that really lessens the objectivity. Having a specific threshold kind of makes it more objective. The group will have to agree to move in that there.
doesnt allow for as much gaming as the current process does. Thats actually very helpful.
Table 5. Evaluation Results Summary on Perceived Ease-of-Use
Themes (with discussion)
Exemplar Quoted Comments
2.1 Understanding the perceptual map.
All the interviewees agreed that the visualization makes the decision process easier, especially the profile charts. The overall approach is also easy to understand, as participants could actually be able to correctly interpret the visual elements following exercises during the interview.
This was a very easy tool to grasp, the 6 points, the individual projects and then the map. The clustering was very simple and easy to grasp. Visualization of things we have not been able to adequately visualize before.
Yes, absolutely. I think so. I think when you look at that, and you start going around and comparing; I like that thing you just lay it down on top of it, and go where they fit it, and things you got three ways of doing it. Thats really easy! That makes total sense! Its logical.
For me, being able to look at a chart like this, I can understand what the chart is trying to tell me. Whereas if I have a list of projects with numbers, you know [negative expression], coz what you are trying to show is very complex.
2.2 Recalling profiles of projects and portfolios.
It is easy to remember and recall project profiles and portfolio profiles in the Region Profile View. Interviewees found visualizations easier to remember and recognize when
It all depends on how well you define those parameters, the criteria and how meaningful those are If they were meaningful to me, yeah, those zones will stick to my head and I can carry around and use it.
Sure. Especially once you learned the six points, to me, it wouldnt take much before you start thinking that way. You look up and then
31
they are familiar with the dimensions used to form those shapes.
you go, hmm, thats gonna fall into that cluster.
The chart you showed before [clusters summary report, and profile chart comparison] will stick with me. That will stick on my mind because its bigger and I can see where all these points are [dimension labels].
Well it is right now [imprinted in my mind] [participant laughs]. Yeah, it has a lasting effect in terms of the groupings of the projects. For overall high level, its got a lasting effect; for more detail, probably it doesnt.
2.3 Operating/using the prototype.
The prototype is easy to use. Some users could actually use the tool quite well during the interview.
First of all, [ repeating the use process] for that, how hard is that? I mean, you are the one thats doing it. Well, heres my number. Right? And then its pretty self-explanatory after that one he put the numbers in and do(es) whatever. I like it.
This would be easy, I got the whole idea going from the cells, and clusters, making my own clusters this is slick.
In one interview, in a conversation of the potential use of the tool, the participant suddenly approached the projected screen and started to imagine/envision that he put the tool into a real case. He explored the map and at the same time explained his thoughts and analysis, on a scenario of manpower assignment and project selection.
Having the flexibility to determine on your own what the view is gonna be, [this system] is very good.
5 Discussion and Conclusion 5.1 Contributions
This work makes two major contributions 1) to the project management area, and 2) to the business information visualization area.
1. The research has resulted in a new approach for project portfolio management and a working prototype that instantiates the approach. A general contribution of design science research is the creation or enhancement of IT artifacts (i.e., methods, models, algorithms, instantiations) for a business need in an appropriate environment (Hevner et al., 2004). A review of the literature shows that current methods and
32
tools are limited in visualizing and analyzing multidimensional project information in an easy and intuitive way. This research directly addresses the challenges by designing a novel approach and evaluating its instantiation. It directly contributes to practical solutions to aid business operations in an appropriate environment, which has high relevance and value to the problem domain. The theoretical justification of the approach lies in its theoretical basis as summarized at the beginning of the evaluation section (Section 4). An initial evaluation of the instantiation of the approach has indicated that the approach is promising. With some adaptations, the system is applicable in other domains where structured multidimensional data are the basis for decision making (for example, strategy management, performance evaluation, team building, talent management, candidate selection, etc.).
2. For business information visualization, the research has theoretical contributions to the design and use of the multidimensional perceptual map approach for decision support. One of the challenges in business information visualizations is lack of emphasis on how a human solves problems and how information visualization can enhance human problem solving processes (Zhang, 2001). More research has focused on the generation and manipulation of visualizations but less work has been done on the exploratory aspects of the visualization (Jankun-Kelly et al., 2007). There are few formal models to describe the process and the framework to design such visualization systems. This research has resulted in models and processes that explain how such a system can be designed and used for general management analyses and tasks. It offers a unique design approach that uses the multidimensional perceptual map, and tries to define a set of new theoretical concepts and constructs (see Figure 2, Figure 5, and Figure 11). Further theoretical abstraction of the knowledge can contribute to an enhanced understanding of perceptual map based visual information exploration for decision support. This can be the basis of a mid-range design theory (Kuechler and Vaishnavi, 2008) that can guide the development and application of similar approaches and systems, which will be a valuable contribution (Gregor, 2009).
33
5.2 Limitations and future research
The first limitation stems from the problem area itself. There are many issues in project portfolio management, and decision making in a dynamic environment is a complex activity. It has always been one of the challenges for PPM to identify and capture key organizational data, rules, policies, objectives and criteria for prioritization (IDC, 2006). Because of this, we scoped our research to focus on the issue of comprehending multidimensional project data for prioritization decisions, and we chose a visual exploration approach. We have made certain assumptions for using the approach. First, it is assumed that the project data are structured and projects can be evaluated on selected attributes (including those derived from calculations or more complex statistical models). Second, the selection of dimensions is a separate problem which is not addressed in our approach. Many existing methods deal with those problems and our approach can actually work with them. For future research, one potential direction is to use the proposed approach in conjunction with other analytical approaches, such as AHP (Al-Harbi, 2001, Saaty, 2008).
The second limitation lies in the current evaluation of the proposed approach, which relied on qualitative evaluation of an instantiation of the approach (prototype) to gain evidence on two dependent variables, perceived usefulness and perceived ease-of-use. While this work throws light on users intention to use the system (Davis, 1989) and provides useful feedback on the approach and the prototype, it still does not convincingly demonstrate the potential value and effectiveness of the approach. For obtaining the desired evidence (on usefulness and effectiveness of the approach) the prototype needs to be actually used in an organization for project prioritization and selection, and compared with an existing approach or tool used by the organization (to serve as a benchmark). This is what we plan to do in future research after going through an iteration of refinement of the approach and the corresponding tool based on the qualitative evaluation data we have collected. Such work can be considered to constitute later phases of action design research (Sein et al., 2011).
34
5.3 Conclusion
This research identified the challenge in dealing with multidimensional data in project prioritization and selection, and proposed a visual exploration approach to address the challenge. Adopting a design science research perspective (Vaishnavi and Kuechler, 2004), we designed the multidimensional perceptual map approach, argued for its theoretical justification, created a prototype system based on the approach, and evaluated the prototype through qualitative data obtained from interviews of potential users. The rich qualitative data on perceived usefulness and ease-of-use obtained from real-world project managers not only provided initial confirmatory evidence but also provided meaningful and constructive comments, which will be useful for improving the design of the prototype and possibly for refinement of the design approach.
Acknowledgement
The authors would like to thank the anonymous reviewers and the Senior Editors of the special issue for their insightful and constructive comments which have helped in improving the paper. They would also like to thank Mr. Art Vandenberg for his advice and encouragement in conducting this research.
References
Al-Harbi, K. M. A.-S. (2001) Application of the AHP in project management, International Journal of Project Management (19) 1, pp. 19-27.
Baker, J., J. Burkman, and D. R. Jones (2009) Using Visual Representations of Data to Enhance Sensemaking in Data Exploration Tasks, Journal of the Association for Information Systems (10) 7, pp. 533-559.
Borg, I. and P. J. F. Groenen (2010) Modern Multidimensional Scaling: Theory and Applications: Springer.
Bowers, K. S., G. Regehr, C. Balthazard, and K. Parker (1990) Intuition in the Context of Discovery, Cognitive Psychology (22), pp. 72-110.
Brandon, D. (2006) Project Management for Modern Information Systems. Hershey, PA: IRM Press.
Buss, M. (1983) How to Rank Computer Projects, Harvard Business Review (61) 1, pp. 118- 125.
35
Cooper, R. G., S. J. Edgett, and E. J. Kleinschmidt (2000) New Problems, New Solutions: Making Portfolio Management More Effective, Research Technology Management (43) 2, pp. 18-33.
Craft, B. and P. Cairns. (2005 of Conference) Beyond Guidelines: What Can We Learn from the Visual Information Seeking Mantra? in ProceedingsNinth International Conference on Information Visualisation (IV05), London, England, UK, 2005 of Conference
Davis, F. D. (1989) Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology, MIS Quarterly (13) 3, pp. 319-340.
Deboeck, G. and T. Kohonen (1998) Visual Explorations in Finance with Self-Organizing Maps. London: Springer.
Denbo, A. D. and R. W. Guthrie (2003) Prioritizing IT Projects: An Empirical Application of an IT Investment Model, Communications of the International Information Management Association (3) 2, pp. 135-142.
Dickinson, M. W., A. C. Thornton, and S. Graves (2001) Technology Portfolio Management: Optimizing Interdependent Projects over Multiple Time Periods, IEEE Transactions on Engineering Management (48) 4, pp. 518-527.
Dyer, J. S., P. C. Fishburn, R. E. Steuer, J. Wallenius, and S. Zionts (1992) Multiple Criteria Decision Making, Multi-Attribute Utility Theory: The Next Ten Years, Management Science (38) 5, pp. 645-654.
Ghasemzadeh, F. and N. P. Archer (2000) Project Portfolio Selection through Decision Support, Decision Support Systems (29), pp. 73-88.
Gower, J., P. J. F. Groenen, M. V. d. V. l, and K. Vines (2010) Perceptual Maps: The Good, the Bad and the Ugly, ERIM Report Series Reference No. ERS-2010-011-MKT.
Gregor, S. (2009) Building theory in the sciences of the artificial. in Proceedings Proceedings of the 4th International Conference on Design Science Research in Information Systems and Technology, 2009, 2009
Grinstein, G. G. and m. O. Ward (2001) Introduction to Data Visualization, in U. Fayyad, G. G. Grinstein, and A. Wierse (Eds.) Information Visualization in Data Mining and Knowledge Discovery, San Francisco, USA: Morgan Kaufmann.
Hevner, A., S. March, J. Park, and S. Ram (2004) Design Science in Information Systems Research, MIS Quarterly (28) 1, pp. 75-105.
Hoffman, P. E. and G. G. Grinstein (2002) A Survey of Visualizations for High-Dimensional Data Mining, in U. Fayyad, G. G. Grinstein, and A. Wierse (Eds.) Information Visualization in Data Mining and Knowledge Discovery, London, UK: Acamedic Press, pp. 47-82.
IDC. (2006) IT Project and Portfolio Management and the Application Life Cycle: Understanding the Market and Enabling IT/Business Coordination. IDC: 451.
Inselberg, A. (1985) The Plain with Parallel Coordinates, The Visual Computer (1), pp. 69-91. Isaack, T. S. (1978) Intuition: an ignored dimension of management, Academy of Management
Review (3) 4, pp. 917-921.
Jain, A. K., M. N. Murty, and P. J. Flynn (1999) Data Clustering: A Review, ACM Computing
Surveys (31) 3, pp. 264-323.
Jankun-Kelly, T. J., K.-L. Ma, and M. Gertz (2007) A Model and Framework for Visualization
Exploration, IEEE Transactions on Visualization and Computer Graphics (13) 2, pp. 357-369.
36
Jarvenpaa, S. L. and G. W. Dickson (1988) Graphics and Managerial Decision Making: Research Based Guidelines, Communications of the ACM (31) 6, pp. 764-774.
Jolly, D. (2003) The Issue of Weightings in Technology Portfolio Management, Technovation (23), pp. 383391.
Kaplan, R. S. and D. P. Norton (1996) The Balanced Scorecard: Translating Strategy into Action. Cambridge, MA: Harvard Business School Press.
Kaski, S., T. Honkela, K. Lagus, and T. Kohonen (1998) WEBSOM Self-Organizing Maps of Document Collections, Neurocomputing (21) 1-3, pp. 101- 117.
Keim, D. A. (2002) Information Visualization and Visual Data Mining, IEEE Transactions on Visualization and Computer Graphics (8) 1, pp. 1-8.
King, N. (1998) Template Analysis, in G. Symon and C. Cassell (Eds.) Qualitative Methods and Analysis in Organizational Research, London: Sage Publications.
Kohonen, T. (2001) Self-Organizing Maps, Third edition. Vol. 30. Berlin, Germany: Springer. Kohonen, T., J. Hynninen, J. Kangas, and J. Laaksonen. (1996) SOM_PAK: The Self-Organizing
Map Program Package. Helsinki University of Technology: A31.
Kuechler, W. and V. Vaishnavi (2008) On Theory Development in Design Science Research: Anatomy of a Research Project, European Journal on Information Systems (17) 5, pp.
489-504.
Kuo, F.-Y. (1998) Managerial intuition and the development of executive support systems,
Decision Support Systems (24), pp. 89-103.
Leliveld, I. and M. Jeffery (2003) IT Portfolio Management Challenges and Best Practices,
Kellogg/DiamondCluster White Paper.
Lurie, N. H. and C. H. Mason (2007) Visual Representation: Implications for Decision
Making, Journal of Marketing (71), pp. 160-177.
Maio, A. D., R. Verganti, and M. Corso (2002) A Multiproject Management Framework for
New Product Development, in J. S. Pennypacker and L. D. Dye (Eds.) Managing
Multiple Projects, Boca Raton, FL: CRC Press, pp. 127-151.
McFarlan, F. W. (1981) Portfolio Approach to Information Systems, Harvard Business Review
(59) 5, pp. 142-150.
Meyer, A. D. (1991) Visual Data in Organizational Research, Organization Science (2) 2, pp.
218-236.
Oliveira, M. C. F. d. and H. Levkowitz (2003) From Visual Data Exploration to Visual Data
Mining: A Survey, IEEE Transactions on Visualization and Computer Graphics (9) 3,
pp. 378-394.
Robey, D. (1983) Cognitive Style and DSS Design: A Comment on Hubers Paper,
Management Science, (29) 5, pp. 580-582.
Roussinov, D. G. and H. Chen (2001) Information Navigation on the Web by Clustering and
Summarizing Query Results, Information Processing and Management (37), pp. 789-
816.
Saaty, T. L. (1990) How to Make a Decision: The Analytic Hierarchy Process, European
Journal of Operational Research (48) 1, pp. 9-26.
Saaty, T. L. (2008) Decision making with the analytic hierarchy process, International Journal
of Services Sciences (1) 1, pp. 83 98.
Santhanam, R. and J. Kyparisis (1995) A Multiple Criteria Decision Model for Information
System Project Selection, Computers and Operations Research (22) 8, pp. 807-818.
37
View publication stats
Sein, M. K., O. Henfridsson, S. Purao, M. Rossi, and R. Lindgren (2011) Action Design Research, MIS Quarterly (35) 1, pp. 37-56.
Shneiderman, B. (1996) The Eyes Have It: a task by data type taxonomy for information visualizations. in Proceedings IEEE Symposium on Visual Languages, 1996, pp. 336- 343. Boulder, CO, USA, 1996
Soukup, T. and I. Davidson (2002) Visual Data Mining. New York: Jone Wiley & Sons.
Tan, Y. S. and N. M. Fraser (1998) The Modified Star Graph and the Petal Diagram: Two New
Visual Aids for Discrete Alternative Multicriteria Decision Making, Journal of Multi-
Criteria Decision Analysis (7), pp. 20-33.
Tegarden, D. P. (1999) Business Information Visualization, Communications of the AIS (1) 4, Tremblay, M. C., A. R. Hevner, and D. J. Berndt. (2008) The Use of Focus Groups in Design
Science Research. in Proceedings 3rd International Conference on Design Science
Research in Information Systems and Technology, 2008. Atlanta, GA, May 7-9, 2008 Vaidya, O. S. and S. Kumar (2006) Analytic hierarchy process: An overview of applications,
European Journal of Operational Research (169), pp. 1-29.
Vaishnavi, V. and W. Kuechler (2004) Design Research in Information Systems, Retrieved Jan
8, 2007, from http://www.isworld.org/Researchdesign/drisISworld.htm Vaishnavi, V. and W. Kuechler (2008) Design Science Research Methods and Patterns:
Innovating Information and Communication Technology. New York: Auerbach
Publications.
Vesanto, J. (1999) SOM-Based Data Visualization Methods, Intelligent Data Analysis (3) 2,
pp. 111-126.
Vessey, I. (1991) Cognitive Fit: A Theory-Based Analysis of the Graphs Versus Tables
Literature, Decision Sciences (22) 2, pp. 219-241.
Wang, W. and J. Yang (2005) Mining High Dimensional Data, in O. Maimon and L. Rokach
(Eds.) Data Mining and Knowledge Discovery Handbook, Norwell, MA: Kluwer
Academic Publishers, pp. 793-799.
Ward, J. M. (1990) A Portfolio Approach to Evaluating Information Systems Investments and
Setting Priorities, Journal of Information Technology (5) 4, pp. 222-231.
Weir, B. (2004) IT Investment Decisions That Defy Arithmetic, EDUCAUSE Quarterly (27) 1,
pp. 10-13.
Wilson, T. D. (1981) On user studies and information needs, Journal of Librarianship (37) 1,
pp. 3-15.
Yeh, C.-H. (2002) A Problem-based Selection of Multi-Attribute Decision-Making Methods,
International Transactions in Operational Research (9), pp. 169-181.
Zhang, P. (2001) Business Information Visualization: Guidance for Research and Practice,
Encyclopedia of Microcomputers (27), pp. 61-77.
Zhao, H. and S. Ram (2004) Clustering Schema Elements for Semantic Integration of
Heterogeneous Data Sources, Journal of Database Management (15) 4, pp. 88-106.
38
Reviews
There are no reviews yet.