Decson Support by Interval SMART/SWING Incorporatng Imprecson nto SMART and SWING Methods Abstract: Interval judgments are a way of handlng preferental and nformatonal mprecson n multcrtera decson analyss. In ths paper, we study the use of ntervals n SMART and SWING weghtng methods. We generalze the methods by allowng the reference attrbute to be any attrbute, not just the most or least mportant one, and by allowng the decson maker to reply wth ntervals to the weght rato questons to account for hs/her judgmental mprecson. We also study the practcal and procedural mplcatons of usng mprecson ntervals n these methods. These nclude, for example, how to select the reference attrbute n order to dentfy as many domnated alternatves as possble. Based on the results of a smulaton study, we suggest gudelnes for how to carry out the weghtng procedure n practce. Computer support can be used to make the weghtng process vsual and nteractve. We descrbe the WINPRE software for nterval SMART/SWING, PAIRS and preference programmng. The use of nterval SMART/SWING s llustrated by a job selecton example. Keywords: Mult-Crtera Decson Makng, Uncertanty Modelng, Imprecson, Decson Support Systems 1
Decson Support by Interval SMART/SWING Incorporatng Imprecson nto SMART and SWING Methods Jyr Mustajok, Ramo P. Hämälänen and Aht A. Salo Helsnk Unversty of Technology Systems Analyss Laboratory P.O. Box 1100, FIN-02015 HUT, Fnland E-mals: jyr.mustajok@hut.f, ramo@hut.f, aht.salo@hut.f Tel. +358-9-451 3065 Fax +358-9-451 3096 2
INTRODUCTION Multcrtera decson analyss (MCDA) s an approach to systematcally evaluate a set of alternatves wth multple crtera. Interval judgments provde a convenent way to account for preferental uncertanty, or mprecson, and ncomplete nformaton n the weght rato and value estmates (see e.g. Weber, 1987). Then, the assgned ntervals descrbe the range of possble varaton allowed n these estmates due to mprecson. Interval modelng has been appled n varous methods. ARIADNE (Alternatve Rankng Interactve Ad based on DomNance structural nformaton Elctaton) (Sage and Whte, 1984; Whte et al., 1984) was the frst decson support system to use nterval judgments through drect constrants on values and weghts. HOPIE (Holstc Orthogonal Parameter Incomplete Estmaton) (Weber, 1985) was based on holstc nterval judgments on a set of hypothetcal alternatves allowng also constrants for parwse comparsons of the alternatves. Preference programmng (Arbel, 1989; Salo and Hämälänen, 1995, 2003) generalzes parwse comparsons of the AHP (Analytc Herarchy Process) (Saaty, 1980, 1994; Salo and Hämälänen, 1997) to ntervals. In PAIRS (Preference Assessment by Imprecse Rato Statements) (Salo and Hämälänen, 1992), the attrbutes are also compared n pars, but the alternatves are evaluated wthn a value tree framework. In PRIME (Preference Ratos In Multattrbute Evaluaton) (Salo and Hämälänen, 2001), the attrbute weghts are elcted through nterval trade-off comparsons of value dfferences. Lee et al. (2001) and Eum et al. (2001) have developed extended nterval methods for dentfyng domnance and potental optmalty. 3
In ths paper, we dscuss the use of nterval judgments n SMART (Smple Mult- Attrbute Ratng Technque) (Edwards, 1977; von Wnterfeldt and Edwards, 1986) and SWING (von Wnterfeldt and Edwards, 1986) methods. They are smple multattrbute weghtng methods based on rato estmaton. Whle the dea of modelng mprecse nformaton wth ntervals s not new, the use of ntervals explctly n SMART and SWING has not been prevously presented n the lterature. In our dscusson, we deal wth SMART and SWING smultaneously as one method, and refer to ths generalzed method as nterval SMART/SWING. In practce, the true usefulness of the methods s determned by the procedural aspects. Easy-to-use approaches such as SMART and SWING are nowadays the common bass of many appled MCDA studes (Belton and Stewart, 2001). Thus, we beleve that the related generalzed approaches wth ntervals would be of nterest to the practtoners as nterval SMART/SWING preserves the cogntve smplcty of the orgnal methods. Computatonally the nterval SMART/SWING weghtng process leads to smlar optmzaton problem as n PAIRS. Thus, from purely mathematcal pont of vew, ths paper does not provde new methodologcal contrbutons to nterval modelng, and the tools presented wth the PAIRS method can be drectly used n the calculatons. However, from the procedural and practcal elctaton vewponts, the explct consderaton of the nterval SMART/SWING method s justfed especally due to the popularty of SMART and SWING. From these vewponts, the method has characterstcs, whch should be addressed n the assessment of the weght ntervals and n the analyss of the results. These orgnate manly from the fact that n nterval SMART/SWING the preference comparsons are done wth respect to a certan reference attrbute only. The man objectve 4
of ths paper s to dscuss these procedural and practcal aspects of the method. We shall, for example, dscuss the mplcatons of havng a certan reference attrbute, and study the effects of usng dfferent attrbutes as a reference. Based on the results of a smulaton study, we also suggest gudelnes for how to select the reference attrbute. Computer support s needed to solve the overall value ntervals, and t can facltate the process by makng t nteractve and vsual. To help the reader get an dea of the practcal possbltes, we shall also descrbe the WINPRE (Workbench for INteractve PREference Programmng) (Hämälänen and Helenus, 1997) software, whch supports nterval SMART/SWING, PAIRS and preference programmng approaches. Ths paper s organzed as follows. Frst, we descrbe relevant rato estmaton methods. Then, we dscuss the use of ntervals n preference judgments, and practcal and procedural ssues related to the selecton of the reference attrbute. The use of the method wth WINPRE software s demonstrated next by an llustratve example. Fnally, we dscuss the possble extensons to the method and gve the concludng remarks. RATIO ESTIMATION METHODS In multattrbute value theory (MAVT), the overall values of the alternatves are composed of the ratngs of the alternatves wth respect to each attrbute, and of the weghts of the attrbutes. If the attrbutes are mutually preferentally ndependent (see e.g. Keeney and Raffa, 1976), an addtve value functon can be used to calculate the overall values. The overall value for alternatve x s 5
n v( x) = w v ( x ), (1) = 1 where n s the number of attrbutes, w s the weght of attrbute, x s the consequence or the measurement value of alternatve x wth respect to attrbute, and v (x ) s ts ratng. One should note that also other terms, such as a component value, an attrbute value and a score, are used n lterature to characterze v (x ). The sum of the weghts s normalzed to one, and the ratngs are scaled onto the range [0, 1], for example, by usng value functons. Weghts w can be gven drectly by pont allocaton, or by dfferent weghtng procedures such as SMART or SWING. In SWING, the decson maker (DM) s frst asked to consder a hypothetcal alternatve whch has all the attrbutes on ther worst consequence levels. Then, he/she s asked to dentfy the most mportant attrbute,.e. an attrbute whose consequence he/she most preferably would change from ts worst level to ts best level. Ths s gven hundred ponts. Next, the DM s asked to dentfy an attrbute, whose consequence he/she next preferably would change to ts best level. To ths, the DM s asked to assgn fewer ponts to denote the relatve mportance of the change n ths compared to the change n the most mportant attrbute. The procedure contnues smlarly on the other attrbutes. The actual attrbute weghts are elcted by normalzng the sum of ponts to one. In SMART, the DM gves ten ponts to the least mportant attrbute. Then, he/she gves more ponts to the other attrbutes to address ther relatve mportance. The weghts are elcted by normalzng the sum of the ponts to one. However, t has been stressed that the comparson of the mportances of the attrbutes s meanngless, f t does not reflect the consequence ranges of the attrbutes as well (von Wnterfeldt and Edwards, 1986; 6
Edwards and Barron, 1994). These can be ncluded by applyng SWING weghtng to SMART. That s, n the comparson of the mportances of the attrbutes, the DM should explctly focus on the attrbute changes from ther worst consequence level to the best level. Edwards and Barron (1994) named ths varant as SMARTS (SMART usng Swngs), but the term SMART s also commonly used for ths method. In ths paper, we use the term SMART to refer to both of these. SMART and SWING are algebrac methods,.e. the weghts are derved from a set of a mnmum number (n-1) of lnearly ndependent judgments on preference relatons wth some smple system of equatons (Weber and Borcherdng, 1993). Another way to elct the weghts s to derve them from a larger set of judgments wth an estmaton method (see Table 1). In an extreme case, the set of all the possble n (n-1)/2 parwse judgments s used. For example, n the AHP the weghts are elcted from ths set wth the egenvalue procedure (Saaty, 1980). Interval estmate methods can be classfed n the same way. Interval SMART/SWING uses the mnmum number of judgments, but there are also nterval methods whch allow more judgments, such as PAIRS and preference programmng. The mnmum number of judgments n nterval methods s 2 (n-1), as both the upper and lower bounds are gven for each preference relaton. Table 1. A set of rato methods classfed by the type of judgments used. Mnmum number of parwse judgments More than mnmum number of judgments allowed Exact pont estmates SMART, SWING AHP, Regresson analyss Interval estmates Interval SMART/SWING PAIRS, Preference programmng 7
INTERVAL SMART/SWING In nterval SMART/SWING, we generalze SMART and SWING methods () by allowng the reference attrbute to be any attrbute, not just the most or least mportant one, and () by allowng the DM to use nterval judgments on the weght rato questons and on the evaluaton the alternatves, to represent related mprecson. Reference Attrbute In SMART and SWING the reference attrbute s the least and the most mportant one, respectvely. However, n many cases t would be easer to use an easly measurable attrbute, for example money, as a reference. Ths knd of an approach has also been recommended n other methods. For example, n the trade-off and Even Swap methods, t has been suggested to make the easest trade-offs frst (see e.g. Keeney, 1992; Hammond et al., 1998, 1999). In nterval SMART/SWING, we consder SMART and SWING as one method by allowng the reference attrbute to be any one of the attrbutes. In ths generalzaton the reference attrbute s gven a fxed number of ponts, whle the other attrbutes receve ponts that reflect ther relatve mportance. The weghts are then elcted by normalzng the sum of the ponts to one. In practce, any number of ponts can be assgned to the reference attrbute, as far as the ponts assgned to the other attrbutes are relatve to these ponts. For example, f the DM s famlar wth the SMART method, t s natural to assgn 10 ponts to the reference attrbute n nterval SMART/SWING, too. 8
Techncally, the weghts are calculated dentcally as n SMART and n SWING, and also n drect weghtng and n the pont allocaton method (Schoemaker and Wad, 1982), for example. Thus, the dstncton between these methods s based on procedural dfferences only. If the DM s consstent n hs/her weghtng, the weghts elcted on the bass of dfferent reference attrbutes should be the same. However, behavoral research has shown that dfferent weghtng methods may gve dvergng results (see e.g. Weber and Borcherdng, 1993; Pöyhönen and Hämälänen, 2001). One explanaton for ths s that the DMs tend to gve ponts n multples of ten, whch mples that the set of possble ratos between the ponts becomes lmted. For example, n SMART ths set s (10/10=1, 20/10=2, 30/10=3, ) and the correspondng recprocals, whereas n SWING t s (100/100=1, 100/90=1.11,, 100/10=10),.e. a completely dfferent set. The DMs may also restrct only to the rankng of the attrbutes rather than to the strength of the preferences. For example, the DM can gve values 100, 90 and 70 wth SWING and values 40, 20 and 10 wth SMART for the same attrbutes, even f the weght ratos between the attrbutes are clearly not the same (Pöyhönen et al., 2001). Thus, here as well as wth any other MCDA method, the DM should be well nformed about the proper use of the method to avod such procedural bases. The DM hm/herself should understand the meanng of the weghts, as the recognton of bases by an outsde observer s lkely to be very dffcult. Ths s especally mportant when usng any attrbute as a reference, as the ponts gven can be both hgher and lower than those for the reference attrbute. 9
Interval Judgments Another generalzaton of nterval SMART/SWING s to allow the DM to reply wth ntervals to the rato questons to descrbe the possble mprecson n these. These ntervals set constrants for the feasble weghts of the attrbutes, and smlar constrants can be set for the ratngs of the alternatves. As a result, the overall values of the alternatves wll also be ntervals descrbng the possble varaton n these due to allowed varaton n the attrbute weghts and the ratngs of the alternatves. Domnance concepts can be appled to further analyze the relatons between the alternatves. In nterval SMART/SWING, the reference attrbute s gven a fxed number of ponts, but the ponts for the other attrbutes are gven as ntervals representng the mprecson n the judgments. From these we can derve constrants for the attrbutes weght ratos n a straghtforward manner by takng the extreme ratos of the ponts gven to the reference attrbute and the other attrbutes,.e. ref max wref ref, (2) w mn where ref stands for the ponts gven to the reference attrbute and max (mn ) for the maxmum (mnmum) number of ponts gven to a non-reference attrbute. For example, f the reference attrbute s gven 1.0 pont and attrbute an nterval from 0.5 to 3 ponts, the constrants for the weght rato become 1/3 w ref /w 1/0.5 = 2, or wth an other notaton w ref /w [1/3, 2]. The gven constrants n (2), n addton to the weght n = 1 normalzaton constrant w = 1, determne the feasble regon of the weghts, S. 10
Imprecson n the ratngs of the alternatves can also be modeled wth ntervals by gvng lower and upper bounds for these. In practce, these can be assgned drectly (e.g. 0.2 v (x ) 0.5) or, for example, by settng bounds for the value functons from whch to derve the constrants for the ratngs. The results can be presented as the upper and lower bounds for the overall values of the alternatves. An addtve value functon can be used to calculate these by assumng the mutual preferental ndependence of the attrbutes. The lower bound for the overall value of alternatve x (v(x)) s elcted as the mnmum value, when allowng the weghts and attrbute values to vary wthn the gven constrants. That s, w S n v( x) = mn w v ( x ), (3) = 1 where v (x ) s the lower bound for v (x ), and w=(w 1,, w n ) S, whch s the feasble regon bounded by the constrants n (2) for all =1,,n and the normalzaton constrant n = 1 w = 1. The upper bound s obtaned analogously. The mnmzaton problem (3) can be solved by lnear programmng. Techncally ths problem s smlar to the one of the PAIRS method, and for computatonal detals, see Salo and Hämälänen (1992). An analyss of the alternatves value ntervals can be employed to determne the domnance relatons (see e.g. Weber, 1987; Salo and Hämälänen, 1992). Alternatve x domnates alternatve y f the value of x s greater than the value of y for every feasble combnaton of the weghts,.e. f w S n mn w ( v ( x ) v ( y )) > 0 (4) = 1 11
for any set of feasble weghts w S. More specfcally, one can say that ths s a defnton of parwse domnance. Example To llustrate the nterval SMART/SWING analyss, consder a case wth two alternatves (A and B) and three attrbutes (1, 2 and 3) (Fgure 1). Attrbute 1 s chosen as the reference attrbute and gven 1.0 pont. Attrbute 2 s gven an nterval from 0.5 to 2.0 ponts and attrbute 3 an nterval from 1.0 to 3.0 ponts to reflect judgmental mprecson n the mportances of these. In the WINPRE software (Fgure 1), the selected button ndcates the reference attrbute, and the dark colored bars represent the ranges of the ponts gven to the attrbutes. The weght rato constrants are derved from the ratos of these accordng to (2), and they are w 1 /w 2 [1.0/2.0, 1.0/0.5] = [1/2, 2] and w 1 /w 3 [1.0/3.0, 1.0/1.0] = [1/3, 1]. These constrants defne the feasble regon of the weghts S. Fgure 2 shows the feasble regon on the smplex representng the weght space, where the sum of the weghts s normalzed to 1. For example, the top vertex of the weght space s pont w=(1, 0, 0). The ratngs of the alternatves are v A) = v ( A) 0. 0, 1 ( 1 = v B) = v ( B) 1.0, v A) = v ( A) 1. 0, v B) = v ( B) 0. 8, v A) = v ( A) 1. 0 and 1 ( 1 = v 3 ( B) = v3( B) = 2 ( 2 = 2 ( 2 = 3 ( 3 = 0.0. For smplcty, we assume here that there s no mprecson n the ratngs, whch mples that the upper and lower bounds of these are the same. As a result we get the overall values ntervals of [0.60, 0.83] for alternatve A and [0.31, 0.65] for alternatve B (see Fgure 1). 12
Fgure 1. Interval SMART/SWING analyss wth three attrbutes (1, 2 and 3) and two alternatves (A and B). A screen capture from the WINPRE software. Fgure 2. Feasble regon S on the smplex representng the weght space where w 1 +w 2 +w 3 =1. There can be domnance even f the overall value ntervals overlap. In ths example alternatve A domnates alternatve B although the lower bound of the value of A s smaller that the upper bound of the value of B. Thus, the value of A s greater than the value of B for any sngle weght combnaton wthn the feasble regon S. For example, 13
the hghest value of B (0.65) s obtaned at the pont w=(0.25, 0.5, 0.25), but at ths same pont the value of A s stll hgher (0.75). Remarks Techncally, the mnmzaton problem (3) n nterval SMART/SWING s the same as the one n the PAIRS method (Salo and Hämälänen, 1992). However, from the procedural pont of vew, the relaton between nterval SMART/SWING and PAIRS can be seen analogous to the relaton between SMART (or SWING) and drect weghtng. Ths apples, f we consder the attrbute weghts n the pontwse methods to correspond to the weght rato constrants n the nterval methods. That s, n drect weghtng and PAIRS, the DM explctly gves the attrbute weghts and the weght rato constrants, respectvely, whereas n SMART and n nterval SMART/SWING, these are obtaned as a result of a specfc weght elctaton process. Ths process ncludes the selecton of the reference attrbute, assgnng ponts or pont ntervals to the attrbutes, and the elctaton of the attrbute weghts or the constrants for the weght ratos from these. However, n nterval SMART/SWING, only constrants nvolvng the reference attrbute are elcted, whereas n PAIRS, the constrants can be assgned to the weght ratos of any attrbute pars. In nterval SMART/SWING, the preference relatons between the non-reference attrbutes are not explctly stated, whch s characterstc to all algebrac rato estmaton methods. However, the upper bounds for the weght ratos between these can be mplctly derved from the constrants elcted wth equaton (2). For example, an upper bound for the weght rato between two non-reference attrbutes 2 and 3 (w 2 /w 3 ) can be derved from constrants ref/max 2 w ref /w 2 and w ref /w 3 ref/mn 3, from whch we get w 2 max 2 /ref 14
w ref max 2 /ref (ref/mn 3 w 3 ) w 2 /w 3 max 2 /mn 3. In our example, max 2 =2 and mn 3 =1, from whch we get the bound w 2 /w 3 2. The lower bound can be calculated smlarly, and as a result we get the weght rato nterval w 2 /w 3 [1/6, 2] represented by the dotted lnes n Fgure 2. However, these constrants are clearly redundant as they do not restrct the feasble regon more than the other constrants do. The use of a mnmum number of preference judgments also mples that the feasble regon wll never become empty, as may occur n methods usng more judgments (e.g. PAIRS). On the other hand, n these methods an empty feasble regon would ndcate nconsstency n the DM s preference assessment. In such a case the DM s requested to evaluate hs/her preferences. As n algebrac methods the DM cannot gve nconsstent judgments, t would often be useful to separately check the consstency of the statements. Ths can be carred out by assessng a few weght rato constrants also between the nonreference attrbutes, even f ths s not explctly requred by the method (Weber and Borcherdng, 1993). In ths paper, we only concentrate on non-herarchcal value trees havng one attrbute level. However, the method can also be appled n herarchcal trees wth many attrbute levels, smlarly as PAIRS. Then, the nterval weghtng s carred out on each branch of the value tree separately. For computatonal detals see Salo and Hämälänen (1992). HOW TO SELECT THE REFERENCE ATTRIBUTE A common goal n MCDA s to dentfy domnated alternatves. In nterval SMART/SWING, the choce of the reference attrbute may affect to the occurrence of the 15
domnances. Now we shall dscuss how the reference attrbute can be effcently selected,.e. so that as many domnated alternatves as possble are dentfed wth as few procedural actons as possble. In general, the smaller the feasble regon s, the more domnated alternatves are lkely to be dentfed. Therefore, a natural way s to select the reference attrbute so that mprecson ntervals become as tght as possble. However, procedurally the evaluaton of these ntervals s carred out only after selectng the reference attrbute. Thus, n general ths nformaton cannot be assumed to be avalable at ths phase of the process. Yet there are also cases, where the DM may be able to easly dentfy the attrbute wth least mprecson beforehand. For example, the above-mentoned money may be such an attrbute to many DMs. Fgure 3. The dark colorng ndcates an area, where alternatve A domnates alternatve B. On the other hand, the shape of the feasble regon and ts poston on the weght space also have an effect on the occurrence of the domnances. To llustrate ths, let us further consder our example n the prevous secton (Fgure 2). From the whole weght space we 16
can separate an area where alternatve A domnates alternatve B (shaded area n Fgure 3). Ths area can be formed accordng to (4),.e. by ncludng all the weght vectors w = n = 1 (w 1,,w n ), such that w ( v ( A) v ( B)) > 0, n t. As the constranng functon s lnear, ths area s obtaned by constranng the weght space wth the correspondng hyperplane. If the feasble regon of the weghts s now wthn ths area as a whole, the correspondng domnance occurs. Ths s clearly the case n our example (Fgure 3). Thus, f consderng the shape of the feasble regon, t would be desrable that the feasble regon would be evenly stretched nto all drectons, so that t would be entrely ncluded n as many domnance areas as possble. The sze of the feasble regon could be analytcally measured, for example, by an area (or a content n general case), or t could be approxmated by usng some measure, e.g. by the consstency measure of Salo and Hämälänen (1997). However, there are not straghtforward analytcal ways to smultaneously take nto account the shape and the poston of the regon, too. Thus, we carred out a smulaton experment to study the effects of selectng dfferent reference attrbutes. Smulaton Study The objectve of the smulaton study was to fnd out what would be the best choce for the reference attrbute. The strateges compared were the ones where the th most mportant attrbute ( = 1,,n) was chosen as a reference attrbute. Thus, we assumed that the DM can specfy the rankng, or some ranks, for the relatve mportances of the attrbutes. Imprecson on each strategy was modeled by assgnng ntervals on the weght ratos between the reference attrbute and each other attrbute. 17
We generated a set of problems, and n each problem nstance each strategy was measured by two dfferent effcency measures. The goal was to determne, whether there s statstcal dfference between the average effcences of the strateges. In addton, we also studed the effects of the problem sze, whch was characterzed by the number of the attrbutes (n) and the number of the alternatves (m). In practce, 1000 smulaton rounds were conducted on each combnaton of the values of n=3, 5, 8 and m=3, 5, 8. We dd not study any larger problems, because the effects of parameter varaton already emerged wth these values. The smulatons were carred out wth the MATLAB software. On each smulaton round, the problem nstance was generated as follows. We randomly generated pontwse (.e. the lower and upper bounds were the same) measurement values x from (0, 1) normal dstrbuton for each alternatve x on each attrbute. Thus, these values were ndependent of each other. The ratngs v (x ) were then derved from these by mappng the measurement value ranges lnearly to nterval [0, 1]. That s, v ( x ) = ( x x ) /( x x ) (5) where x and x represent the maxmum and mnmum measurement values of attrbute, respectvely. By assumng that unt ncreases n the measurement values are equally preferred on each attrbute, the weght for an attrbute (w o ) s relatve to the range of the correspondng measurement values,.e. w o n j = 1 = ( x x ) / ( x j x ). (6) j 18
Thus, as a result of ths process we got a problem nstance havng pontwse estmates for both the weghts of the attrbutes and the ratngs of the alternatves. The mprecson n each problem nstance was modeled by assgnng error rato R (see Salo and Hämälänen, 2001) on all the ratos between the generated weghts of the reference attrbute (w o ref) and any other attrbute (w o ). Thus, we assumed that relatvely each weght rato had equal mprecson assgned. In practce, each weght rato was multpled by factor R to get the upper bound for t, and dvded by R to get the lower bound. That s, o 1 w R w ref o o wref wref R, = 1,..., n, ref o w w (7) where w o s the ntally generated pontwse weght of attrbute, and ref denotes the reference attrbute. For example, f w o ref=0.5 and w o 2=0.2, the weght rato nterval wth R=1.5 for w ref /w 2 was [(1/1.5) (0.5/0.2), 1.5 (0.5/0.2)] = [1.67, 3.75]. As a result, we got constrants on the same weght ratos that would have been assgned wth nterval SMART/SWING. The smulatons were carred out wth error rato R=1.5. In addton, to study the possble effects of error rato R, smulatons wth R=1.2, 1.4, 1.8, 2 and 3 were carred out for the case n=m=5. Ths settng appears realstc n many cases, as real events are often normally dstrbuted. However, some settngs would mply essentally dfferent dstrbutons for the weghts and the ratngs. For example, a settng havng specfed gradng scales on each attrbute would requre a smulaton settng wth fxed ranges for the measurement values. Thus, the smulaton was also tested by usng other dstrbutons, for example, the unform 19
dstrbuton both on the weght space and on the ratngs of the alternatves. However, conclusons drawn from the smulatons carred out wth these other dstrbutons were essentally the same. Two further strateges were also studed: a strategy where mprecson ntervals were gven for all the parwse judgments (PAIRS), and a strategy where the constrants were sequentally gven for adjacent pars of attrbutes,.e. between the most and the second mportant ones, the second and the thrd mportant ones, and so on. The objectve was to have a reference to technques not havng a certan reference attrbute selected. However, we do not dscuss how to elct the constrants n these strateges n practce, but take these as drectly gven. The effcency of each strategy was measured by two dfferent measures. The frst one was the average number of domnated alternatves obtaned wth each strategy. The second one was the average of the maxmum loss of value,.e. the maxmum value dfference between ntally (at pont w o ) the best alternatve x * and all the other alternatves, * * max ( v( x) v( x )), w S, x X \ { x } (8) where S s the feasble regon of the weghts, and X the set of all the alternatves. If the maxmum loss of value s negatve, the value of alternatve x * s greater than any other alternatve at every pont of the feasble regon,.e. t domnates all the other alternatves. The smulaton results are presented n Tables 2 and 3. The strateges are named after ther rank n the order of mportance. For example, strategy 1 represents the strategy where the most mportant attrbute,.e. an attrbute havng the hghest ntal weght w o, was chosen as a reference. Seq and All stand for the strateges usng sequental and all the 20
possble judgments, respectvely. The percentages n Table 2 represent the share of the maxmum number of alternatves that can be made domnated wth each strategy (m 1). Table 2. The average numbers of domnated alternatves wth each strategy. Strategy R m n 1 2 3 4 5 6 7 8 All Seq 1.5 3 3 1.683 1.657 1.571 1.729 1.657 (84.2%) (82.9%) (78.6%) (86.5%) (82.9%) 5 1.423 1.386 1.339 1.299 1.263 1.592 1.340 (71.2%) (69.3%) (67.0%) (65.0%) (63.2%) (79.6%) (67.0%) 8 1.129 1.109 1.069 1.037 1.029 0.997 0.977 0.950 1.437 0.958 (56.5%) (54.5%) (53.5%) (51.9%) (51.5%) (49.9%) (48.9%) (47.5%) (71.9%) (47.9%) 5 3 3.506 (87.7%) 5 3.117 (77.9%) 8 2.611 (65.3%) 8 3 6.362 (90.9%) 5 5.815 (83.1%) 8 5.113 (73.0%) 3.484 (87.1%) 3.037 (75.9%) 2.578 (64.5%) 6.323 (90.3%) 5.778 (82.5%) 5.066 (72.4%) 3.423 (85.6%) 3.023 (75.6%) 2.557 (63.9%) 6.295 (89.9%) 5.739 (82.0%) 5.018 (71.7%) 2.989 (74.7%) 2.527 (63.2%) 5.702 (81.5%) 5.016 (71.7%) 2.945 (73.6%) 2.495 (62.4%) 5.659 (80.8%) 4.958 (70.8%) 2.501 (62.5%) 4.957 (70.8%) 2.460 (61.5%) 4.937 (70.5%) 2.430 (60.8%) 4.888 (69.8%) 3.599 (90.0%) 3.400 (85.0%) 3.167 (79.2%) 6.489 (92.7%) 6.255 (89.4%) 5.968 (85.3%) 3.484 (87.1%) 2.972 (74.3%) 2.294 (57.4%) 6.323 (90.3%) 5.661 (80.9%) 4.581 (65.4%) 1.2 5 5 3.615 (90.4%) 1.4 5 5 3.250 (81.3%) 1.8 5 5 2.681 (67.0%) 2 5 5 2.455 (61.4%) 3 5 5 1.679 (42.0%) 3.606 (90.2%) 3.205 (80.1%) 2.596 (64.9%) 2.359 (59.0%) 1.622 (40.6%) 3.585 (89.6%) 3.176 (79.4%) 2.595 (64.9%) 2.352 (58.8%) 1.575 (39.4%) 3.568 (89.2%) 3.158 (79.0%) 2.524 (63.1%) 2.291 (57.3%) 1.594 (39.9%) 3.558 (89.0%) 3.095 (77.4%) 2.451 (61.3%) 2.180 (54.5%) 1.516 (37.9%) 3.745 3.572 (93.6%) (89.3%) 3.512 3.153 (87.8%) (78.8%) 3.092 2.520 (77.3%) (63.0%) 2.928 2.246 (73.2%) (56.2%) 2.277 1.525 (56.9%) (38.1%) For each problem sze, the same ntal weghts (w o ) were used as a bass on whch the mprecson ntervals were assgned for all the strateges. Thus, n a sngle problem nstance the effcences of the strateges were not ndependent of each other, and tradtonal statstcal measures (e.g. standard devaton) for these are meanngless. In addton, accordng to some normalty tests (Lllefors, Jarque-Bera) ths data cannot be assumed to be normally dstrbuted. However, to get statstcal foundaton for the analyss 21
of the results, nonparametrc tests can be used to study the dfferences n the effcences between any two strateges. In practce, we calculated the dfferences both n the maxmum loss of value and n the number of domnated alternatves for each strategy par n each problem nstance. Then we used the Wlcoxon sgn rank test to test whether the averages of these dfferences sgnfcantly dffered from zero. Table 3. The average of the maxmum loss of value wth each strategy. Strategy R m n 1 2 3 4 5 6 7 8 All Seq 1.5 3 3-0.167-0.157-0.129-0.187-0.157 5-0.049-0.038-0.024-0.014 0.001-0.100-0.020 8 0.032 0.041 0.049 0.053 0.059 0.065 0.069 0.076-0.042 0.088 5 3-0.067-0.061-0.045-0.088-0.061 5 0.008 0.014 0.020 0.027 0.034-0.039 0.033 8 0.053 0.058 0.061 0.064 0.067 0.068 0.072 0.076-0.010 0.108 8 3-0.033-0.027-0.019-0.052-0.027 5 0.023 0.028 0.032 0.036 0.040-0.018 0.045 8 0.058 0.060 0.062 0.064 0.065 0.066 0.068 0.071 0.002 0.111 1.2 5 5-0.070-0.068-0.066-0.063-0.060-0.091-0.060 1.4 5 5-0.017-0.012-0.007-0.002 0.004-0.055 0.004 1.8 5 5 0.073 0.083 0.091 0.102 0.113 0.005 0.110 2 5 5 0.111 0.123 0.132 0.146 0.158 0.030 0.154 3 5 5 0.255 0.272 0.284 0.307 0.329 0.131 0.306 Dscusson The smulaton results show that f the error ratos on all the preference judgments are the same, a strategy of havng a more mportant attrbute as a reference s generally sgnfcantly more effcent (wth alpha level 0.05). In the statstcal tests on the loss of value, ths appled on all problem szes and strategy pars, except for strategy par 4 5 n the case n=m=8. In the tests on the number of domnated alternatves, there were a few cases, where the strategy wth a more mportant reference attrbute was not sgnfcantly more effcent. These cases occurred manly n the cases n=8 between the strateges where 22
an ntermedate attrbute was as a reference. However, for example, n all the tests between strategy 1 and a strategy where the least mportant attrbute was as a reference, strategy 1 dentfed sgnfcantly more domnated alternatves. In ths respect the most effcent way s to choose the most mportant attrbute as a reference attrbute. On the average, ths strategy dentfed most domnances and gave the smallest losses of value n all the smulaton runs usng dfferent parameters. However, the use of the strategy assumes that the most mportant attrbute can be dentfed, but often ths can be easly done. The use of the most mportant attrbute as a reference also has other advantages. It s certanly meanngful to the DM, whereas the comparsons to some less mportant attrbute may become mprecse due to neglgble mportance of ths. The DM has also presumably gven more thought to the most mportant attrbute than to the less mportant ones, and through ths may have reduced the related mprecson. On the other hand, the results also show that even a small reducton n the error rato affects the effcency more than choosng the most mportant attrbute as a reference. For example, n the case n=m=5, R=1.4, any strategy (except strategy 5) dentfed more domnances than strategy 1 n the case n=m=5, R=1.5. Thus, f the DM can easly pck out an attrbute contanng least mprecson, ths s lkely to be worth choosng as a reference attrbute nstead of the most mportant one. To sum up these observatons, we suggest the followng rules to select the reference attrbute: 23
1. If the DM can easly dentfy an attrbute contanng least mprecson, ths should be selected as a reference attrbute. 2. If the mprecson related to the attrbutes cannot be dfferentated, the most mportant attrbute should be selected as a reference attrbute. If the DM can dentfy nether the attrbute contanng least mprecson nor the most mportant attrbute, the attrbutes are lkely to be such on equal terms that no specfc recommendatons can be gven. As a result of the ntal weghtng process, there may stll be non-domnated alternatves so that several further adjustments to the parameters are requred untl the best alternatve can be dentfed. The DM can try to gve more precse preference judgments, for example, by tghtenng the already stated constrants. Another way s to try to reduce mprecson related to the values of the alternatves. Especally, f the set of alternatves has been reduced by elmnatng domnated alternatves, the workload needed to consder the mprecson related to the alternatves s also smaller. Decson rules can also be appled to rank alternatves for whch domnances do not hold (see Salo and Hämälänen, 2001). Rules based on centralzaton, such as the central values of the overall value ntervals or the use of the central weghts, can also be recommended here. However, some other rules such as maxmn or maxmax (.e. maxmzng the mnmum or the maxmum of the overall value nterval) may cause some bas when appled n nterval SMART/SWING. Ths s because there are no explct constrants between non-reference attrbutes, and thus these wll generally have wder 24
weght rato ntervals. Consequently, the alternatves strong n these attrbutes wll also have wder ntervals for the overall values. Comparson to the Strateges Usng Sequental and All the Possble Judgments The strateges usng a certan reference attrbute were also compared to the strateges usng sequental and all the possble judgments. In the sequental strategy, the number of explctly gven judgments s 2 (n-1),.e. the same as when usng a reference attrbute. However, all the explctly gven judgments are needed to elct an upper bound for the weght rato constrant between the most and least mportant attrbute. Consequently, ths constrant would nclude all the mprecson n these judgments. In comparson, by usng a reference attrbute, the bounds for the weght ratos between any non-reference attrbutes are elcted only from two explctly gven judgments, as demonstrated n the prevous Secton. Thus, by default the sequental strategy should produce wder ntervals than strateges usng a certan reference attrbute, and the more attrbutes we have the more neffcent the sequental strategy should be. Ths s supported by the smulaton results. In the case n=3, there s no dfference as then the sequental strategy actually corresponds wth strategy 2, but n the case n=8 the sequental approach s generally the most neffcent one. The DM can also carry out parwse preference comparsons between all the attrbutes. Then the number of the gven judgments ncreases from 2 (n-1) to n (n-1). For example n the case n=8 ths means an ncrease from 14 judgments to 56. However, as the frst 2 (n-1) judgments are gven by usng the same reference attrbute, these already set some 25
bounds for all the attrbute pars, whereas the further judgments only tghten these. Thus, expectedly the further judgments shall not be as effcent n dentfyng new domnances as the 2 (n-1) frst judgments. The smulaton results also clearly support ths. For example, n the case n=m=8, R=1.5, by gvng the frst 25% (14 of 56) of all the possble parwse judgments, 73.1% of the domnances were dentfed. If one further gave all the rest of the judgments, the percentage ncreased only to 85.3%. Thus, the result suggests that nstead of assgnng constrants on all the possble attrbute pars, the DM should consder other ways of tryng to reduce the mprecson, e.g. n the ratngs of the alternatves. From the behavoral vewpont, t s plausble to assume that the mprecson related to some attrbute decreases when more preference judgments are gven, because ths attrbute becomes more famlar. In the case of sequental strategy, ths effect would further reduce ts effcency, as the attrbutes under judgment change all the tme. In the case of gvng all the parwse judgments there may be some nfluence n favor of ths strategy. However, we see that ths effect s so small compared to the extra workload needed to gve all the judgments that we stll cannot suggest usng ths strategy to further reduce the mprecson. EXAMPLE WITH COMPUTER SUPPORT As an example, we consder Vncent Sahd s job decson problem (Fgure 4) adapted from Hammond et al. (1998). Vncent s task s to select the best job from fve alternatves evaluated wth respect to sx attrbutes (Table 4). In the orgnal example, the problem was approached wth the Even Swaps method (Hammond et al., 1998, 1999). Now we descrbe how to use nterval SMART/SWING to model the possble mprecson n the example. 26
One should note that the gven ntervals are based on our subjectve nterpretaton of the case descrpton, as the approach n Hammond et al. (1998) dd not gve them explctly. Fgure 4. Value tree for Vncent Sahd s job decson. We llustrate the process by usng the WINPRE software, avalable n the Decsonarum Web ste (Hämälänen, 2004). WINPRE provdes a graphcal user nterface to support dfferent phases of the analyss, for example, the creaton of the value tree, the elctaton of the attrbute weghts and the analyss of results. The analyss of the results s truly nteractve, as WINPRE gves nstantaneous feedback on how the overall values and domnance relatons change due to changes n attrbute weghts and n alternatves ratngs. Another software developed later by our research team to support nterval rato methods s PRIME Decsons (Salo et al., 1999). It supports the PRIME method (Salo and Hämälänen, 2001), and allows nterval SMART/SWING to be used n the weght elctaton. For a detaled dscusson of PRIME Decsons, see Gustafsson et al. (2001). Interval SMART/SWING s sutable for ths problem for several reasons. Frst, there are dfferent types of mprecson related to the attrbutes, whch can all be modeled wth ntervals. For a general dscusson on the orgns of mprecson, see e.g. Wallsten (1990) 27
or French (1995). Secondly, there are relatvely many attrbutes. Thus, wth nterval SMART/SWING the number of attrbute comparsons does not become too hgh, as t depends only lnearly on the number of attrbutes. Table 4. Consequences table for Vncent Sahd s job decson (Hammond et al., 1998). Job A Job B Job C Job D Job E Monthly salary $2000 $2400 $1800 $1900 $2200 Flexblty of work schedule Busness sklls development Vacaton (annual days) Benefts Moderate Low Hgh Moderate None Computer Manage people, computer Operatons, computer Organzaton 14 12 10 15 12 Health, dental, retrement Health, dental Health Health, retrement Tme management, multple taskng Enjoyment Great Good Good Great Borng Health, dental Let us frst look at the mprecson related to the measurement of the alternatves. In attrbutes busness sklls development and benefts, there may be mprecson, for example, due to ncomplete job descrptons. Ths mprecson can be modeled by usng ntervals to cover the possble dfferences between the gven job descrptons and the realty. In practce, the process s carred out by frst thnkng on each attrbute what are the best and the worst possble consequence levels for ths, and by settng the ratng nterval [0, 1] accordng to these. Then, the DM should go through all the alternatves one by one, and assgn the ratng ntervals on these. A plausble way for ths s, for example, to frst set the ntervals very wde, and then tghten these bt by bt untl t cannot be stated for sure that the true value belongs to ths nterval. 28
Fgure 5. Interval evaluaton for the attrbute enjoyment. Attrbutes flexblty of work schedule and enjoyment are evaluated by classfyng the alternatves nto a set of verbal explanatons. However, often there s some mprecson around these explanatons. For example, two alternatves may both be classfed as good on some attrbute, although n practce the other one may be somewhat better. Ths mprecson can be modeled by assocatng a ratng nterval wth each of the verbal explanatons. For example, on attrbute enjoyment we have used ntervals: borng = [0.0, 0.2], good = [0.5, 0.7] and great = [0.8, 1.0]. Wth WINPRE, these ntervals can be gven ether graphcally or numercally (Fgure 5). In practce, the mappng from verbal statements to numercal values s typcally subjectve. However, f some general scale s appled, the most convenent way s usually to dvde the numercal scale nto ntervals of the same sze, and then use a lnear mappng from verbal statements to these ntervals. Exact pont estmates can also be used by settng the upper and lower bounds of the ntervals the same. In ths example, the consequences n attrbutes salary and vacaton for each alternatve are pontwse estmates, whch are mapped lnearly on the value scale. Table 5 presents each alternatve s value ntervals on the attrbutes. 29
Table 5. Value ntervals for the attrbutes. Attrbute and ts range Job A Job B Job C Job D Job E Monthly salary [$1800, $2400] [1/3,1/3] [1,1] [0,0] [1/6,1/6] [2/3,2/3] Flexblty of work schedule [0, 1] [0.5,0.7] [0.2,0.4] [0.8,1.0] [0.5,0.7] [0.0,0.0] Busness sklls development [0, 1] [0.3,0.7] [0.7,1.0] [0.5,0.8] [0.0,0.3] [0.6,0.9] Vacaton [10, 15] [0.8,0.8] [0.4,0.4] [0.0,0.0] [1.0,1.0] [0.4,0.4] Benefts [0, 1] [0.8,1.0] [0.3,0.4] [0.0,0.0] [0.5,0.6] [0.3,0.4] Enjoyment [0, 1] [0.8,1.0] [0.5,0.7] [0.5,0.7] [0.8,1.0] [0.0,0.2] In attrbute weghtng, there may be mprecson, for example, due to the DM s nablty to assess hs/her weghts precsely. Fgure 6 presents the nterval SMART/SWING weghtng of the attrbutes n our example. Monthly salary s chosen as the reference attrbute for two reasons. Frst, t s an easly measurable and understandable attrbute. Thus, the attrbute comparsons can be expected to be an easer process than n the case where busness sklls development, for example, s to be compared wth the other attrbutes. Secondly, salary s the most mportant attrbute (jontly wth enjoyment), and thus all the comparsons are carred out to less mportant attrbutes. Fgure 6. Interval SMART/SWING weghtng n Vncent Sahd s job selecton example. 30
In practce, attrbute weghtng s carred out by frst choosng the reference attrbute, and then assgnng ntervals for the other attrbutes one by one. WINPRE allows a graphcal or numercal approach to ndcate the mportance ntervals for the other attrbutes. A smlar nterval tghtenng procedure as n the case of alternatves can also be used on attrbute weghtng. However, all the tme the DM should bear n mnd to compare the ntervals to the ponts gven to the reference attrbute n respect of the consequence ranges on these attrbutes. As a result we get the overall value ntervals for the alternatves and the possble domnance relatons (Fgure 7). Now alternatves Job C and Job E are domnated by Job B (and Job E also by Job A). Thus, any combnaton of the weghts satsfyng the gven constrants cannot gve Job C or Job E a better overall value than Job B has. Fgure 7. Overall value ntervals and domnance relatons. We can contnue our analyss by specfyng the gven nformaton to get more accurate results. We can, for example, defne subclasses for verbal descrptons. As alternatves Job C and Job E are domnated, the new nformaton s only needed for the classes concernng Jobs A, B and D. For example, Jobs A and D both have a moderate flexblty (ratng nterval [0.5, 0.7]) and great enjoyment (ratng nterval [0.8, 1.0]). By examnng the 31
stuaton more closely, the DM could end up concludng that these alternatves ndeed are of equal flexblty (e.g. both havng ratng 0.6) and of equal enjoyment (e.g. both havng ratng 1.0). In the lght of ths new nformaton, Job A domnates Job D (Fgure 8). Although the alternatves are stll equally preferred on these two attrbutes, ths new more precse nformaton has decreased the mprecson between these alternatves. Smlarly we can contnue by adjustng the nformaton for other attrbutes or alternatves untl the best alternatve s found. Wth WINPRE, ths knd of process s very easy to carry out, as t ndcates the domnated alternatves nstantaneously when makng changes n the attrbute weghts or n the ratngs of the alternatves. Fgure 8. Overall value ntervals and domnance relatons n the lght of more precse nformaton. Another approach s to try to elmnate only the obvously nferor alternatves. We mght not want to fnd out the best alternatve, but nstead an alternatve that performs satsfactorly n all crcumstances. For example, n ths case we could arbtrarly select ether Job A or B nstead of contnung wth the elctaton of more nformaton, as both of the alternatves perform reasonably well. 32
POTENTIAL EXTENSIONS OF THE METHOD Interpretng Intervals as Confdence Intervals One possble approach to nterpret ntervals s to treat them as confdence ntervals. Then, the DM accepts a certan possblty of makng an erroneous statement n preference assessment. Consequently, the DM also accepts a possblty of erroneous results. However, the confdence wth whch the overall values belong to the resultng ntervals s unknown, and ths cannot be determned wthout nformaton about the weght rato dstrbutons on the local ntervals. On the other hand, the calculaton wth the dstrbutons would lead to a stochastc smulaton approach. Ths has already been appled, for example, n the AHP and preference programmng (see e.g. Saaty and Vargas, 1987; Arbel and Vargas, 1993; Stam and Slva, 1997; Hahn, 2003). The modelng of the dstrbutons s beyond the scope of ths paper, and wll be a subject of future research. Usng an Interval as a Reference In prevous dscusson, we assumed that the reference attrbute s a pont estmate. However, techncally t would be also possble to use an nterval as a reference. Then, the constrants for the weght ratos would be derved as mn max j w, (9) w j max mn j where max (mn ) stands for the maxmum (mnmum) ponts gven to the attrbute, and and j are any (ether reference or non-reference) attrbutes. In other words, the constrants 33
are elcted from the extreme ratos of the gven pont ntervals, ncludng also the ones between the non-reference attrbutes. An appealng feature of ths approach s that t would gve explct constrants on every weght rato by usng only one more judgment,.e. an upper (or lower) bound for the reference attrbute. As an example, we have a case where attrbute 1 s the reference attrbute, and a pont nterval [50, 100] s assgned to t. Attrbutes 2 and 3 are then gven pont ntervals [50, 100] and [100, 150], respectvely. The correspondng feasble regon elcted from these has now explct constrants on all the weght ratos (Fgure 9). One should note that the weght ratos between attrbute pars 1 2 and 1 3 are here actually the same as n our prevous example (Fgure 2). However, as a result of expandng the reference attrbute nto an nterval, we have obtaned explct constrants also for the weght rato w 2 /w 3. Correspondngly, n the case of more than three attrbutes we would get explct constrants for all the possble weght ratos between the non-reference attrbutes. Fgure 9. The feasble regon obtaned wth an nterval as a reference. 34
However, conceptually the understandng of ths approach may become dffcult. Then, each nterval descrbes mprecson, or ambguty, related to the measurement scale used for that partcular attrbute, not mprecson about the weght rato between two attrbutes as n nterval SMART/SWING. Thus, the DM has to consder smultaneously both the wdths of the mprecson ntervals and the relatve mportances of the attrbutes. He/she cannot fx any attrbute to an exact value, but he/she has to bear n mnd that all the ponts can vary wthn the gven ntervals. In practce the constrants may become even wder wth ths approach as the DMs may stll stck nto a pontwse reference and assess the ntervals accordng to that. Thus, due to the ambguous nterpretaton of ntervals we cannot suggest the use of an nterval as a reference as such. However, possble practcal applcatons of usng ths varant of nterval SMART/SWING reman as a subject of further research. CONCLUSIONS In vew of the practcal applcablty of MCDA methods, the easness of the method s often very mportant (see e.g. Stewart, 1992). SMART and SWING are easy-to-use rato estmaton methods. In ths paper, we have generalzed them to allow the selecton of dfferent reference attrbutes and the use of ntervals to model mprecson. The am s to provde the DM a possblty to also model mprecson wthout makng the methods too complex to use. Consequently, these methods can be adapted to cover a wder range of decson makng stuatons. Techncally, the operatons are straghtforward, as these can be carred out smlarly as n the PAIRS method. However, the DM should realze that the selecton of the reference 35