Moral Artificial Intelligence (+ Computational Social Choice) Vincent

Moral Artificial Intelligence (+ Computational Social Choice) Vincent

Moral Artificial Intelligence (+ Computational Social Choice) Vincent Conitzer (Duke University) joint work with all the other people on this slide Walter SinnottArmstrong Rachel Freedman Jana Schaich Borg John P. Dickerson Rupert Freeman Yuan Deng

Max Kramer Markus Brill Yuqian Li Worries about AI - superintelligence writes Nick Bostrom (philosopher at Oxford) influences donates to Elon Musk is cofounded by

writes Max Tegmark Worries about AI - near term technological unemployment autonomous vehicles legal and other issues autonomous weapon systems Some popular articles Moral Decision Making Frameworks for Artificial Intelligence [AAAI17 blue sky track, CCC blue sky award winner] with:

Walter SinnottArmstrong Jana Schaich Borg Yuan Deng Max Kramer The value of generally applicable frameworks for AI research Decision and game theory Example: Markov Decision Processes Can we have a general framework for moral reasoning? Two main approaches Cf. top-down vs. bottom-up distinction [Wallach and Allen 2008]

Extend game theory to directly incorporate moral reasoning Generate data sets of human judgments, apply machine learning nature 1 gets King 1 gets Jack player 1 player 1 raise check raise check player 2

player 2 call fold call fold call yes fold call fold +a, -c, +i, +e, +o, +u: Y +a, -c, +i, -e, -o, -u: Y +a, -c, -i, -e, +o, -u: N +a, +c, +i, -e, +o, -u: N criminal?

1 1 1 -2 1 -1 criminal? yes no yes 2 address? no

+a, -c, +i, +e, +o, +u: Y -a, +c, -i, +e, -o, -u: N +a, -c, +i, -e, -o, -u: Y -a, -c, +i, +e, -o, -u: Y -a, +c, +i, -e, -o, -u: N -a, -c, +i, -e, -o, +u: Y +a, -c, -i, -e, +o, -u: N +a, +c, +i, -e, +o, -u: N -a, +c, -i, +e, -o, -u: -a, -c, +i, +e, -o, -u: -a, +c, +i, -e, -o, -u: -a, -c, +i, -e, -o, +u: N Y N Y no 1 -a, +c, -i, +e, -o, -u: N -a, +c, +i, -e, -o, -u: N

+a, -c, +i, +e, +o, +u: Y +a, -c, +i, -e, -o, -u: Y +a, -c, -i, -e, +o, -u: N +a, +c, +i, -e, +o, -u: N income? yes +a, -c, +i, +e, +o, +u: Y +a, -c, +i, -e, -o, -u: Y no +a, -c, -i, -e, +o, -u: N -a, -c, +i, +e, -o, -u: Y -a, -c, +i, -e, -o, +u: Y THE PARKING GAME (cf. the trust game [Berg et al. 1995]) wait move aside

3,0 steal spot pass 0,3 4,1 Letchford, C., Jain [2008] define a solution concept capturing this Extending representations? do nothing save own patient 0,-100,0 move train to other track save someone elses patient 0, 0, -100 More generally: how to capture framing? (Should we?) Roles? Relationships?

Scenarios You see a woman throwing a stapler at her colleague who is snoring during her talk. How morally wrong is the action depicted in this scenario? Not at all wrong (1) Slightly wrong (2) Somewhat wrong (3) Very wrong (4) Extremely wrong (5) [Clifford, Iyengar, Cabeza, and Sinnott-Armstrong, Moral foundations vignettes: A standardized stimulus database of scenarios based on moral foundations theory. Behavior Research Methods, 2015.]

Collaborative Filtering scenario 1 scenario 2 scenario 3 scenario 4 subject 1 very wrong - wrong not wrong subject 2 wrong

wrong - wrong subject 3 wrong very wrong - not wrong Bonnefon, Shariff, Rahwan, The social dilemma of autonomous vehicles. Science 2016 Noothigattu et al, A VotingBased System for Ethical Decision Making, AAAI18

Concerns with the ML approach What if we predict people will disagree? Social-choice theoretic questions [see also Rossi 2016, and Noothigattu et al. 2018 for moral machine data] This will at best result in current human-level moral decision making [raised by, e.g., Chaudhuri and Vardi 2014] though might perform better than any individual person because individuals errors are voted out How to generalize appropriately? Representation? Social-choice-theoretic approaches C. et al. [AAAI17]: [give] the AI some type of social-choice-theoretic aggregate of the moral values that we have inferred (for example, by letting our models of multiple peoples moral values vote over the relevant alternatives, or using only the moral values that are common to all of them). C. et al. [Trustworthy Algorithmic Decision Making Workshop17]: One possible solution is to let the models of multiple subjects vote over the possible choices. But exactly how should this be done? Whose preferences should count and what should be the voting rule used? How do we remove bias, prejudice, and confusion from the subjects judgments? These are novel problems in computational social choice. Noothigattu et al. [AAAI18]: I. Data collection: Ask human voters to compare pairs of alternatives (say a few dozen per voter). In the

autonomous vehicle domain, an alternative is determined by a vector of features such as the number of victims and their gender, age, health even species! II. Learning: Use the pairwise comparisons to learn a model of the preferences of each voter over all possible alternatives. III. Summarization: Combine the individual models into a single model, which approximately captures the collective preferences of all voters over all possible alternatives. IV. Aggregation: At runtime, when encountering an ethical dilemma involving a specific subset of alternatives, use the summary model to deduce the preferences of all voters over this particular subset, and apply a voting rule to aggregate these preferences into a collective decision. Adapting a Kidney Exchange Algorithm to Align with Human Values [AAAI18, honorable mention for outstanding student paper] with: Rachel Freedman Jana Schaich Borg Walter SinnottArmstrong

John P. Dickerson Kidney exchange [Roth, Snmez, and nver 2004] Kidney exchanges allow patients with willing but incompatible live donors to swap donors Algorithms developed in the AI community are used to find optimal matchings (starting with Abraham, Blum, and Sandholm [2007]) Another example Different profiles for our study MTurkers judgments Bradley-Terry model scores Effect of tiebreaking by profiles

Monotone transformations of the weights seem to make little difference Classes of pairs of blood types [Ashlagi and Roth 2014; Toulis and Parkes 2015] When generating sufficiently large random markets, patient-donor pairs situations can be categorized according to their blood types Underdemanded pairs contain a patient with blood type O, a donor with blood type AB, or both Overdemanded pairs contain a patient with blood type AB, a donor with blood type O, or both Self-demanded pairs contain a patient and donor with the same blood type Reciprocally demanded pairs contain one person with blood type A, and one person with blood type B Most of the effect is felt by underdemanded pairs

Crowdsourcing Societal Tradeoffs (AAMAS15 blue sky paper; AAAI16; ongoing work.) with Rupert Freeman, Markus Brill, Yuqian Li Example Decision Scenario Benevolent government would like to get old inefficient cars off the road But disposing of a car and building a new car has its own energy (and other) costs Which cars should the government aim to get off the road? even energy costs are not directly comparable (e.g., perhaps gasoline contributes to energy dependence, coal does not) The basic version of our problem is as bad as using x gallons

of gasoline producing 1 bag of landfill trash How to determine x? One Approach: Lets Vote! x should be 2 x should be 4 x should be 10 x What should the outcome be? Average? Median? 1 =x Assuming that preferences are single-peaked, selecting the median is strategy-proof and has other desirable

social choice-theoretic properties Consistency of tradeoffs Clearing forest [square meters] Consistency: z = xy y Using gasoline [gallons] z x Producing trash [bags] A paradox forest

forest forest 100 200 300 300 200 600 gasoline trash gasoline

trash gasoline trash 2 Just taking medians pairwise results in inconsistency 3 1 forest 200 300

gasoline trash 2 A first attempt at a rule satisfying consistency Let ta,b,i be voter is tradeoff between a and b Aggregate tradeoff t has score i a,b | ta,b - ta,b,i | forest forest forest 100 200 300 300

200 600 gasoline trash gasoline trash gasoline trash distance: 100 to v1 100 to v2 forest 200

gasoline 3 1 2 300 trash distance: 100 to v1 300 to v3 total distance: 602.5 (minimum) 3/2 distance: 1/2 to v ,1/2 to v , 3/2 to v 1 2 3

A nice property This rule agrees with the median when there are only two activities! x should be 2 x should be 4 x should be 10 x distance: 2+8=10 distance: 2+6=8 distance: 8+6=14 Not all is rosy, part 1 What if we change units? Say forest from m2 to cm2 (divide

by 10,000) forest forest forest 0.01 0.02 0.03 0.03 0.02 0.06 gasoline trash

gasoline trash gasoline trash 1 2 distance: (negligible) forest 0.015 0.03 gasoline

trash 2 distance: (negligible) distance: 1 to v1, 1 to v3 3 different from before! fails independence of other activities units Not all is rosy, part 2 Back to original units, but lets change some edges direction forest forest forest

1/100 1/200 1/300 1/300 1/200 1/600 gasoline trash gasoline trash gasoline

trash 2 distance: (negligible) 1 forest ? ? gasoline distance: (negligible) trash 2

distance: 1 to v1, 1 to v3 3 different from before! fails independence of other edges directions Summarizing Let ta,b,i be voter is tradeoff between a and b Aggregate tradeoff t has score i a,b | ta,b - ta,b,i | Upsides: Coincides with median for 2 activities Downsides: Dependence on choice of units: | ta,b - ta,b,i | | 2ta,b - 2ta,b,i | Dependence on direction of edges: | ta,b - ta,b,i | | 1/ta,b - 1/ta,b,i | We dont have a general algorithm A generalization

Let ta,b,i be voter is tradeoff between a and b Let f be a monotone increasing function say, f(x) = x2 Aggregate tradeoff t has score i a,b | f(ta,b) - f(ta,b,i) | Still coincides with median for 2 activities! Theorem: These are the only rules satisfying this property, agent separability, and edge separability ta,b 1 f(ta,b) 1 2 3 4 9

So whats a good f? Intuition: Is the difference between tradeoffs of 1 and 2 the same as between 1000 and 1001, or as between 1000 and 2000? So how about f(x)=log(x)? (Say, base e remember loga(x)=logb(x)/logb(a) ) ta,b ln(ta,b) 12 1000 2000 ln(1) ln(2) ln(1000) ln(2000)

0 0.69 6.91 7.60 On our example forest forest forest 100 200 300 300

200 600 gasoline trash gasoline trash gasoline trash 3 1 2

forest 200 400 gasoline trash 2 Properties Independence of units | log(1) - log(2) | = | log(1/2) | = | log(1000/2000) | = | log(1000) - log(2000) | More generally: | log(ax) - log(ay) | = | log(x) - log(y) | Independence of edge direction | log(x) - log(y) | = | log(1/y) - log(1/x) | = | log(1/x) - log(1/y) | Theorem. The logarithmic distance based rule is unique in satisfying independence of units.*

* Depending on the exact definition of independence of units, may need another minor condition about the function locally having bounded derivative. Consistency constraint becomes additive xy = z is equivalent to log(xy) = log(z) is equivalent to log(x) + log(y) = log(z) An additive variant I think basketball is 5 units more fun than football, which in turn is 10 units more fun than baseball basketball 5 15 football

baseball 10 Aggregation in the additive variant basketball 5 basketball 15 football baseball 10 -5 basketball

15 football baseball 10 football 20 Natural objective: minimize i a,b da,b,i where da,b,i = | ta,b - ta,b,i | is the distance between the aggregate difference ta,b and the subjective difference ta,b,i 40 baseball

30 basketball 5 25 football baseball 20 objective value 70 (optimal) A linear program for the additive variant qa: aggregate assessment of quality of activity a (were really interested in qa - qb = ta,b) da,b,i: how far is is preferred difference ta,b,i from aggregate qa - qb, i.e., da,b,i = |qa - qb - ta,b,i| minimize i a,b da,b,i subject to

for all a,b,i: da,b,i qa - qb - ta,b,i for all a,b,i: da,b,i ta,b,i - qa + qb (Can arbitrarily set one of the q variables to 0) Applying this to the logarithmic rule in the multiplicative variant forest forest forest 100 200 300 300 200 600

gasoline trash gasoline trash gasoline trash 3 1 2 Just take logarithms on the edges, solve the additive variant, and exponentiate back forest

forest forest 4.605 5.298 5.704 5.704 5.298 6.397 gasoline trash gasoline trash

gasoline trash 0.693 0 1.099 A simpler algorithm (hill climbing / greedy) Initialize qualities qa arbitrarily If some qa can be individually changed to improve the objective, do so WLOG, set qa to the median of the (#voters)*(#activities-1) implied votes on it

Continue until convergence (possibly to local optimum) Flow-based exact algorithm with: Hanrui Zhang Yu Cheng Decomposition Idea: Break down activities to relevant attributes sa e t u b

i r t con gasoline use to s t uni contributes b units to con trib ute sc uni ts t o global warming

energy dependence Another Paradox Agent 1 attribute 1 1 (global warming) activity A (gasoline) 1 3

3 2 1 11 2 (energy dependence) Agent 3 activity B 3 attribute 2 Agent 2 (trash) 1 1 2

1 aggregation on attribute level aggregation on activity level Other Issues Objective vs. subjective tradeoffs separate process? who determines which is which? Who gets to vote? how to bring expert knowledge to bear? incentives to participate Global vs. local tradeoffs Relevant Topics social choice theory voting judgment aggregation game theory mechanism design

prediction markets peer prediction preference elicitation ... different entities (e.g., countries) may wish to reach their tradeoffs independently only care about opinions of neighbors in my social network Thank you for your attention! Why Do We Care? Inconsistent tradeoffs can result in inefficiency Agents optimizing their utility functions individually leads to solutions that are Pareto inefficient Pigovian taxes: pay the cost your activity imposes on society (the externality of your activity) If we decided using 1 gallon of gasoline came at a cost of $x to society, we could charge

a tax of $x on each gallon But where would we get x? Arthur Cecil Pigou Inconsistent tradeoffs can result in inefficiency Agent 1: 1 gallon = 3 bags = -1 util I.e., agent 1 feels she should be willing to sacrifice up to1 util to reduce trash by 3, but no more Agent 2: 1.5 gallons = 1.5 bags = -1 util Agent 3: 3 gallons = 1 bag = -1 util Cost of reducing gasoline by x is x2 utils for each agent Cost of reducing trash by y is y2 for each agent Optimal solutions for the individual agents: Agent 1 will reduce by 1/2 and 1/6 Agent 2 will reduce by 1/3 and 1/3 Agent 3 will reduce by 1/6 and 1/2

But if agents 1 and 3 each reduce everything by 1/3, the total reductions are the same, and their costs are 2/9 rather than 1/4 + 1/36 which is clearly higher. Could then reduce slightly more to make everyone happier. Single-peaked preferences Definition: Let agent as most-preferred value be pa. Let p and p satisfy: - p p pa, or pa p p The agents preferences are single-peaked if the agent always weakly prefers p to p p p pa Perhaps more reasonable x should be

between 0 and 4 x should be between 2 and 6 x should be between 9 and 11 x E.g., due to missing information or plain uncertainty 1 =x How to aggregate these interval votes? [Farfel & Conitzer 2011] Median interval mechanism Construct a consensus interval from the median lower bound and the median upper bound 1

[ 1 [ 1 ] 2 [ 1 1 1 ] [ ] 1 ] 1

[ 2 11 Strategy-proof if preferences are single-peaked over intervals ] [] Single-peaked preferences over intervals Definition: Let agent as most-preferred value interval be Pa = [la, ua]. Let S = [l, u] and S = [l, u] be any two value intervals satisfying the following constraints: - Either l l la, or la l l - Either u u ua, or ua u u The agents preferences over intervals are singlepeaked if the agent always weakly prefers S to S [

[ [ ] ] ] l l la u u ua

Recently Viewed Presentations

  • Slide 1

    Slide 1

    You must read all of the publications of the Shepherd's Rod Message, from Shepherd's Rod Volume 1 to 10 Symbolic Code No. 2, the last publication by the prophet, both inclusive, if you have not read all of the SROD...
  • Telah hampir 40 tahun sejak satelit pertama di

    Telah hampir 40 tahun sejak satelit pertama di

    Telah hampir 40 tahun sejak satelit pertama di dunia diluncurkan, sejak saat itu pula berbagai aplikasi satelit dikembangkan. Dan sejak tahun 1964, hampir semua satelit komunikasi berada pada posisi Geostasionary Earth Orbit (GEO). Posisi GEO ini kira-kira berada pada ketinggian...
  • Présentation PowerPoint

    Présentation PowerPoint

    Support for all. Sharing information about job opportunities. Organizing awareness raising workshops and debates which include employers and disabled job seekers
  • Earth Changes - Laurel County

    Earth Changes - Laurel County

    Earth Changes Vocabulary landforms weathering erosion landslide volcano Earth changes can be… slow or rapid Some slow Earth changes are weathering and erosion. Weather can be physical or chemical and is caused by temperature changes, water, ice, chemicals, and other...
  • PowerPoint Presentation

    PowerPoint Presentation

    Customer logs on UniPrint portal. Customer submits requests for quotation . UniPrint sources prices from all preferred suppliers. UniPrint nominates printer based on cost & capability. UniPrint email quotation to customer
  • Beyond Exascale Computing: A Public -Private Partnership

    Beyond Exascale Computing: A Public -Private Partnership

    Power (watts) # Cores. Well at least we've got Moore's Law (lithography scaling) But in 2025 we won't have that any more, and our current play of doubling processor cores that gets us to exascale is no longer going to...
  • Chapter 7

    Chapter 7

    Forum Sentry is the industry's only FIPS 140-2 and NIAP NDPP certified XML Gateway for enabling secure connectivity between users, applications and the cloud. With the industry's most comprehensive built-in support for data formats and communication protocols, Forum Sentry enables...
  • Entraide Universitaire Mondiale Du Canada (Wusc/Eumc ...

    Entraide Universitaire Mondiale Du Canada (Wusc/Eumc ...

    ENTRAIDE UNIVERSITAIRE MONDIALE DU CANADA (WUSC/EUMC) Introduction à l'EUMC. NOTRE VISION. L'EUMC travaille à créer un monde dans lequel tous les jeunes peuvent s'épanouir dans des environnements sûrs, sécuritaires et favorables; où ils peuvent apprendre, travailler et jouer un ...